From redhatrises at gmail.com Tue Dec 1 02:31:06 2015 From: redhatrises at gmail.com (Gabe Alford) Date: Mon, 30 Nov 2015 19:31:06 -0700 Subject: [Freeipa-devel] [PATCH 0065] Message-ID: Hello, Patch fix for the following tickets: https://fedorahosted.org/freeipa/ticket/5022 https://fedorahosted.org/freeipa/ticket/5320 Thanks, Gabe -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rga-0064-ipa-replica-install-prints-incorrect-error-message-w.patch Type: text/x-patch Size: 2592 bytes Desc: not available URL: From jcholast at redhat.com Tue Dec 1 06:58:30 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 1 Dec 2015 07:58:30 +0100 Subject: [Freeipa-devel] [PATCH 0106] perform IPA client uninstallation as a last step of server uninstall In-Reply-To: <565C94F6.1060704@redhat.com> References: <5658745A.3060701@redhat.com> <565C94F6.1060704@redhat.com> Message-ID: <565D4516.7050002@redhat.com> On 30.11.2015 19:27, Martin Basti wrote: > > > On 27.11.2015 16:18, Martin Babinsky wrote: >> This patch fixes https://fedorahosted.org/freeipa/ticket/5410 >> >> >> > Functional ACK > Martin^2 Is this waiting for my LGTM? LGTM. -- Jan Cholasta From jcholast at redhat.com Tue Dec 1 07:19:59 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 1 Dec 2015 08:19:59 +0100 Subject: [Freeipa-devel] [PATCH 0385] replicainstall: Add possiblity to install client in one In-Reply-To: <1448907476.3747.89.camel@redhat.com> References: <5652FD85.3050105@redhat.com> <5653343B.6020200@redhat.com> <56547AB1.5030405@redhat.com> <56556AC2.1090302@redhat.com> <5656FC28.6000009@redhat.com> <56570AEF.7040607@redhat.com> <565C0806.5090103@redhat.com> <565C3236.5010703@redhat.com> <1448907476.3747.89.camel@redhat.com> Message-ID: <565D4A1F.30907@redhat.com> On 30.11.2015 19:17, Simo Sorce wrote: > On Mon, 2015-11-30 at 12:25 +0100, Tomas Babej wrote: >> + # Perform only if we have the necessary options >> + if not any([installer.admin_password, installer.keytab]): >> + sys.exit("IPA client is not configured on this system.\n" >> + "You must use a replica file or join the system " >> + "either by using by running 'ipa-client-install'. " >> + "Alternatively, you may specify enrollment related options " >> + "directly, see man ipa-replica-install.") >> + > > There is a typo "either by using by " > > Also this seem to be run in promote_check, so you should not mention > replica files, as promotion can only be run at domain level 1 where > replica files cannot be used. One more thing from me: admin password should be passed to ipa-client-install through stdin. Apply the following changes (tested and working) to make it so: args.extend(["--hostname", installer.host_name]) if installer.admin_password: - args.extend(["--password", installer.admin_password]) args.extend(["--principal", installer.principal or "admin"]) if installer.keytab: args.extend(["--keytab", installer.keytab]) @@ -792,7 +791,13 @@ def ensure_enrolled(installer): args.append("--no-sshd") if installer.mkhomedir: args.append("--mkhomedir") - ipautil.run(args) + + if installer.admin_password: + stdin = installer.admin_password + else: + stdin = None + + ipautil.run(args, stdin=stdin) except Exception as e: sys.exit("Configuration of client side components failed!\n" "ipa-client-install returned: " + str(e)) -- Jan Cholasta From jcholast at redhat.com Tue Dec 1 07:52:33 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 1 Dec 2015 08:52:33 +0100 Subject: [Freeipa-devel] [PATCH 0350] raise time limit for ldapsearch in upgrade In-Reply-To: <565C4BE7.5090201@redhat.com> References: <564C6F7E.1060803@redhat.com> <564C7529.8090909@redhat.com> <564C7A9B.6060107@redhat.com> <564C7C26.5000107@redhat.com> <564C7C74.3030607@redhat.com> <564DBBD5.6040805@redhat.com> <564DCA0E.40304@redhat.com> <564DCAFB.3050602@redhat.com> <564ED304.4020000@redhat.com> <56542AC5.4020804@redhat.com> <565BF86E.5050409@redhat.com> <565C4BE7.5090201@redhat.com> Message-ID: <565D51C1.3010508@redhat.com> On 30.11.2015 14:15, Martin Basti wrote: > > > On 30.11.2015 08:19, Jan Cholasta wrote: >> On 24.11.2015 10:15, Martin Basti wrote: >>> >>> >>> On 20.11.2015 09:00, Jan Cholasta wrote: >>>> On 19.11.2015 14:13, Jan Cholasta wrote: >>>>> On 19.11.2015 14:09, Martin Babinsky wrote: >>>>>> On 11/19/2015 01:08 PM, Martin Basti wrote: >>>>>>> >>>>>>> >>>>>>> On 18.11.2015 14:26, Martin Basti wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 18.11.2015 14:24, Martin Kosek wrote: >>>>>>>>> On 11/18/2015 02:18 PM, Martin Basti wrote: >>>>>>>>>> >>>>>>>>>> On 18.11.2015 13:55, Martin Kosek wrote: >>>>>>>>>>> On 11/18/2015 01:30 PM, Martin Basti wrote: >>>>>>>>>>>> + DEFAULT_TIME_LIMIT = -1.0 >>>>>>>>>>>> + DEFAULT_SIZE_LIMIT = 0 >>>>>>>>>>> ... >>>>>>>>>>>> if time_limit is None or time_limit == 0: >>>>>>>>>>>> - time_limit = -1.0 >>>>>>>>>>>> + if self.time_limit is not None and self.time_limit >>>>>>>>>>>> != 0: >>>>>>>>>>>> + time_limit = self.time_limit >>>>>>>>>>>> + else: >>>>>>>>>>>> + time_limit = self.DEFAULT_TIME_LIMIT >>>>>>>>>>>> + >>>>>>>>>>> I wonder why is the -1 default time limit a float number, I >>>>>>>>>>> would >>>>>>>>>>> expect that >>>>>>>>>>> some trouble may emerge out of it. Maybe Rob knows? >>>>>>>>>>> >>>>>>>>>> Python LDAP allows to have smaller granularity than seconds >>>>>>>>>> and it >>>>>>>>>> internally >>>>>>>>>> convert time limit values to float. >>>>>>>>> Hm, ok. I was just curious since the value we expose in API is Int >>>>>>>>> and the >>>>>>>>> default does not use the smaller granularity, so I was thinking >>>>>>>>> "Why >>>>>>>>> bother?". >>>>>>>>> If it is fixed in your patch, good. If not, good also, no need to >>>>>>>>> bikeshed here >>>>>>>>> I suppose. >>>>>>>> I just keep the original values there, I do not want to touch it >>>>>>>> with >>>>>>>> this patch >>>>>>>> >>>>>>> Updated patch attached, the original one has wrong default limit for >>>>>>> ldap2 >>>>>>> >>>>>>> >>>>>> ACK >>>>> >>>>> Hold your horses. >>>> >>>> 1) The upgrade_ldapsearch_limit option is not very useful, either add >>>> generic search_time_limit and search_records_limit options, or don't >>>> add any options at all (which, incidentally, is exactly what the >>>> ticket asks for). >>>> >>>> 2) The limits are not passed to the connection using __init__() >>>> anywhere, so there is no need to have arguments for them. >>>> >>>> 3) Rather than adding layers of if statements everywhere, implement >>>> the time_limit and size_limit attributes as properties. >>>> >>>> 4) Use config_entry.single_value when setting defaults in >>>> ldap2.get_ipa_config(). >>>> >>>> BTW, add comments only when it's not obvious what the code does - a >>>> comment saying "set defaults" above two .setdefault() calls is pretty >>>> redundant. >>>> >>> Updated patch attached >> >> In ldap2.create_connection(), you should set self.time_limit only if >> time_limit is not None, ditto for size limit, instead of handling None >> in the properties' setters. >> >> You don't need to handle the limits in ldap2.find_entries() anymore, >> since it is handled by LDAPClient.find_entries(), so you can remove >> the related code completely from ldap2.find_entries(). >> > Updated patch attached. The defaults should be set in LDAPClient class attributes rather than in __init__(). Besides that ACK. To speed things up I have amended your patch myself, see attachment. Pushed to master: 2a1a3c498a71e85193af76a25333ebe9011e6b2a -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0350.5-Upgrade-increase-time-limit-for-upgrades.patch Type: text/x-patch Size: 7602 bytes Desc: not available URL: From jcholast at redhat.com Tue Dec 1 08:04:49 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 1 Dec 2015 09:04:49 +0100 Subject: [Freeipa-devel] [PATCH] 938 rename topology suffixes to "domain" and "ca" In-Reply-To: <565C35ED.6090701@redhat.com> References: <565C35ED.6090701@redhat.com> Message-ID: <565D54A1.1050509@redhat.com> On 30.11.2015 12:41, Petr Vobornik wrote: > see > https://www.redhat.com/archives/freeipa-devel/2015-November/msg00485.html LGTM, but I would s/_SUFFIX/_SUFFIX_NAME/. -- Jan Cholasta From abokovoy at redhat.com Tue Dec 1 08:07:32 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 1 Dec 2015 10:07:32 +0200 Subject: [Freeipa-devel] [PATCH 560] Allow to set allowed krb authz data type per user In-Reply-To: <1448473396.17462.4.camel@redhat.com> References: <1448406570.29102.26.camel@redhat.com> <56555E8C.5020608@redhat.com> <1448473396.17462.4.camel@redhat.com> Message-ID: <20151201080732.GI9605@redhat.com> On Wed, 25 Nov 2015, Simo Sorce wrote: >On Wed, 2015-11-25 at 08:09 +0100, Jan Cholasta wrote: >> On 25.11.2015 00:09, Simo Sorce wrote: >> > This patch is untested and mostly an RFC. >> > >> > I think it is all we need to allow to specify authz data types per user >> > and by setting the attribute to NONE preventing a user from getting >> > MS-PAC data in their ticket. >> > >> > Alexander you changed quite a bit the code around here so I'd like to >> > know if you think the change I made in the KDC will cause any issue with >> > the special PACs we generate for master's principals. As far as I can >> > tell it shouldn't. >> > >> > Any opinion is welcome. >> >> Before your change, the server entry was checked for AS requests, now >> only the client entry is checked for AS requests. I'm not very familiar >> with ipa-kdb, but shouldn't the server entry still be checked as a >> fallback when there is no authorization data in the client entry? > >This is partly why I CCed Alexander, the way the get function works is >that it will get policy on the entry itself and if nothing is there it >will try with the global policy, so in both cases the global policy is >sourced as fallback. > >For AS requests though you are generally asking for a TGT so the >"server" is the krbtgt entry that has no policy. It is through though >that a client *can* ask for a ticket directly via an AS request, that is >uncommon and it is unclear to me what we should do in that case if >client and server have incompatible options. > >Well this is why it is a RFC after all :) Can we source global policy for the direct AS request as well? >> The attribute is exposed in the service plugin, shouldn't it be exposed >> in the user plugin as well? > >I didn't do it on purpose yet but eventually we may want to expose it, >indeed. The reason I didn't is that we may want to use something like >CoS to populate the attribute based on group membership and I am not >sure we want to expose it per user, up top debate. I don't want to expose it in the config too. -- / Alexander Bokovoy From abokovoy at redhat.com Tue Dec 1 08:11:45 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 1 Dec 2015 10:11:45 +0200 Subject: [Freeipa-devel] [PATCH 556-557] Add option to disable setkeytab extended operations In-Reply-To: <1448908709.3747.93.camel@redhat.com> References: <1448394152.29102.10.camel@redhat.com> <1448395038.29102.11.camel@redhat.com> <1448399848.29102.13.camel@redhat.com> <56555AC7.2000908@redhat.com> <5655BF5B.6020405@redhat.com> <1448462853.29102.44.camel@redhat.com> <1448908709.3747.93.camel@redhat.com> Message-ID: <20151201081145.GJ9605@redhat.com> On Mon, 30 Nov 2015, Simo Sorce wrote: >On Wed, 2015-11-25 at 09:47 -0500, Simo Sorce wrote: >> On Wed, 2015-11-25 at 09:02 -0500, Rob Crittenden wrote: >> > Jan Cholasta wrote: >> > > On 24.11.2015 22:17, Simo Sorce wrote: >> > >> On Tue, 2015-11-24 at 14:57 -0500, Simo Sorce wrote: >> > >>> On Tue, 2015-11-24 at 14:42 -0500, Simo Sorce wrote: >> > >>>> Since some time we use the getkeytab operation to fetch keytabs on >> > >>>> newer >> > >>>> clients. According to bug #232 setkeytab can be used to circumvent >> > >>>> password quality controls so it needs to be slowly retired. >> > >>>> >> > >>>> The attached patches implement #5485 in 2 parts. >> > >>>> >> > >>>> The first introduces the option DisableSetKeytab which globally >> > >>>> disables >> > >>>> the setkeytab extended operation. This is set to false by default for >> > >>>> backwards compatibility. >> > >>>> >> > >>>> The second introduces an option called DisableUserSetKeytab, which is >> > >>>> active by default in new installs (but not in upgraded ones), and only >> > >>>> disables the use of setkeytab for ipa suers, but not for >> > >>>> hosts/services. >> > >>>> This is because user's are the ones that may abuse the interface to >> > >>>> escape password policies and users also normally do not acquire >> > >>>> keytabs, >> > >>>> so it is a safe bet to disable just them by default in new installs. >> > >>>> >> > >>>> (Testing in progress) >> > >>> >> > >>> Tested and working as expected. >> > >> >> > >> I realized that adding options to ipaConfig require to add them in the >> > >> UI as well, attached patches add options in API.txt and config.py >> > >> Make now complain I should change API Major or Minor, but it is not >> > >> clear to me why given this are additional values and no real change or >> > >> new function is introduced. What's the recommendation ? >> > > >> > > When does make complain? It is supposed to complain only when API.txt >> > > does not match code. >> > > >> > > Anyway, we usually bump minor version even for backward compatible >> > > changes, see e.g. commit 9549a59. >> > > >> > >> > The point of API.txt (and the heavy client) was to save a round-trip. >> > Being able to pass in an invalid option would void that rule hence >> > having to update the API when new values are added. >> > >> > rob >> >> Ok added version change to the second patch (so we bump it only once >> given these are basically related changes. > >Bump, is this ok ? This patch is fine but please fix setkeytab use in ipa-sam before committing this patch. -- / Alexander Bokovoy From pspacek at redhat.com Tue Dec 1 08:15:49 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 1 Dec 2015 09:15:49 +0100 Subject: [Freeipa-devel] [PATCH] Add option to disable setkeytab extended operations In-Reply-To: <1448394152.29102.10.camel@redhat.com> References: <1448394152.29102.10.camel@redhat.com> Message-ID: <565D5735.7080405@redhat.com> On 24.11.2015 20:42, Simo Sorce wrote: > Since some time we use the getkeytab operation to fetch keytabs on newer > clients. According to bug #232 setkeytab can be used to circumvent > password quality controls so it needs to be slowly retired. > > The attached patches implement #5485 in 2 parts. > > The first introduces the option DisableSetKeytab which globally disables > the setkeytab extended operation. This is set to false by default for > backwards compatibility. > > The second introduces an option called DisableUserSetKeytab, which is > active by default in new installs (but not in upgraded ones), and only > disables the use of setkeytab for ipa suers, but not for hosts/services. > This is because user's are the ones that may abuse the interface to > escape password policies and users also normally do not acquire keytabs, > so it is a safe bet to disable just them by default in new installs. On a related note, how this works with plain kadmin & kpasswd protocols? Do I remember correctly that there is no way to download keytab without re-generating it? -- Petr^2 Spacek From mbasti at redhat.com Tue Dec 1 08:18:15 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 1 Dec 2015 09:18:15 +0100 Subject: [Freeipa-devel] [PATCH 0096] check whether replica exists before executing the domain level 1 deletion code In-Reply-To: <564C6E3E.5040606@redhat.com> References: <564C6E3E.5040606@redhat.com> Message-ID: <565D57C7.40105@redhat.com> On 18.11.2015 13:25, Martin Babinsky wrote: > Additional fix for https://fedorahosted.org/freeipa/ticket/5424 > > In current implementation the topology suffices are checked first and > after that the error about non-existent host is raised. This does not > make much sense to me, we should check for host existence before any > work is done. > > > ACK -------------- next part -------------- An HTML attachment was scrubbed... URL: From abokovoy at redhat.com Tue Dec 1 08:21:28 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 1 Dec 2015 10:21:28 +0200 Subject: [Freeipa-devel] [PATCH] Add option to disable setkeytab extended operations In-Reply-To: <565D5735.7080405@redhat.com> References: <1448394152.29102.10.camel@redhat.com> <565D5735.7080405@redhat.com> Message-ID: <20151201082128.GL9605@redhat.com> On Tue, 01 Dec 2015, Petr Spacek wrote: >On 24.11.2015 20:42, Simo Sorce wrote: >> Since some time we use the getkeytab operation to fetch keytabs on newer >> clients. According to bug #232 setkeytab can be used to circumvent >> password quality controls so it needs to be slowly retired. >> >> The attached patches implement #5485 in 2 parts. >> >> The first introduces the option DisableSetKeytab which globally disables >> the setkeytab extended operation. This is set to false by default for >> backwards compatibility. >> >> The second introduces an option called DisableUserSetKeytab, which is >> active by default in new installs (but not in upgraded ones), and only >> disables the use of setkeytab for ipa suers, but not for hosts/services. >> This is because user's are the ones that may abuse the interface to >> escape password policies and users also normally do not acquire keytabs, >> so it is a safe bet to disable just them by default in new installs. > >On a related note, how this works with plain kadmin & kpasswd protocols? It is unrelated. We don't support principal manipulation via kadmin protocol. >Do I remember correctly that there is no way to download keytab without >re-generating it? New keytab extended operation allows retrieval without re-generation since FreeIPA 4.0. See http://www.freeipa.org/page/V4/Keytab_Retrieval and http://www.freeipa.org/page/V4/Keytab_Retrieval_Management -- / Alexander Bokovoy From mbabinsk at redhat.com Tue Dec 1 08:22:28 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Tue, 1 Dec 2015 09:22:28 +0100 Subject: [Freeipa-devel] [PATCH] 927 topology: treat server suffix as multivalued attribute in API In-Reply-To: <565C2FCE.7030905@redhat.com> References: <564F34A5.60208@redhat.com> <56585FA4.3020205@redhat.com> <56586F56.8020407@redhat.com> <565C2FCE.7030905@redhat.com> Message-ID: <565D58C4.5060800@redhat.com> On 11/30/2015 12:15 PM, Jan Cholasta wrote: > On 27.11.2015 15:57, Petr Vobornik wrote: >> On 11/27/2015 02:50 PM, Martin Babinsky wrote: >>> On 11/20/2015 03:56 PM, Petr Vobornik wrote: >>>> >>>> >>>> >>> ACK >>> >> Pushed to master: c688954c27c219cb18aff968fc1f510afff93981 > > As we discussed offline, the server plugin should use topologysuffix > name instead of the actual suffix DN for iparepltopomanagedsuffix. The > attached patch fixes that. > > Also the correct plural form of "suffix" is "suffixes", not "suffices" > (), I have fixed that in an > additional patch. > > > ACK. -- Martin^3 Babinsky From mbabinsk at redhat.com Tue Dec 1 08:24:55 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Tue, 1 Dec 2015 09:24:55 +0100 Subject: [Freeipa-devel] [PATCH] 927 topology: treat server suffix as multivalued attribute in API In-Reply-To: <565C3532.7050405@redhat.com> References: <564F34A5.60208@redhat.com> <56585FA4.3020205@redhat.com> <56586F56.8020407@redhat.com> <565C2FCE.7030905@redhat.com> <565C3532.7050405@redhat.com> Message-ID: <565D5957.6010004@redhat.com> On 11/30/2015 12:38 PM, Petr Vobornik wrote: > On 11/30/2015 12:15 PM, Jan Cholasta wrote: >> On 27.11.2015 15:57, Petr Vobornik wrote: >>> On 11/27/2015 02:50 PM, Martin Babinsky wrote: >>>> On 11/20/2015 03:56 PM, Petr Vobornik wrote: >>>>> >>>>> >>>>> >>>> ACK >>>> >>> Pushed to master: c688954c27c219cb18aff968fc1f510afff93981 >> >> As we discussed offline, the server plugin should use topologysuffix >> name instead of the actual suffix DN for iparepltopomanagedsuffix. The >> attached patch fixes that. > > >> Also the correct plural form of "suffix" is "suffixes", not "suffices" >> (), I have fixed that in an >> additional patch. >> > > Wrt 519 - we may want to fix it on all places. See the attached wip > patch (coincidentally made on Friday). > > > There are some leftovers in translation string but I think that will be fixed before release. ACK. -- Martin^3 Babinsky From mbasti at redhat.com Tue Dec 1 08:31:37 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 1 Dec 2015 09:31:37 +0100 Subject: [Freeipa-devel] [PATCH] 927 topology: treat server suffix as multivalued attribute in API In-Reply-To: <565D58C4.5060800@redhat.com> References: <564F34A5.60208@redhat.com> <56585FA4.3020205@redhat.com> <56586F56.8020407@redhat.com> <565C2FCE.7030905@redhat.com> <565D58C4.5060800@redhat.com> Message-ID: <565D5AE9.4040905@redhat.com> On 01.12.2015 09:22, Martin Babinsky wrote: > On 11/30/2015 12:15 PM, Jan Cholasta wrote: >> On 27.11.2015 15:57, Petr Vobornik wrote: >>> On 11/27/2015 02:50 PM, Martin Babinsky wrote: >>>> On 11/20/2015 03:56 PM, Petr Vobornik wrote: >>>>> >>>>> >>>>> >>>> ACK >>>> >>> Pushed to master: c688954c27c219cb18aff968fc1f510afff93981 >> >> As we discussed offline, the server plugin should use topologysuffix >> name instead of the actual suffix DN for iparepltopomanagedsuffix. The >> attached patch fixes that. >> >> Also the correct plural form of "suffix" is "suffixes", not "suffices" >> (), I have fixed that in an >> additional patch. >> >> >> > > ACK. > Pushed to master: 4d24d8b26cd720f2ac24d05cbb00fd69ebd675a9 From pspacek at redhat.com Tue Dec 1 08:33:06 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 1 Dec 2015 09:33:06 +0100 Subject: [Freeipa-devel] [PATCH 0064-0065] ipa-dns-install offers IP addresses from resolv.conf as default forwarder In-Reply-To: <5656BC67.7070202@redhat.com> References: <56421416.8060104@redhat.com> <5642EA7B.1060505@redhat.com> <5642FBE4.8040501@redhat.com> <5642FD27.1050103@redhat.com> <5642FDF2.3060609@redhat.com> <5643505A.7000800@redhat.com> <5656BC67.7070202@redhat.com> Message-ID: <565D5B42.5030604@redhat.com> On 26.11.2015 09:01, Jan Cholasta wrote: > On 11.11.2015 15:27, Petr Spacek wrote: >> On 11.11.2015 09:36, Martin Babinsky wrote: >>> On 11/11/2015 09:32 AM, Jan Cholasta wrote: >>>> On 11.11.2015 09:27, Martin Babinsky wrote: >>>>> On 11/11/2015 08:12 AM, Jan Cholasta wrote: >>>>>> On 10.11.2015 16:58, Petr Spacek wrote: >>>>>>> Hello, >>>>>>> >>>>>>> Patch 64: >>>>>>> ipa-dns-install offer IP addresses from resolv.conf as default >>>>>>> forwarders >>>>>>> >>>>>>> In non-interactive more option --auto-forwarders can be used to do the >>>>>>> same. --forward option can be used to supply additional IP addresses. >>>>>>> >>>>>>> https://fedorahosted.org/freeipa/ticket/5438 >>>>>> >>>>>> IMO it's perverse to add option which effectively means "use default >>>>>> value" instead of actually using the value as default. This is >>>>>> inconsistent with every other option and I don't see what makes >>>>>> forwarders so special to require this. >>>>>> >>>>>> NACK unless you have a strong justification for this. >> >> Motivation: >> /etc/resolv.conf holds nearest DNS servers. On the other hand, you want to >> have backup forwarder which may not be local but could work even if local ones >> fail. >> >> Option --default-forwarders reads list of "local" servers from resolv.conf and >> --forwarder option allows you to add additional IP addresses to it. >> >> So your Ansible script can contain call like: >> ipa-server-install --setup-dns --default-forwarder >> --forwarder= >> and you do not need to worry about mapping sites to nearest servers etc. >> >>>>> Is it possible to use default_getter decorator to fetch defaults for the >>>>> 'forwarders' knob from the resolver if it is avaliable like so (warning: >>>>> untested and possibly wrong)? >>>> >>>> Yes, this is exactly how it should be used (although the exception >>>> handling could be better). >>>> >>> That was just a quick example off the top of my head without much thought >>> going into it. >>> >>> Anyway, when running in interactive mode the installer can inform the user >>> that he found these forwarders as defaults and prompt whether they shoud be >>> used. >> >> After discussion in person we decided to not use default_getter decorator >> because that would change current behavior in an unexpected way. >> >> Original option --auto-forwarders was renamed to --default-forwarders because >> it sounds nicer :-D > > Turns out I misunderstood the intent here and after another discussion in > person we decided to go with the --auto-forwarders option. > > ACK on the original patch. > > Petr, could you please rebase patch 65 on top of current master? Sure. I'm sorry for the delay! -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0064-2-ipa-dns-install-offer-IP-addresses-from-resolv.conf-.patch Type: text/x-patch Size: 7327 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0065-2-Remove-global-variable-dns_forwarders-from-ipaserver.patch Type: text/x-patch Size: 4745 bytes Desc: not available URL: From pspacek at redhat.com Tue Dec 1 08:34:42 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 1 Dec 2015 09:34:42 +0100 Subject: [Freeipa-devel] [PATCH] Add option to disable setkeytab extended operations In-Reply-To: <20151201082128.GL9605@redhat.com> References: <1448394152.29102.10.camel@redhat.com> <565D5735.7080405@redhat.com> <20151201082128.GL9605@redhat.com> Message-ID: <565D5BA2.7080608@redhat.com> On 1.12.2015 09:21, Alexander Bokovoy wrote: > On Tue, 01 Dec 2015, Petr Spacek wrote: >> On 24.11.2015 20:42, Simo Sorce wrote: >>> Since some time we use the getkeytab operation to fetch keytabs on newer >>> clients. According to bug #232 setkeytab can be used to circumvent >>> password quality controls so it needs to be slowly retired. >>> >>> The attached patches implement #5485 in 2 parts. >>> >>> The first introduces the option DisableSetKeytab which globally disables >>> the setkeytab extended operation. This is set to false by default for >>> backwards compatibility. >>> >>> The second introduces an option called DisableUserSetKeytab, which is >>> active by default in new installs (but not in upgraded ones), and only >>> disables the use of setkeytab for ipa suers, but not for hosts/services. >>> This is because user's are the ones that may abuse the interface to >>> escape password policies and users also normally do not acquire keytabs, >>> so it is a safe bet to disable just them by default in new installs. >> >> On a related note, how this works with plain kadmin & kpasswd protocols? > It is unrelated. We don't support principal manipulation via kadmin > protocol. Sure, I know that, but I'm trying to find out if we re-invented the wheel or if our wheel has some additional features which cannot be incorporated to the original wheel :-) >> Do I remember correctly that there is no way to download keytab without >> re-generating it? > New keytab extended operation allows retrieval without re-generation > since FreeIPA 4.0. See http://www.freeipa.org/page/V4/Keytab_Retrieval > and http://www.freeipa.org/page/V4/Keytab_Retrieval_Management Sure, I was talking about the original kadmin/kpasswd protocols. -- Petr^2 Spacek From mbasti at redhat.com Tue Dec 1 08:36:17 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 1 Dec 2015 09:36:17 +0100 Subject: [Freeipa-devel] [PATCH] 927 topology: treat server suffix as multivalued attribute in API In-Reply-To: <565D5957.6010004@redhat.com> References: <564F34A5.60208@redhat.com> <56585FA4.3020205@redhat.com> <56586F56.8020407@redhat.com> <565C2FCE.7030905@redhat.com> <565C3532.7050405@redhat.com> <565D5957.6010004@redhat.com> Message-ID: <565D5C01.7060106@redhat.com> On 01.12.2015 09:24, Martin Babinsky wrote: > On 11/30/2015 12:38 PM, Petr Vobornik wrote: >> On 11/30/2015 12:15 PM, Jan Cholasta wrote: >>> On 27.11.2015 15:57, Petr Vobornik wrote: >>>> On 11/27/2015 02:50 PM, Martin Babinsky wrote: >>>>> On 11/20/2015 03:56 PM, Petr Vobornik wrote: >>>>>> >>>>>> >>>>>> >>>>> ACK >>>>> >>>> Pushed to master: c688954c27c219cb18aff968fc1f510afff93981 >>> >>> As we discussed offline, the server plugin should use topologysuffix >>> name instead of the actual suffix DN for iparepltopomanagedsuffix. The >>> attached patch fixes that. >> >> >>> Also the correct plural form of "suffix" is "suffixes", not "suffices" >>> (), I have fixed that in an >>> additional patch. >>> >> >> Wrt 519 - we may want to fix it on all places. See the attached wip >> patch (coincidentally made on Friday). >> >> >> > There are some leftovers in translation string but I think that will > be fixed before release. ACK. > Rebased and pushed Pushed to master: 48548508676a28dae7c5a27b01fbe8bf288c1e8c From pspacek at redhat.com Tue Dec 1 08:37:50 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 1 Dec 2015 09:37:50 +0100 Subject: [Freeipa-devel] [PATCH] Removed duplicate domain name validation function In-Reply-To: <565C9CEA.80105@redhat.com> References: <5655DF48.4060302@redhat.com> <56586981.2010703@redhat.com> <565869D0.4060606@redhat.com> <56586D27.4030802@redhat.com> <56587181.1070004@redhat.com> <565C9CEA.80105@redhat.com> Message-ID: <565D5C5E.1080707@redhat.com> On 30.11.2015 20:00, Martin Basti wrote: > > > On 27.11.2015 16:06, Stanislav Laznicka wrote: >> Please, see the modified patch attached. >> >> Standa >> >> On 11/27/2015 03:48 PM, Martin Basti wrote: >>> >>> >>> On 27.11.2015 15:33, Petr Spacek wrote: >>>> On 27.11.2015 15:32, Martin Basti wrote: >>>>> >>>>> On 25.11.2015 17:18, Stanislav Laznicka wrote: >>>>>> There were two functions for the same purpose. Removed one. >>>>>> >>>>>> >>>>> Hello, >>>>> >>>>> I would like to have "log" param of is_host_resolvable as optional >>>> Is there an immediate need for the optional param? If not, I would not >>>> clutter >>>> the code. >>>> >>> So at least I would like to move log param as the last param, in case of >>> need it can be modified to optional parameter. >>> >>> Or log can be default as root_logger, IMO we us only root_logger >>> everywhere, but this need investigation. >>> >>> Martin >> > It works, but I would like to have Honza's opinion if ipalib/util.py is the > right place for the new method and if we really need to use exceptions. > > IMO is_record_resolvable() should return only True/False and then it can be > located in ipapython module Hmm, I do not see a reason why it should not throw an exception. IMHO the exception should contain the original error/result from resolver, so it is possible to print meaningful error message like "DNS query timed out" instead of "it does not work for some reason, trust us". -- Petr^2 Spacek From mbasti at redhat.com Tue Dec 1 08:40:29 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 1 Dec 2015 09:40:29 +0100 Subject: [Freeipa-devel] [PATCH 0106] perform IPA client uninstallation as a last step of server uninstall In-Reply-To: <565D4516.7050002@redhat.com> References: <5658745A.3060701@redhat.com> <565C94F6.1060704@redhat.com> <565D4516.7050002@redhat.com> Message-ID: <565D5CFD.4010609@redhat.com> On 01.12.2015 07:58, Jan Cholasta wrote: > On 30.11.2015 19:27, Martin Basti wrote: >> >> >> On 27.11.2015 16:18, Martin Babinsky wrote: >>> This patch fixes https://fedorahosted.org/freeipa/ticket/5410 >>> >>> >>> >> Functional ACK >> Martin^2 > > Is this waiting for my LGTM? > > LGTM. > Pushed to master: f6240f21fc9ae2ade54acedf98770e5f5e433d38 From mbasti at redhat.com Tue Dec 1 08:42:43 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 1 Dec 2015 09:42:43 +0100 Subject: [Freeipa-devel] [PATCH 0067] ipa-client-install: add support for Ed25519 SSH keys (RFC 7479) In-Reply-To: <565C988F.8060709@redhat.com> References: <56542657.9090304@redhat.com> <5654269F.80105@redhat.com> <565C988F.8060709@redhat.com> Message-ID: <565D5D83.1090809@redhat.com> On 30.11.2015 19:42, Martin Basti wrote: > > > On 24.11.2015 09:58, Petr Spacek wrote: >> On 24.11.2015 09:56, Petr Spacek wrote: >>> Hello, >>> >>> ipa-client-install: add support for Ed25519 SSH keys (RFC 7479) >>> >>> https://fedorahosted.org/freeipa/ticket/5471 >>> >> Once again ... >> >> >> > ACK > > Pushed to master: fa62480c73ccb860c8c8b4cd110b0782eb4883d5 -------------- next part -------------- An HTML attachment was scrubbed... URL: From abokovoy at redhat.com Tue Dec 1 08:47:20 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 1 Dec 2015 10:47:20 +0200 Subject: [Freeipa-devel] [PATCH] Add option to disable setkeytab extended operations In-Reply-To: <565D5BA2.7080608@redhat.com> References: <1448394152.29102.10.camel@redhat.com> <565D5735.7080405@redhat.com> <20151201082128.GL9605@redhat.com> <565D5BA2.7080608@redhat.com> Message-ID: <20151201084720.GM9605@redhat.com> On Tue, 01 Dec 2015, Petr Spacek wrote: >On 1.12.2015 09:21, Alexander Bokovoy wrote: >> On Tue, 01 Dec 2015, Petr Spacek wrote: >>> On 24.11.2015 20:42, Simo Sorce wrote: >>>> Since some time we use the getkeytab operation to fetch keytabs on newer >>>> clients. According to bug #232 setkeytab can be used to circumvent >>>> password quality controls so it needs to be slowly retired. >>>> >>>> The attached patches implement #5485 in 2 parts. >>>> >>>> The first introduces the option DisableSetKeytab which globally disables >>>> the setkeytab extended operation. This is set to false by default for >>>> backwards compatibility. >>>> >>>> The second introduces an option called DisableUserSetKeytab, which is >>>> active by default in new installs (but not in upgraded ones), and only >>>> disables the use of setkeytab for ipa suers, but not for hosts/services. >>>> This is because user's are the ones that may abuse the interface to >>>> escape password policies and users also normally do not acquire keytabs, >>>> so it is a safe bet to disable just them by default in new installs. >>> >>> On a related note, how this works with plain kadmin & kpasswd protocols? >> It is unrelated. We don't support principal manipulation via kadmin >> protocol. > >Sure, I know that, but I'm trying to find out if we re-invented the wheel or >if our wheel has some additional features which cannot be incorporated to the >original wheel :-) There is no need to incorporate something specific into kadmin protocol, the problem is with the fact that we don't have access controls within our KDC driver. It always connects to LDAP server over ldapi as root and thus maps to cn=Directory Manager which bypasses LDAP ACIs. As such, this means we'll need to have some access control added to KDC DAL driver before we can allow kadmin operations on principals. Adding those access controls is not an easy feat. -- / Alexander Bokovoy From jcholast at redhat.com Tue Dec 1 08:57:58 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 1 Dec 2015 09:57:58 +0100 Subject: [Freeipa-devel] [PATCH] Removed duplicate domain name validation function In-Reply-To: <565D5C5E.1080707@redhat.com> References: <5655DF48.4060302@redhat.com> <56586981.2010703@redhat.com> <565869D0.4060606@redhat.com> <56586D27.4030802@redhat.com> <56587181.1070004@redhat.com> <565C9CEA.80105@redhat.com> <565D5C5E.1080707@redhat.com> Message-ID: <565D6116.9070203@redhat.com> On 1.12.2015 09:37, Petr Spacek wrote: > On 30.11.2015 20:00, Martin Basti wrote: >> >> >> On 27.11.2015 16:06, Stanislav Laznicka wrote: >>> Please, see the modified patch attached. >>> >>> Standa >>> >>> On 11/27/2015 03:48 PM, Martin Basti wrote: >>>> >>>> >>>> On 27.11.2015 15:33, Petr Spacek wrote: >>>>> On 27.11.2015 15:32, Martin Basti wrote: >>>>>> >>>>>> On 25.11.2015 17:18, Stanislav Laznicka wrote: >>>>>>> There were two functions for the same purpose. Removed one. >>>>>>> >>>>>>> >>>>>> Hello, >>>>>> >>>>>> I would like to have "log" param of is_host_resolvable as optional >>>>> Is there an immediate need for the optional param? If not, I would not >>>>> clutter >>>>> the code. >>>>> >>>> So at least I would like to move log param as the last param, in case of >>>> need it can be modified to optional parameter. >>>> >>>> Or log can be default as root_logger, IMO we us only root_logger >>>> everywhere, but this need investigation. >>>> >>>> Martin >>> >> It works, but I would like to have Honza's opinion if ipalib/util.py is the >> right place for the new method and if we really need to use exceptions. ipalib/util.py is fine for the function, since it is not used anywhere in ipapython. I would rename the function to verify_host_resolvable() though, since the is_ prefix suggests the function returns a boolean value rather than raises an exception or not. >> >> IMO is_record_resolvable() should return only True/False and then it can be >> located in ipapython module > > Hmm, I do not see a reason why it should not throw an exception. IMHO the > exception should contain the original error/result from resolver, so it is > possible to print meaningful error message like "DNS query timed out" instead > of "it does not work for some reason, trust us". > -- Jan Cholasta From jcholast at redhat.com Tue Dec 1 09:19:51 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 1 Dec 2015 10:19:51 +0100 Subject: [Freeipa-devel] [PATCH 0064-0065] ipa-dns-install offers IP addresses from resolv.conf as default forwarder In-Reply-To: <565D5B42.5030604@redhat.com> References: <56421416.8060104@redhat.com> <5642EA7B.1060505@redhat.com> <5642FBE4.8040501@redhat.com> <5642FD27.1050103@redhat.com> <5642FDF2.3060609@redhat.com> <5643505A.7000800@redhat.com> <5656BC67.7070202@redhat.com> <565D5B42.5030604@redhat.com> Message-ID: <565D6637.8090307@redhat.com> On 1.12.2015 09:33, Petr Spacek wrote: > On 26.11.2015 09:01, Jan Cholasta wrote: >> On 11.11.2015 15:27, Petr Spacek wrote: >>> On 11.11.2015 09:36, Martin Babinsky wrote: >>>> On 11/11/2015 09:32 AM, Jan Cholasta wrote: >>>>> On 11.11.2015 09:27, Martin Babinsky wrote: >>>>>> On 11/11/2015 08:12 AM, Jan Cholasta wrote: >>>>>>> On 10.11.2015 16:58, Petr Spacek wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Patch 64: >>>>>>>> ipa-dns-install offer IP addresses from resolv.conf as default >>>>>>>> forwarders >>>>>>>> >>>>>>>> In non-interactive more option --auto-forwarders can be used to do the >>>>>>>> same. --forward option can be used to supply additional IP addresses. >>>>>>>> >>>>>>>> https://fedorahosted.org/freeipa/ticket/5438 >>>>>>> >>>>>>> IMO it's perverse to add option which effectively means "use default >>>>>>> value" instead of actually using the value as default. This is >>>>>>> inconsistent with every other option and I don't see what makes >>>>>>> forwarders so special to require this. >>>>>>> >>>>>>> NACK unless you have a strong justification for this. >>> >>> Motivation: >>> /etc/resolv.conf holds nearest DNS servers. On the other hand, you want to >>> have backup forwarder which may not be local but could work even if local ones >>> fail. >>> >>> Option --default-forwarders reads list of "local" servers from resolv.conf and >>> --forwarder option allows you to add additional IP addresses to it. >>> >>> So your Ansible script can contain call like: >>> ipa-server-install --setup-dns --default-forwarder >>> --forwarder= >>> and you do not need to worry about mapping sites to nearest servers etc. >>> >>>>>> Is it possible to use default_getter decorator to fetch defaults for the >>>>>> 'forwarders' knob from the resolver if it is avaliable like so (warning: >>>>>> untested and possibly wrong)? >>>>> >>>>> Yes, this is exactly how it should be used (although the exception >>>>> handling could be better). >>>>> >>>> That was just a quick example off the top of my head without much thought >>>> going into it. >>>> >>>> Anyway, when running in interactive mode the installer can inform the user >>>> that he found these forwarders as defaults and prompt whether they shoud be >>>> used. >>> >>> After discussion in person we decided to not use default_getter decorator >>> because that would change current behavior in an unexpected way. >>> >>> Original option --auto-forwarders was renamed to --default-forwarders because >>> it sounds nicer :-D >> >> Turns out I misunderstood the intent here and after another discussion in >> person we decided to go with the --auto-forwarders option. >> >> ACK on the original patch. >> >> Petr, could you please rebase patch 65 on top of current master? > > Sure. I'm sorry for the delay! Thank you, ACK. Pushed to master: 8f5f0d6edd25d2ca747c0477366fb392a26390f2 -- Jan Cholasta From pspacek at redhat.com Tue Dec 1 09:29:02 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 1 Dec 2015 10:29:02 +0100 Subject: [Freeipa-devel] [PATCH] Add option to disable setkeytab extended operations In-Reply-To: <20151201084720.GM9605@redhat.com> References: <1448394152.29102.10.camel@redhat.com> <565D5735.7080405@redhat.com> <20151201082128.GL9605@redhat.com> <565D5BA2.7080608@redhat.com> <20151201084720.GM9605@redhat.com> Message-ID: <565D685E.9080704@redhat.com> On 1.12.2015 09:47, Alexander Bokovoy wrote: > On Tue, 01 Dec 2015, Petr Spacek wrote: >> On 1.12.2015 09:21, Alexander Bokovoy wrote: >>> On Tue, 01 Dec 2015, Petr Spacek wrote: >>>> On 24.11.2015 20:42, Simo Sorce wrote: >>>>> Since some time we use the getkeytab operation to fetch keytabs on newer >>>>> clients. According to bug #232 setkeytab can be used to circumvent >>>>> password quality controls so it needs to be slowly retired. >>>>> >>>>> The attached patches implement #5485 in 2 parts. >>>>> >>>>> The first introduces the option DisableSetKeytab which globally disables >>>>> the setkeytab extended operation. This is set to false by default for >>>>> backwards compatibility. >>>>> >>>>> The second introduces an option called DisableUserSetKeytab, which is >>>>> active by default in new installs (but not in upgraded ones), and only >>>>> disables the use of setkeytab for ipa suers, but not for hosts/services. >>>>> This is because user's are the ones that may abuse the interface to >>>>> escape password policies and users also normally do not acquire keytabs, >>>>> so it is a safe bet to disable just them by default in new installs. >>>> >>>> On a related note, how this works with plain kadmin & kpasswd protocols? >>> It is unrelated. We don't support principal manipulation via kadmin >>> protocol. >> >> Sure, I know that, but I'm trying to find out if we re-invented the wheel or >> if our wheel has some additional features which cannot be incorporated to the >> original wheel :-) > There is no need to incorporate something specific into kadmin protocol, > the problem is with the fact that we don't have access controls within > our KDC driver. It always connects to LDAP server over ldapi as root and > thus maps to cn=Directory Manager which bypasses LDAP ACIs. As such, > this means we'll need to have some access control added to KDC DAL > driver before we can allow kadmin operations on principals. > > Adding those access controls is not an easy feat. Hmm, why is that? We could use LDAP Proxy control (RFC 4370) and let DS to evaluate ACIs as usual. The change in kadmin would be adding LDAP Proxy control with proper DN (i.e. mapping from principal name to DN) to the LDAP requests. -- Petr^2 Spacek From mkubik at redhat.com Tue Dec 1 09:31:14 2015 From: mkubik at redhat.com (=?UTF-8?Q?Milan_Kub=c3=adk?=) Date: Tue, 1 Dec 2015 10:31:14 +0100 Subject: [Freeipa-devel] [patch 0025] Separated Tracker implementations into standalone package In-Reply-To: <565C91B0.2070908@redhat.com> References: <564F31CE.5010402@redhat.com> <2089136880.12558785.1448049270339.JavaMail.zimbra@redhat.com> <5652DFAE.90906@redhat.com> <5658694F.3080404@redhat.com> <56586A7F.3070004@redhat.com> <56586B5D.2040507@redhat.com> <647877470.20906478.1448902246584.JavaMail.zimbra@redhat.com> <565C91B0.2070908@redhat.com> Message-ID: <565D68E2.5080008@redhat.com> On 11/30/2015 07:13 PM, Martin Basti wrote: > NACK > > 1) > With this patch I received this error in test_user_plugin.py > > E AssertionError: assert_deepequal: expected != got. > E 0106: user_status: Query status of "tuser1" > E expected = 1 > E got = 2 > E path = ('count',) > > I have just admin user on my system > > 2) > ______________________________________________________________________________ > ERROR collecting test_xmlrpc/test_stageuser_plugin.py > ______________________________________________________________________________ > test_xmlrpc/test_stageuser_plugin.py:28: in > from ipatests.test_xmlrpc.tracker.user_plugin import UserTracker > E ImportError: No module named tracker.user_plugin > ______________________________________________________________________________ > ERROR collecting test_xmlrpc/test_stageuser_plugin.py > ______________________________________________________________________________ > test_xmlrpc/test_stageuser_plugin.py:28: in > from ipatests.test_xmlrpc.tracker.user_plugin import UserTracker > E ImportError: No module named tracker.user_plugin > > 3) > ________________________________________________________________________________ > ERROR collecting test_xmlrpc/test_group_plugin.py > ________________________________________________________________________________ > test_xmlrpc/test_group_plugin.py:34: in > from ipatests.test_xmlrpc.tracker.user_plugin import UserTracker > E ImportError: No module named tracker.user_plugin > ________________________________________________________________________________ > ERROR collecting test_xmlrpc/test_group_plugin.py > ________________________________________________________________________________ > test_xmlrpc/test_group_plugin.py:34: in > from ipatests.test_xmlrpc.tracker.user_plugin import UserTracker > E ImportError: No module named tracker.user_plugin > > 4) > ________________________________________________________________________________ > ERROR collecting test_xmlrpc/test_host_plugin.py > _________________________________________________________________________________ > test_xmlrpc/test_host_plugin.py:42: in > from ipatests.test_xmlrpc.tracker.host_plugin import HostTracker > E ImportError: No module named tracker.host_plugin > ________________________________________________________________________________ > ERROR collecting test_xmlrpc/test_host_plugin.py > _________________________________________________________________________________ > test_xmlrpc/test_host_plugin.py:42: in > from ipatests.test_xmlrpc.tracker.host_plugin import HostTracker > E ImportError: No module named tracker.host_plugin > > 5) > ________________________________________________________________________________ > ERROR collecting test_xmlrpc/test_caacl_plugin.py > ________________________________________________________________________________ > test_xmlrpc/test_caacl_plugin.py:15: in > from ipatests.test_xmlrpc.test_certprofile_plugin import > default_profile > ../_pytest/assertion/rewrite.py:171: in load_module > py.builtin.exec_(co, mod.__dict__) > test_xmlrpc/test_certprofile_plugin.py:17: in > from ipatests.test_xmlrpc.tracker.certprofile_plugin import > CertprofileTracker > E ImportError: No module named tracker.certprofile_plugin > ________________________________________________________________________________ > ERROR collecting test_xmlrpc/test_caacl_plugin.py > ________________________________________________________________________________ > test_xmlrpc/test_caacl_plugin.py:15: in > from ipatests.test_xmlrpc.test_certprofile_plugin import > default_profile > ../_pytest/assertion/rewrite.py:171: in load_module > py.builtin.exec_(co, mod.__dict__) > test_xmlrpc/test_certprofile_plugin.py:17: in > from ipatests.test_xmlrpc.tracker.certprofile_plugin import > CertprofileTracker > E ImportError: No module named tracker.certprofile_plugin > > > .... > > N) > rpm -ql freeipa-tests | grep tracker > returns nothing, IMO the tracker directory in not in RPM > > > On 30.11.2015 17:50, Ale? Mare?ek wrote: >> Tested with today's master, ACK. >> - alich - >> >> ----- Original Message ----- >>> From: "Milan Kub?k" >>> To: freeipa-devel at redhat.com >>> Sent: Friday, November 27, 2015 3:40:29 PM >>> Subject: Re: [Freeipa-devel] [patch 0025] Separated Tracker >>> implementations into standalone package >>> >>> On 11/27/2015 03:36 PM, Milan Kub?k wrote: >>> >>> >>> >>> On 11/27/2015 03:31 PM, Milan Kub?k wrote: >>> >>> >>> On 11/23/2015 10:43 AM, Lenka Doudova wrote: >>> >>> >>> NACK - there's a "typo" in ..../tracker/user_plugin.py, line 17-18: >>> >>> def get_user_dn(cn): >>> >>> return DN(('cn', cn), api.env.container_user, api.env.basedn) >>> >>> >>> should be >>> >>> def get_user_dn(uid): >>> >>> return DN(('uid', uid), api.env.container_user, api.env.basedn) >>> >>> >>> Some tests may fail because of that. >>> Lenka >>> >>> >>> On 11/20/2015 08:54 PM, Ale? Mare?ek wrote: >>> >>> >>> Looks good. ACK. >>> >>> ----- Original Message ----- >>> >>> >>> From: "Milan Kub?k" >>> To: "freeipa-devel" >>> Cc: "Filip Skola" , "Ales Marecek" >>> >>> Sent: Friday, November 20, 2015 3:44:30 PM >>> Subject: [patch 0025] Separated Tracker implementations into standalone >>> package >>> >>> Fixes https://fedorahosted.org/freeipa/ticket/5467 >>> Patches attached. >>> >>> -- >>> Milan Kubik >>> >>> >>> >>> Fixed the function and moved it into different module. >>> Updated patches attached. >>> >>> >>> >>> Self nack, some imports missing >>> >>> -- >>> Milan Kubik >>> >>> >>> Patches updated. >>> >>> -- >>> Milan Kubik >>> >>> -- >>> Manage your subscription for the Freeipa-devel mailing list: >>> https://www.redhat.com/mailman/listinfo/freeipa-devel >>> Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code > My sincere apologies for this unspoken transgression on the packaging dieties. Updated patches attached. -- Milan Kubik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkubik-0025-4-Separated-Tracker-implementations-into-standalone-pa.patch Type: text/x-patch Size: 112817 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkubik-ipa42-0025-4-Separated-Tracker-implementations-into-standalone-pa.patch Type: text/x-patch Size: 112836 bytes Desc: not available URL: From abokovoy at redhat.com Tue Dec 1 09:53:48 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 1 Dec 2015 11:53:48 +0200 Subject: [Freeipa-devel] [PATCH] Add option to disable setkeytab extended operations In-Reply-To: <565D685E.9080704@redhat.com> References: <1448394152.29102.10.camel@redhat.com> <565D5735.7080405@redhat.com> <20151201082128.GL9605@redhat.com> <565D5BA2.7080608@redhat.com> <20151201084720.GM9605@redhat.com> <565D685E.9080704@redhat.com> Message-ID: <20151201095348.GO9605@redhat.com> On Tue, 01 Dec 2015, Petr Spacek wrote: >On 1.12.2015 09:47, Alexander Bokovoy wrote: >> On Tue, 01 Dec 2015, Petr Spacek wrote: >>> On 1.12.2015 09:21, Alexander Bokovoy wrote: >>>> On Tue, 01 Dec 2015, Petr Spacek wrote: >>>>> On 24.11.2015 20:42, Simo Sorce wrote: >>>>>> Since some time we use the getkeytab operation to fetch keytabs on newer >>>>>> clients. According to bug #232 setkeytab can be used to circumvent >>>>>> password quality controls so it needs to be slowly retired. >>>>>> >>>>>> The attached patches implement #5485 in 2 parts. >>>>>> >>>>>> The first introduces the option DisableSetKeytab which globally disables >>>>>> the setkeytab extended operation. This is set to false by default for >>>>>> backwards compatibility. >>>>>> >>>>>> The second introduces an option called DisableUserSetKeytab, which is >>>>>> active by default in new installs (but not in upgraded ones), and only >>>>>> disables the use of setkeytab for ipa suers, but not for hosts/services. >>>>>> This is because user's are the ones that may abuse the interface to >>>>>> escape password policies and users also normally do not acquire keytabs, >>>>>> so it is a safe bet to disable just them by default in new installs. >>>>> >>>>> On a related note, how this works with plain kadmin & kpasswd protocols? >>>> It is unrelated. We don't support principal manipulation via kadmin >>>> protocol. >>> >>> Sure, I know that, but I'm trying to find out if we re-invented the wheel or >>> if our wheel has some additional features which cannot be incorporated to the >>> original wheel :-) >> There is no need to incorporate something specific into kadmin protocol, >> the problem is with the fact that we don't have access controls within >> our KDC driver. It always connects to LDAP server over ldapi as root and >> thus maps to cn=Directory Manager which bypasses LDAP ACIs. As such, >> this means we'll need to have some access control added to KDC DAL >> driver before we can allow kadmin operations on principals. >> >> Adding those access controls is not an easy feat. > >Hmm, why is that? We could use LDAP Proxy control (RFC 4370) and let DS to >evaluate ACIs as usual. The change in kadmin would be adding LDAP Proxy >control with proper DN (i.e. mapping from principal name to DN) to the LDAP >requests. Not sure how does it help: $ ldapsearch -Dcn=Directory\ Manager -W -e \!authzid=dn:uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li -b uid=admin,cn=users,cn=accounts,dc=vda,dc=li userPassword Enter LDAP Password: # extended LDIF # # LDAPv3 # base with scope subtree # filter: (objectclass=*) # requesting: userPassword # # admin, users, accounts, vda.li dn: uid=admin,cn=users,cn=accounts,dc=vda,dc=li userPassword:: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX= = # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1 $ ldapsearch -Y GSSAPI -e \!authzid=dn:uid=admin,cn=users,cn=accounts,dc=vda,dc=li -b uid=admin,cn=users,cn=accounts,dc=vda,dc=li userPassword SASL/GSSAPI authentication started SASL username: abokovoy at VDA.LI SASL SSF: 56 SASL data security layer installed. # extended LDIF # # LDAPv3 # base with scope subtree # filter: (objectclass=*) # requesting: userPassword # # search result search: 4 result: 0 Success # numResponses: 1 As you can see, posing different authzid does not allow to get through ACLs unless you were already a directory manager. Which means the proxyauth control is not really useful here: [01/Dec/2015:10:51:22 +0100] conn=12466 op=0 BIND dn="cn=Directory Manager" method=128 version=3 [01/Dec/2015:10:51:22 +0100] conn=12466 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager" [01/Dec/2015:10:51:22 +0100] conn=12466 op=1 SRCH base="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" scope=2 filter="(objectClass=*)" attrs="userPassword" authzid="uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li" [01/Dec/2015:10:51:22 +0100] conn=12466 op=1 RESULT err=0 tag=101 nentries=1 etime=0 notes=U [01/Dec/2015:10:51:22 +0100] conn=12466 op=2 UNBIND and [01/Dec/2015:10:48:44 +0100] conn=12465 op=0 BIND dn="" method=sasl version=3 mech=GSSAPI [01/Dec/2015:10:48:44 +0100] conn=12465 op=0 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind in progress [01/Dec/2015:10:48:44 +0100] conn=12465 op=1 BIND dn="" method=sasl version=3 mech=GSSAPI [01/Dec/2015:10:48:44 +0100] conn=12465 op=1 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind in progress [01/Dec/2015:10:48:44 +0100] conn=12465 op=2 BIND dn="" method=sasl version=3 mech=GSSAPI [01/Dec/2015:10:48:44 +0100] conn=12465 op=2 RESULT err=0 tag=97 nentries=0 etime=0 dn="uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li" [01/Dec/2015:10:48:44 +0100] conn=12465 op=3 SRCH base="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" scope=2 filter="(objectClass=*)" attrs="userPassword" authzid="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" [01/Dec/2015:10:48:44 +0100] conn=12465 op=3 RESULT err=0 tag=101 nentries=0 etime=0 notes=U [01/Dec/2015:10:48:44 +0100] conn=12465 op=4 UNBIND -- / Alexander Bokovoy From abokovoy at redhat.com Tue Dec 1 11:00:57 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 1 Dec 2015 13:00:57 +0200 Subject: [Freeipa-devel] [PATCH] Add option to disable setkeytab extended operations In-Reply-To: <20151201095348.GO9605@redhat.com> References: <1448394152.29102.10.camel@redhat.com> <565D5735.7080405@redhat.com> <20151201082128.GL9605@redhat.com> <565D5BA2.7080608@redhat.com> <20151201084720.GM9605@redhat.com> <565D685E.9080704@redhat.com> <20151201095348.GO9605@redhat.com> Message-ID: <20151201110057.GP9605@redhat.com> On Tue, 01 Dec 2015, Alexander Bokovoy wrote: >On Tue, 01 Dec 2015, Petr Spacek wrote: >>On 1.12.2015 09:47, Alexander Bokovoy wrote: >>>On Tue, 01 Dec 2015, Petr Spacek wrote: >>>>On 1.12.2015 09:21, Alexander Bokovoy wrote: >>>>>On Tue, 01 Dec 2015, Petr Spacek wrote: >>>>>>On 24.11.2015 20:42, Simo Sorce wrote: >>>>>>>Since some time we use the getkeytab operation to fetch keytabs on newer >>>>>>>clients. According to bug #232 setkeytab can be used to circumvent >>>>>>>password quality controls so it needs to be slowly retired. >>>>>>> >>>>>>>The attached patches implement #5485 in 2 parts. >>>>>>> >>>>>>>The first introduces the option DisableSetKeytab which globally disables >>>>>>>the setkeytab extended operation. This is set to false by default for >>>>>>>backwards compatibility. >>>>>>> >>>>>>>The second introduces an option called DisableUserSetKeytab, which is >>>>>>>active by default in new installs (but not in upgraded ones), and only >>>>>>>disables the use of setkeytab for ipa suers, but not for hosts/services. >>>>>>>This is because user's are the ones that may abuse the interface to >>>>>>>escape password policies and users also normally do not acquire keytabs, >>>>>>>so it is a safe bet to disable just them by default in new installs. >>>>>> >>>>>>On a related note, how this works with plain kadmin & kpasswd protocols? >>>>>It is unrelated. We don't support principal manipulation via kadmin >>>>>protocol. >>>> >>>>Sure, I know that, but I'm trying to find out if we re-invented the wheel or >>>>if our wheel has some additional features which cannot be incorporated to the >>>>original wheel :-) >>>There is no need to incorporate something specific into kadmin protocol, >>>the problem is with the fact that we don't have access controls within >>>our KDC driver. It always connects to LDAP server over ldapi as root and >>>thus maps to cn=Directory Manager which bypasses LDAP ACIs. As such, >>>this means we'll need to have some access control added to KDC DAL >>>driver before we can allow kadmin operations on principals. >>> >>>Adding those access controls is not an easy feat. >> >>Hmm, why is that? We could use LDAP Proxy control (RFC 4370) and let DS to >>evaluate ACIs as usual. The change in kadmin would be adding LDAP Proxy >>control with proper DN (i.e. mapping from principal name to DN) to the LDAP >>requests. >Not sure how does it help: >$ ldapsearch -Dcn=Directory\ Manager -W -e \!authzid=dn:uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li -b uid=admin,cn=users,cn=accounts,dc=vda,dc=li userPassword >Enter LDAP Password: # extended LDIF ># ># LDAPv3 ># base with scope subtree ># filter: (objectclass=*) ># requesting: userPassword # > ># admin, users, accounts, vda.li >dn: uid=admin,cn=users,cn=accounts,dc=vda,dc=li >userPassword:: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX= >= > ># search result >search: 2 >result: 0 Success > ># numResponses: 2 ># numEntries: 1 >$ ldapsearch -Y GSSAPI -e \!authzid=dn:uid=admin,cn=users,cn=accounts,dc=vda,dc=li -b uid=admin,cn=users,cn=accounts,dc=vda,dc=li userPassword >SASL/GSSAPI authentication started >SASL username: abokovoy at VDA.LI >SASL SSF: 56 >SASL data security layer installed. ># extended LDIF ># ># LDAPv3 ># base with scope subtree ># filter: (objectclass=*) ># requesting: userPassword # > ># search result >search: 4 >result: 0 Success > ># numResponses: 1 > >As you can see, posing different authzid does not allow to get through >ACLs unless you were already a directory manager. Which means the >proxyauth control is not really useful here: > >[01/Dec/2015:10:51:22 +0100] conn=12466 op=0 BIND dn="cn=Directory Manager" method=128 version=3 >[01/Dec/2015:10:51:22 +0100] conn=12466 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager" >[01/Dec/2015:10:51:22 +0100] conn=12466 op=1 SRCH base="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" scope=2 filter="(objectClass=*)" attrs="userPassword" authzid="uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li" >[01/Dec/2015:10:51:22 +0100] conn=12466 op=1 RESULT err=0 tag=101 nentries=1 etime=0 notes=U >[01/Dec/2015:10:51:22 +0100] conn=12466 op=2 UNBIND > >and > >[01/Dec/2015:10:48:44 +0100] conn=12465 op=0 BIND dn="" method=sasl version=3 mech=GSSAPI >[01/Dec/2015:10:48:44 +0100] conn=12465 op=0 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind in progress >[01/Dec/2015:10:48:44 +0100] conn=12465 op=1 BIND dn="" method=sasl version=3 mech=GSSAPI >[01/Dec/2015:10:48:44 +0100] conn=12465 op=1 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind in progress >[01/Dec/2015:10:48:44 +0100] conn=12465 op=2 BIND dn="" method=sasl version=3 mech=GSSAPI >[01/Dec/2015:10:48:44 +0100] conn=12465 op=2 RESULT err=0 tag=97 nentries=0 etime=0 dn="uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li" >[01/Dec/2015:10:48:44 +0100] conn=12465 op=3 SRCH base="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" scope=2 filter="(objectClass=*)" attrs="userPassword" authzid="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" >[01/Dec/2015:10:48:44 +0100] conn=12465 op=3 RESULT err=0 tag=101 nentries=0 etime=0 notes=U >[01/Dec/2015:10:48:44 +0100] conn=12465 op=4 UNBIND > Ludwig confirmed that ACL plugin short-circuits DM access before proxy auth checked. I've filed a bug about it: https://fedorahosted.org/389/ticket/48366 -- / Alexander Bokovoy From pviktori at redhat.com Tue Dec 1 11:12:48 2015 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 1 Dec 2015 12:12:48 +0100 Subject: [Freeipa-devel] [PATCH] 0749 Package ipapython, ipalib, ipaplatform, ipatests for Python 3 In-Reply-To: <565C0FF3.8040406@redhat.com> References: <563C9810.1050604@redhat.com> <5656E487.8050703@redhat.com> <5658508D.1020105@redhat.com> <565C0FF3.8040406@redhat.com> Message-ID: <565D80B0.9000500@redhat.com> Hello, I noticed I didn't attach an updated patch last time, which probably affected the discussion here. Sorry for that; it's attached this time. We seem to have a disconenct here: there are two different strategies to do the packaging. Your idea of the port seems to be to have the py2 and py3 packages "in sync", so they're almost exact copies of each other. This keeps things consistent, but I'm not sure this specific kind of consistency is better than the alternative. My approach is to make new, better packages for Python 3. I treat this as a bit of a sandbox, a new chance to do things right. My patch tries to keep the existing Python 2 packages untouched where it's feasible, to minimize regressions. (I did add new Provides based on your review, though, so users can use the "consistent" names.) When the py3 packages are finalized, the changes can be applied to the py2 ones as well. (Or not, if it's decided it's a case of fixing things that ain't broke.) So, two paths to get to the same goal. I'm not saying yours is bad, but I fail to see how it's objectively better. If it is, could you explain why? Otherwise please take this patch as an incremental improvement with priorities that are a bit different than yours. On 11/30/2015 09:59 AM, Jan Cholasta wrote: > On 27.11.2015 13:46, Petr Viktorin wrote: >> On 11/26/2015 11:52 AM, Jan Cholasta wrote: >>> 1) The freeipa-common subpackage is not necessary: /etc/ipa/dnssec >>> should be owned by freeipa-server and everything else in /etc/ipa >>> currently owned by freeipa-python should be owned by freeipa-client. >> >> If the common files are in freeipa-server or freeipa-client, then the >> Python 3 packages would need to depend on those. I'd like to avoid that. > > I'm not sure I follow. The files do not belong to freeipa-python, since > they are used only by freeipa-server and freeipa-client. What Python 3 > packages would need to depend on them? python3-ipaserver and python3-ipaclient, eventually. Anyway, /etc/ipa/default.conf is used directly by ipapython.config, so at least that should be owned by a common package. And once there's a common package, I don't think there's a problem with it holding the other tiny files as well. Also, a lot of the other files are mentioned in ipaplatform.base.paths, which is part of the freeipa-python, so it seems fitting that that's what should require them, rather than the -client or -server packages. >>> This should be a separate patch. I have prepared one, see attachment. >>> >>> >>> 2) IMO this patch does too much, it should only add the Python 3 >>> equivalent of freeipa-python - python3-ipalib - but none of the other >>> packages/provides. >> >> My reasoning for the new packages: >> - python3-ipalib is the main point > > Right. > >> - freeipa-common contains files that both the py2 and py3 versions need > > See above. > >> - python-ipap11helper has compiled code: with this pulled out, >> python3-ipalib can be noarch > > This is not the goal here, but if you insist on doing it, do it for > Python 2 as well. It is definitely the goal of this patch to make the py3 packages as good as possible. That includes making them noarch. On the other hand improving the py2 packages is not a goal of this particular patch. >> - python3-ipatests is needed if we want to start testing the py3 >> packages in CI > > Right. > >> >> As for new provides, Fedora's Python packaging guidelines say: >> >> """ >> Using a fictional module named "example", the subpackage containing >> the python2 version must provide python2-example. This is of course >> always the case if the subpackage is named python2-example [...] >> If the subpackage has some other name then then Provides: python2-example >> must be added explicitly (but see the %python_provide macro below). >> >> The python3 subpackage must provide python3-example. However, as the >> naming guidelines mandate that the python3 subpackage be named >> python3-example, this will happen automatically. >> """ >> >> so I'm now adding Provides for the top-level modules. > > The goal of this work is to add support for Python 3, not to comply with > Fedora packaging guidelines. FreeIPA on Fedora uses its own spec file > anyway. The goal of this patch is to add new packages that support Python 3. Yes, the Fedora spec is different, but it's heavily based on the upstream one, and this is a good thing. I consider the Fedora guidelines the standard in Python RPM packaging. If IPA uses different packaging guidelines, can you point me to them? > Again, if you insist on doing this, do it for Python 2 as well. Improving the py2 packages would be great, but again it is not the goal of this patch. >>> 3) Speaking of freeipa-python, it should be renamed to python-ipalib, >>> for consistency. >>> >>> This should also be a separate patch, again see attachment. >> >> I think that would just be unnecessary churn. Python 3 gives us a chance >> for a fresh start, so I'm taking advantage of that, but renaming >> existing packages is a pain. > > Something similar was done in SSSD not long ago and according to Luk?? > (CCd), it was no pain. OK. But again, not the goal here. >> With the right Provides, "dnf install python-ipalib" (or python2-ipalib) >> will do the right thing, and I thing that's enough. > > I don't think that's good enough, since it creates an inconsistency. > >> >>> 4) There should be a python3-devel (and also other python3- equivalents >>> of python- packages) BuildRequires when Python 3 support is enabled. >> >> I did miss that somehow. Thanks for checking. >> I added python3-devel; the others are unnecessary because pylint is not >> currently run on the Python3 version. > > OK, I think that will do for now. Note that many of them are required > for makeapi and makeaci as well. > > When can we expect a Python 3 pylint patch which adds the remaining > BuildRequires? That's something we'll need to talk about in the future: do we want to run python2-pylint and python3-pylint on every build? Anyway, I'd like to see the Python3 tests for ipapython and ipalib run in CI first. -- Petr Viktorin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0749.2-Package-ipapython-ipalib-ipaplatform-ipatests-for-Py.patch Type: text/x-patch Size: 24563 bytes Desc: not available URL: From jcholast at redhat.com Tue Dec 1 11:19:26 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 1 Dec 2015 12:19:26 +0100 Subject: [Freeipa-devel] [PATCHES 509-514] replica promotion: use host credentials when setting up replication In-Reply-To: <1448290024.7892.15.camel@redhat.com> References: <564DE017.4020408@redhat.com> <1447951434.5132.27.camel@redhat.com> <564F4105.1040204@redhat.com> <1448038723.5132.59.camel@redhat.com> <5652C63E.5090206@redhat.com> <1448289289.7892.13.camel@redhat.com> <565324C4.1090007@redhat.com> <1448290024.7892.15.camel@redhat.com> Message-ID: <565D823E.5090600@redhat.com> On 23.11.2015 15:47, Simo Sorce wrote: > On Mon, 2015-11-23 at 15:37 +0100, Jan Cholasta wrote: >> >> Ad alternative is to add the host to ipaservers before the checks are >> done and remove it again if any of them fail. > > Too error prone, I am ok with the current way in your patches > until/unless I can think of a fail safe way. :-) Updated patches attached. Note that 520 should be applied between 509 and 510. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-509.1-aci-add-IPA-servers-host-group-ipaservers.patch Type: text/x-patch Size: 9835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-520-aci-replace-per-server-ACIs-with-ipaserver-based-ACI.patch Type: text/x-patch Size: 11571 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-510.1-aci-allow-members-of-ipaservers-to-set-up-replicatio.patch Type: text/x-patch Size: 5563 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-511.1-ipautil-use-file-in-a-temporary-dir-as-ccache-in-pri.patch Type: text/x-patch Size: 1392 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-512.1-replica-promotion-use-host-credentials-when-setting-.patch Type: text/x-patch Size: 4918 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-514.1-replica-promotion-automatically-add-the-local-host-t.patch Type: text/x-patch Size: 3141 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-521-custodia-do-not-modify-memberPrincipal-on-key-update.patch Type: text/x-patch Size: 970 bytes Desc: not available URL: From mbabinsk at redhat.com Tue Dec 1 11:21:24 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Tue, 1 Dec 2015 12:21:24 +0100 Subject: [Freeipa-devel] [PATCH 0107] fix 'iparepltopomanagedsuffix' attribute consumers Message-ID: <565D82B4.50005@redhat.com> This patch fixes a regression caused by recently pushed topologysuffix-related patches. -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0107-fix-iparepltopomanagedsuffix-attribute-consumers.patch Type: text/x-patch Size: 2297 bytes Desc: not available URL: From pviktori at redhat.com Tue Dec 1 11:26:04 2015 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 1 Dec 2015 12:26:04 +0100 Subject: [Freeipa-devel] [PATCH] 0748 Handle encoding for ipautil.run In-Reply-To: <565C01FD.7050906@redhat.com> References: <5638B8F9.6010703@redhat.com> <564D0FD9.4070505@redhat.com> <5652B5AC.2040508@redhat.com> <5652E174.50301@redhat.com> <56548E73.1090105@redhat.com> <565587AE.2070303@redhat.com> <5655CA08.8090701@redhat.com> <565C01FD.7050906@redhat.com> Message-ID: <565D83CC.8020802@redhat.com> On 11/30/2015 08:59 AM, Jan Cholasta wrote: > On 25.11.2015 15:47, Petr Viktorin wrote: >> On 11/25/2015 11:04 AM, Jan Cholasta wrote: >>> On 24.11.2015 17:21, Petr Viktorin wrote: >>>> On 11/23/2015 10:50 AM, Jan Cholasta wrote: >>>>> On 23.11.2015 07:43, Jan Cholasta wrote: >>>>>> On 19.11.2015 00:55, Petr Viktorin wrote: >>>>>>> On 11/03/2015 02:39 PM, Petr Viktorin wrote: >>>>>>>> Hello, >>>>>>>> Python 3's strings are Unicode, so data coming to or leaving a >>>>>>>> Python >>>>>>>> program needs to be decoded/encoded if it's to be handled as a >>>>>>>> string. >>>>>>>> One of the boundaries where encoding is necessary is external >>>>>>>> programs, >>>>>>>> specifically, ipautil.run. >>>>>>>> Unfortunately there's no one set of options that would work for all >>>>>>>> run() invocations, so I went through them all and specified the >>>>>>>> stdin/stdout/stderr encoding where necessary. I've also updated the >>>>>>>> call >>>>>>>> sites to make it clearer if the return values are used or not. >>>>>>>> If an encoding is not set, run() will accept/return bytes. (This >>>>>>>> is a >>>>>>>> fail-safe setting, since it can't raise errors, and using bytes >>>>>>>> where >>>>>>>> strings are expected generally fails loudly in py3.) >>>>>>>> >>>>>>>> Note that the changes are not effective under Python 2. >>>>>>> >>>>>>> ping, >>>>>>> Could someone look at this patch? >>>>>> >>>>>> Looking. >>>>> >>>>> 1) I think a better approach would be to use str for >>>>> stdin/stdout/stderr >>>>> by default in both Python 2 and 3, i.e. do nothing in Python 2 and >>>>> encode/decode using locale.getpreferredencoding() in Python 3. This is >>>>> consistent with sys.std{in,out,err} in both Python 2 and 3. Using >>>>> bytes >>>>> or different encoding should be opt-in. >>>>> >>>>> Note that different encoding should be used only if the LC_ALL or LANG >>>>> variables are overriden in the command's environment. >>>> >>>> That would assume the output of *everything* run via ipautil.run can be >>>> decoded using the locale encoding. Any stray invalid byte would make >>>> IPA >>>> crash, even in cases where we don't care about the output. IPA calls >>>> too >>>> many weird tools to assume they all output text. >>> >>> Such a stray invalid byte may bubble through our code and cause havoc >>> somewhere else, which is much harder to troubleshoot than a crash in >>> ipautil.run(), where you can see that it was a misbehaving command that >>> caused the crash and exactly what command it was. >> >> The invalid byte can't bubble through code, because if you don't specify >> an encoding you get bytes back. You'll get a crash when you try using it >> as a string. > > Invalid bytes *can* bubble through our code, into configuration files > and other external places. The fact that it has not caused much grief so > far suggests that it should be safe to use the locale encoding for most > commands. Yes, "suggests" for "most". I'm still wary of using the locale encoding for everything by default. > If we do subscribe to the statement that any command can output invalid > bytes at any time, I don't see a point in handling encoding in > ipautil.run() at all - just always return bytes and let the caller worry > about encoding and handling related errors. Right, that's actually the basic idea. However, the encoding is two lines of boilerplate code ("if six.PY3" and the actual encode). I've turned this into a convenience keyword argument to minimize the repetition. >> locale.getpreferredencoding() is not used consistently in all software, >> especially if it's not written in Python. For instance, wget won't >> magically re-encode the data it fetches for us. It's better to >> explicitly specify the encoding every time. > > I would say it is used consistently in most software used by IPA. The > wget example is obviously not relevant to this discussion, since it's > returning external data. > > IMO setting the encoding to 'ascii' without also setting LC_ALL=C in the > environment, as seen frequently in your patch, has even higher > probability of breaking something than using the locale encoding. The patch should use 'ascii' with programs that don't output localized messages: things like base64-encoded certs or the output of `ip`. If you see a particular call where that's not the case, it's a bug in the patch, please point it out. >>> If we don't care about output somewhere, we should not capture it there >>> at all. >> >> Then people need to remember to put "capture_output=True" everywhere >> (except that also disables logging of the output, so we'd need an >> additional option). > > capture_output=True is the default and capture_output=False should be > set where output is not needed. "capture_output=True" being the default would mean that it's still possible to leave "capture_output=False" out, but ignore the output. It would make the wrong usage easier to type than the right one, which is something to avoid. So I still prefer the encoding being explicit. > IMO the output should always be logged if possible, regardless of > capture_output value. File a bug for that? -- Petr Viktorin From mbabinsk at redhat.com Tue Dec 1 11:28:43 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Tue, 1 Dec 2015 12:28:43 +0100 Subject: [Freeipa-devel] [PATCH 0107] fix 'iparepltopomanagedsuffix' attribute consumers In-Reply-To: <565D82B4.50005@redhat.com> References: <565D82B4.50005@redhat.com> Message-ID: <565D846B.4060403@redhat.com> On 12/01/2015 12:21 PM, Martin Babinsky wrote: > This patch fixes a regression caused by recently pushed > topologysuffix-related patches. > > > self-NACK. -- Martin^3 Babinsky From mbabinsk at redhat.com Tue Dec 1 11:36:10 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Tue, 1 Dec 2015 12:36:10 +0100 Subject: [Freeipa-devel] [PATCH 0107] fix 'iparepltopomanagedsuffix' attribute consumers In-Reply-To: <565D846B.4060403@redhat.com> References: <565D82B4.50005@redhat.com> <565D846B.4060403@redhat.com> Message-ID: <565D862A.2010408@redhat.com> On 12/01/2015 12:28 PM, Martin Babinsky wrote: > On 12/01/2015 12:21 PM, Martin Babinsky wrote: >> This patch fixes a regression caused by recently pushed >> topologysuffix-related patches. >> >> >> > self-NACK. > This patch should actually work. -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0107.1-fix-iparepltopomanagedsuffix-attribute-consumers.patch Type: text/x-patch Size: 2630 bytes Desc: not available URL: From ofayans at redhat.com Tue Dec 1 11:58:47 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Tue, 1 Dec 2015 12:58:47 +0100 Subject: [Freeipa-devel] dns discovery broken? Message-ID: <565D8B77.8090702@redhat.com> Hi all, I am unable to enroll a client with the latest upstream ipa code I've successfully installed ipa-server with --setup-dns and --forwarder provided. I configured client's resolv.conf to use master's ip as the first dns server. But when I issued the ipa-client-install command, it fails with the following stdout: [12:24:28]root at vm-058-116:/home/ofayans]$ ipa-client-install Skip ldap.corp.redhat.com: not an IPA server Skip ldap01.intranet.prod.int.sin2.redhat.com: not an IPA server Skip ldap02.intranet.prod.int.sin2.redhat.com: not an IPA server Skip ldap02.intranet.prod.int.ams2.redhat.com: not an IPA server Skip ldap01.intranet.prod.int.ams2.redhat.com: not an IPA server DNS discovery failed to determine your DNS domain Provide the domain name of your IPA server (ex: example.com): The packages were built today from the upstream branch. [12:28:02]root at vm-058-116:/home/ofayans]$ rpm -qa | grep freeipa freeipa-admintools-4.2.90.201512010847GITfa62480-0.fc23.x86_64 freeipa-server-dns-4.2.90.201512010847GITfa62480-0.fc23.x86_64 freeipa-python-4.2.90.201512010847GITfa62480-0.fc23.x86_64 freeipa-server-trust-ad-4.2.90.201512010847GITfa62480-0.fc23.x86_64 freeipa-server-4.2.90.201512010847GITfa62480-0.fc23.x86_64 freeipa-debuginfo-4.2.90.201512010847GITfa62480-0.fc23.x86_64 freeipa-tests-4.2.90.201512010847GITfa62480-0.fc23.x86_64 freeipa-client-4.2.90.201512010847GITfa62480-0.fc23.x86_64 Does anybody have any ideas? -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From tbabej at redhat.com Tue Dec 1 12:05:16 2015 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 1 Dec 2015 13:05:16 +0100 Subject: [Freeipa-devel] dns discovery broken? In-Reply-To: <565D8B77.8090702@redhat.com> References: <565D8B77.8090702@redhat.com> Message-ID: <565D8CFC.70508@redhat.com> On 12/01/2015 12:58 PM, Oleg Fayans wrote: > Hi all, > > I am unable to enroll a client with the latest upstream ipa code > > I've successfully installed ipa-server with --setup-dns and --forwarder > provided. I configured client's resolv.conf to use master's ip as the > first dns server. > But when I issued the ipa-client-install command, it fails with the > following stdout: > > [12:24:28]root at vm-058-116:/home/ofayans]$ ipa-client-install > Skip ldap.corp.redhat.com: not an IPA server > Skip ldap01.intranet.prod.int.sin2.redhat.com: not an IPA server > Skip ldap02.intranet.prod.int.sin2.redhat.com: not an IPA server > Skip ldap02.intranet.prod.int.ams2.redhat.com: not an IPA server > Skip ldap01.intranet.prod.int.ams2.redhat.com: not an IPA server > DNS discovery failed to determine your DNS domain > Provide the domain name of your IPA server (ex: example.com): > > The packages were built today from the upstream branch. > > [12:28:02]root at vm-058-116:/home/ofayans]$ rpm -qa | grep freeipa > freeipa-admintools-4.2.90.201512010847GITfa62480-0.fc23.x86_64 > freeipa-server-dns-4.2.90.201512010847GITfa62480-0.fc23.x86_64 > freeipa-python-4.2.90.201512010847GITfa62480-0.fc23.x86_64 > freeipa-server-trust-ad-4.2.90.201512010847GITfa62480-0.fc23.x86_64 > freeipa-server-4.2.90.201512010847GITfa62480-0.fc23.x86_64 > freeipa-debuginfo-4.2.90.201512010847GITfa62480-0.fc23.x86_64 > freeipa-tests-4.2.90.201512010847GITfa62480-0.fc23.x86_64 > freeipa-client-4.2.90.201512010847GITfa62480-0.fc23.x86_64 > > Does anybody have any ideas? > I don't think the discovery is broken, I used it successfully today. Check your DNS setup, in particular, try to resolve server's records just as ipa-client-install does, to poinpoint the problem. As a sanity check, at least the _ldap._tcp record should be resolvable in the domain from the client machine. See ipa-client/ipaclient/ipadiscovery.py for the full logic. HTH, Tomas From mbasti at redhat.com Tue Dec 1 12:47:57 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 1 Dec 2015 13:47:57 +0100 Subject: [Freeipa-devel] [PATCH 0107] fix 'iparepltopomanagedsuffix' attribute consumers In-Reply-To: <565D862A.2010408@redhat.com> References: <565D82B4.50005@redhat.com> <565D846B.4060403@redhat.com> <565D862A.2010408@redhat.com> Message-ID: <565D96FD.6030601@redhat.com> On 01.12.2015 12:36, Martin Babinsky wrote: > On 12/01/2015 12:28 PM, Martin Babinsky wrote: >> On 12/01/2015 12:21 PM, Martin Babinsky wrote: >>> This patch fixes a regression caused by recently pushed >>> topologysuffix-related patches. >>> >>> >>> >> self-NACK. >> > This patch should actually work. > > > ACK Pushed to master: 525f6281d820ba7d3be780127d79a62221c5f1ad -------------- next part -------------- An HTML attachment was scrubbed... URL: From dkupka at redhat.com Tue Dec 1 13:27:28 2015 From: dkupka at redhat.com (David Kupka) Date: Tue, 1 Dec 2015 14:27:28 +0100 Subject: [Freeipa-devel] [PATCH 0069] ipa-replica-install support caless install with promotion. In-Reply-To: <565C7858.1030702@redhat.com> References: <565710A8.1020306@redhat.com> <56571376.6020209@redhat.com> <565715BB.9020200@redhat.com> <5657FEC2.3080503@redhat.com> <565C7858.1030702@redhat.com> Message-ID: <565DA040.4000509@redhat.com> On 30/11/15 17:24, Jan Cholasta wrote: > Hi, > > On 27.11.2015 07:57, David Kupka wrote: >> On 26/11/15 15:22, David Kupka wrote: >>> On 26/11/15 15:13, David Kupka wrote: >>>> On 26/11/15 15:01, David Kupka wrote: >>>>> https://fedorahosted.org/freeipa/ticket/5441 >>>>> >>>>> >>>> Replaced accidentally inserted tabs. >>>> >>>> >>>> >>> Fixed indentation I screwed up when replacing tabs :-/ > > 1) The deprecated --*_pkcs12 and --*_pin aliases should not be supported > in ipa-replica-install. > > > 2) This check from ipa-replica-prepare should be added to > Replica.__init__() as well: > > # If any of the PKCS#12 options are selected, all are required. > cert_file_req = (options.dirsrv_cert_files, > options.http_cert_files) > cert_file_opt = (options.pkinit_cert_files,) > if any(cert_file_req + cert_file_opt) and not all(cert_file_req): > self.option_parser.error( > "--dirsrv-cert-file and --http-cert-file are required > if any " > "PKCS#12 options are used.") > > > 3) This check from ipa-replica-prepare should be added below the > pkcs12_info initialization block in promote_check(): > > if (options.http_cert_files and options.dirsrv_cert_files and > http_ca_cert != dirsrv_ca_cert): > raise admintool.ScriptError( > "Apache Server SSL certificate and Directory Server SSL " > "certificate are not signed by the same CA certificate") > > > 4) This check should use the same message as ipa-replica-prepare: > "Cannot issue certificates: a CA is not installed. Use the > --http-cert-file, --dirsrv-cert-file options to provide custom > certificates.": > > + if not options.dirsrv_cert_files: > + root_logger.error("The remote master does not have a CA " > + "installed, can't proceed without > certs") > + sys.exit(3) > > > 5) Please use the common "You cannot specify a --option together with > replica file" error message here: > > + if any(self.ca.dirsrv_pkcs12_file, self.ca.http_pkcs12_file, > + self.ca.pkinit_pkcs12_file): > + raise RuntimeError("You cannot provide certificates > together " > + "with replica file") > > > 6) Please make the ca_is_enabled argument of install_replica_ds() and > install_http() mandatory and fill as appropriate when called, it will > make the code more readable. > > > 7) > > $ git diff -U0 | pep8 --diff > ./ipaserver/install/server/replicainstall.py:99:80: E501 line too long > (82 > 79 characters) > ./ipaserver/install/server/replicainstall.py:161:80: E501 line too long > (82 > 79 characters) > ./ipaserver/install/server/replicainstall.py:1289:13: E265 block comment > should start with '# ' > ./ipaserver/install/server/replicainstall.py:1291:17: E125 continuation > line with same indent as next logical line > ./ipaserver/install/server/replicainstall.py:1291:17: E128 continuation > line under-indented for visual indent > > > 8) Nitpicks: > > s/ca_configured/ca_is_configured/ in install_replica_ds(), for consistency. > > Set ca_enabled = False in the else branch rather than before the if > statement in promote_check(). > > Put the "#pylint: disable=no-member" in Replica.__init__() in the same > spot as it is in Server.__init__(). > > > Honza > Thank you for review. Updated and rebased patch attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0069.3-ipa-replica-install-support-caless-install-with-prom.patch Type: text/x-patch Size: 20697 bytes Desc: not available URL: From jcholast at redhat.com Tue Dec 1 13:37:11 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 1 Dec 2015 14:37:11 +0100 Subject: [Freeipa-devel] [PATCH] 0749 Package ipapython, ipalib, ipaplatform, ipatests for Python 3 In-Reply-To: <565D80B0.9000500@redhat.com> References: <563C9810.1050604@redhat.com> <5656E487.8050703@redhat.com> <5658508D.1020105@redhat.com> <565C0FF3.8040406@redhat.com> <565D80B0.9000500@redhat.com> Message-ID: <565DA287.6030403@redhat.com> On 1.12.2015 12:12, Petr Viktorin wrote: > Hello, > I noticed I didn't attach an updated patch last time, which probably > affected the discussion here. Sorry for that; it's attached this time. > > We seem to have a disconenct here: there are two different strategies to > do the packaging. > > Your idea of the port seems to be to have the py2 and py3 packages "in > sync", so they're almost exact copies of each other. This keeps things > consistent, but I'm not sure this specific kind of consistency is better > than the alternative. > > My approach is to make new, better packages for Python 3. I treat this > as a bit of a sandbox, a new chance to do things right. My patch tries > to keep the existing Python 2 packages untouched where it's feasible, to > minimize regressions. > (I did add new Provides based on your review, though, so users can use > the "consistent" names.) > When the py3 packages are finalized, the changes can be applied to the > py2 ones as well. (Or not, if it's decided it's a case of fixing things > that ain't broke.) > > So, two paths to get to the same goal. I'm not saying yours is bad, but > I fail to see how it's objectively better. If it is, could you explain > why? Otherwise please take this patch as an incremental improvement with > priorities that are a bit different than yours. Developer and user convenience. If your usage of the packages goes beyond "yum install package", being self-consistent should prevent raised eyebrows. > > > On 11/30/2015 09:59 AM, Jan Cholasta wrote: >> On 27.11.2015 13:46, Petr Viktorin wrote: >>> On 11/26/2015 11:52 AM, Jan Cholasta wrote: >>>> 1) The freeipa-common subpackage is not necessary: /etc/ipa/dnssec >>>> should be owned by freeipa-server and everything else in /etc/ipa >>>> currently owned by freeipa-python should be owned by freeipa-client. >>> >>> If the common files are in freeipa-server or freeipa-client, then the >>> Python 3 packages would need to depend on those. I'd like to avoid that. >> >> I'm not sure I follow. The files do not belong to freeipa-python, since >> they are used only by freeipa-server and freeipa-client. What Python 3 >> packages would need to depend on them? > > python3-ipaserver and python3-ipaclient, eventually. freeipa-server and freeipa-client already own some non-Python files and you will have to deal with them either way. > > Anyway, /etc/ipa/default.conf is used directly by ipapython.config, so > at least that should be owned by a common package. And once there's a > common package, I don't think there's a problem with it holding the > other tiny files as well. /etc/ipa/default.conf is managed by freeipa-client and thus should be owned by it. This is a common pattern in other packages (even other FreeIPA sub-packages) and I don't see any reason not to follow it here as well. > Also, a lot of the other files are mentioned in ipaplatform.base.paths, > which is part of the freeipa-python, so it seems fitting that that's > what should require them, rather than the -client or -server packages. Following this logic, every non-Python file in every sub-package should be owned by freeipa-python (or freeipa-common), which is definitely not something we want. > >>>> This should be a separate patch. I have prepared one, see attachment. >>>> >>>> >>>> 2) IMO this patch does too much, it should only add the Python 3 >>>> equivalent of freeipa-python - python3-ipalib - but none of the other >>>> packages/provides. >>> >>> My reasoning for the new packages: >>> - python3-ipalib is the main point >> >> Right. >> >>> - freeipa-common contains files that both the py2 and py3 versions need >> >> See above. >> >>> - python-ipap11helper has compiled code: with this pulled out, >>> python3-ipalib can be noarch >> >> This is not the goal here, but if you insist on doing it, do it for >> Python 2 as well. > > It is definitely the goal of this patch to make the py3 packages as good > as possible. That includes making them noarch. It is completely unnecessary for the initial py3 support. I would rather maintain internal consistency than make the py3 packages "perfect". > On the other hand improving the py2 packages is not a goal of this > particular patch. Which is exactly the reason I have provided patches for py2 packages myself. > >>> - python3-ipatests is needed if we want to start testing the py3 >>> packages in CI >> >> Right. >> >>> >>> As for new provides, Fedora's Python packaging guidelines say: >>> >>> """ >>> Using a fictional module named "example", the subpackage containing >>> the python2 version must provide python2-example. This is of course >>> always the case if the subpackage is named python2-example [...] >>> If the subpackage has some other name then then Provides: python2-example >>> must be added explicitly (but see the %python_provide macro below). >>> >>> The python3 subpackage must provide python3-example. However, as the >>> naming guidelines mandate that the python3 subpackage be named >>> python3-example, this will happen automatically. >>> """ >>> >>> so I'm now adding Provides for the top-level modules. >> >> The goal of this work is to add support for Python 3, not to comply with >> Fedora packaging guidelines. FreeIPA on Fedora uses its own spec file >> anyway. > > The goal of this patch is to add new packages that support Python 3. > Yes, the Fedora spec is different, but it's heavily based on the > upstream one, and this is a good thing. I consider the Fedora guidelines > the standard in Python RPM packaging. If IPA uses different packaging > guidelines, can you point me to them? FreeIPA never fully complied to Fedora packaging guidelines AFAIK and I don't see any reason to start now, since nobody seemed to care so far. Following them in just py3 sub-packages does not improve the state of FreeIPA as a whole and only brings inconsistency into it, so there's no benefit in doing it at all. > >> Again, if you insist on doing this, do it for Python 2 as well. > > Improving the py2 packages would be great, but again it is not the goal > of this patch. Like it or not, whatever you do for py3 support must integrate nicely in FreeIPA, and that might mean touching py2 stuff as well. > >>>> 3) Speaking of freeipa-python, it should be renamed to python-ipalib, >>>> for consistency. >>>> >>>> This should also be a separate patch, again see attachment. >>> >>> I think that would just be unnecessary churn. Python 3 gives us a chance >>> for a fresh start, so I'm taking advantage of that, but renaming >>> existing packages is a pain. >> >> Something similar was done in SSSD not long ago and according to Luk?? >> (CCd), it was no pain. > > OK. But again, not the goal here. Which is again exactly the reason I provided the patch myself. > >>> With the right Provides, "dnf install python-ipalib" (or python2-ipalib) >>> will do the right thing, and I thing that's enough. >> >> I don't think that's good enough, since it creates an inconsistency. >> >>> >>>> 4) There should be a python3-devel (and also other python3- equivalents >>>> of python- packages) BuildRequires when Python 3 support is enabled. >>> >>> I did miss that somehow. Thanks for checking. >>> I added python3-devel; the others are unnecessary because pylint is not >>> currently run on the Python3 version. >> >> OK, I think that will do for now. Note that many of them are required >> for makeapi and makeaci as well. >> >> When can we expect a Python 3 pylint patch which adds the remaining >> BuildRequires? > > That's something we'll need to talk about in the future: do we want to > run python2-pylint and python3-pylint on every build? > Anyway, I'd like to see the Python3 tests for ipapython and ipalib run > in CI first. Sure, no problem. -- Jan Cholasta From simo at redhat.com Tue Dec 1 13:38:05 2015 From: simo at redhat.com (Simo Sorce) Date: Tue, 01 Dec 2015 08:38:05 -0500 Subject: [Freeipa-devel] [PATCH 556-557] Add option to disable setkeytab extended operations In-Reply-To: <20151201081145.GJ9605@redhat.com> References: <1448394152.29102.10.camel@redhat.com> <1448395038.29102.11.camel@redhat.com> <1448399848.29102.13.camel@redhat.com> <56555AC7.2000908@redhat.com> <5655BF5B.6020405@redhat.com> <1448462853.29102.44.camel@redhat.com> <1448908709.3747.93.camel@redhat.com> <20151201081145.GJ9605@redhat.com> Message-ID: <1448977085.9040.1.camel@redhat.com> On Tue, 2015-12-01 at 10:11 +0200, Alexander Bokovoy wrote: > On Mon, 30 Nov 2015, Simo Sorce wrote: > >On Wed, 2015-11-25 at 09:47 -0500, Simo Sorce wrote: > >> On Wed, 2015-11-25 at 09:02 -0500, Rob Crittenden wrote: > >> > Jan Cholasta wrote: > >> > > On 24.11.2015 22:17, Simo Sorce wrote: > >> > >> On Tue, 2015-11-24 at 14:57 -0500, Simo Sorce wrote: > >> > >>> On Tue, 2015-11-24 at 14:42 -0500, Simo Sorce wrote: > >> > >>>> Since some time we use the getkeytab operation to fetch keytabs on > >> > >>>> newer > >> > >>>> clients. According to bug #232 setkeytab can be used to circumvent > >> > >>>> password quality controls so it needs to be slowly retired. > >> > >>>> > >> > >>>> The attached patches implement #5485 in 2 parts. > >> > >>>> > >> > >>>> The first introduces the option DisableSetKeytab which globally > >> > >>>> disables > >> > >>>> the setkeytab extended operation. This is set to false by default for > >> > >>>> backwards compatibility. > >> > >>>> > >> > >>>> The second introduces an option called DisableUserSetKeytab, which is > >> > >>>> active by default in new installs (but not in upgraded ones), and only > >> > >>>> disables the use of setkeytab for ipa suers, but not for > >> > >>>> hosts/services. > >> > >>>> This is because user's are the ones that may abuse the interface to > >> > >>>> escape password policies and users also normally do not acquire > >> > >>>> keytabs, > >> > >>>> so it is a safe bet to disable just them by default in new installs. > >> > >>>> > >> > >>>> (Testing in progress) > >> > >>> > >> > >>> Tested and working as expected. > >> > >> > >> > >> I realized that adding options to ipaConfig require to add them in the > >> > >> UI as well, attached patches add options in API.txt and config.py > >> > >> Make now complain I should change API Major or Minor, but it is not > >> > >> clear to me why given this are additional values and no real change or > >> > >> new function is introduced. What's the recommendation ? > >> > > > >> > > When does make complain? It is supposed to complain only when API.txt > >> > > does not match code. > >> > > > >> > > Anyway, we usually bump minor version even for backward compatible > >> > > changes, see e.g. commit 9549a59. > >> > > > >> > > >> > The point of API.txt (and the heavy client) was to save a round-trip. > >> > Being able to pass in an invalid option would void that rule hence > >> > having to update the API when new values are added. > >> > > >> > rob > >> > >> Ok added version change to the second patch (so we bump it only once > >> given these are basically related changes. > > > >Bump, is this ok ? > This patch is fine but please fix setkeytab use in ipa-sam before > committing this patch. This patch does not disable setkeytab yet, so it can go in right away (it blocks other patches already). We have a bug to change ipa-sam, should we mark it blocker for 4.4 ? Simo. -- Simo Sorce * Red Hat, Inc * New York From mbabinsk at redhat.com Tue Dec 1 13:40:45 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Tue, 1 Dec 2015 14:40:45 +0100 Subject: [Freeipa-devel] [PATCH 0098-0099] domain level 1 topology checks during IPA server uninstall In-Reply-To: <565CA4B8.1010701@redhat.com> References: <564E04BE.1020304@redhat.com> <56586187.5090402@redhat.com> <565C3399.5030109@redhat.com> <565C6E8C.2090900@redhat.com> <565C844B.7050901@redhat.com> <565C8A57.2000305@redhat.com> <565CA4B8.1010701@redhat.com> Message-ID: <565DA35D.6010804@redhat.com> On 11/30/2015 08:34 PM, Martin Basti wrote: > > > On 30.11.2015 18:41, Martin Babinsky wrote: >> On 11/30/2015 06:15 PM, Martin Basti wrote: >>> >>> >>> On 30.11.2015 16:43, Martin Babinsky wrote: >>>> On 11/30/2015 12:31 PM, Jan Cholasta wrote: >>>>> Hi, >>>>> >>>>> On 27.11.2015 14:58, Martin Babinsky wrote: >>>>>> On 11/19/2015 06:19 PM, Martin Babinsky wrote: >>>>>>> These two patches fix the following tickets: >>>>>>> >>>>>>> https://fedorahosted.org/freeipa/ticket/5377 >>>>>>> https://fedorahosted.org/freeipa/ticket/5409 >>>>>>> >>>>>>> I have added a new option '--ignore-disconnected-topology' which >>>>>>> forces >>>>>>> IPA master uninstall despite reported errors in topology. I'm not >>>>>>> quite >>>>>>> sure if we want to flood ipa-server-install with uninstall-specific >>>>>>> options, maybe it is better to skip the check in unattended mode and >>>>>>> just print a warning about disconnected topology and what to do >>>>>>> about >>>>>>> it. >>>>>>> >>>>>>> I would like to hear your opinions about this. >>>>>>> >>>>>>> >>>>>>> >>>>>> Attaching rebased and updated patches. >>>>> >>>>> Patch 0098: LGTM >>>>> >>>>> >>>>> Patch 0099: >>>>> >>>>> a) This check should be done in Server.__init__() rather than >>>>> install_check(): >>>>> >>>>> + if options.ignore_disconnected_topology: >>>>> + print("'--ignore-disconnected-topology' is used only during " >>>>> + "uninstallation") >>>>> + sys.exit(1) >>>>> >>>>> >>>>> b) s/--ignore-disconnected-topology/--ignore-topology-disconnect/, for >>>>> consistency with other options, e.g. --no-ui-redirect. >>>>> >>>>> Maybe even shorten it to --ignore-topology? But we probably don't want >>>>> people to use this option much, so it might be better to keep it long? >>>>> >>>> I would rather leave it with the long option name, it is more apparent >>>> what this switch should be around. >>>>> >>>>> c) I'm fine with uninstall options, you can remove the TODO: >>>>> >>>>> + # TODO: ask jcholast about uninstallation options >>>>> >>>>> >>>>> Honza >>>>> >>>> >>>> Attaching updated patches. >>>> >>> NACK >>> >>> ipa-server-install --uninstall >>> >>> 2015-11-30T17:14:30Z DEBUG Destroyed connection >>> context.ldap2_140081152041808 >>> 2015-11-30T17:14:30Z DEBUG Traceback (most recent call last): >>> File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>> line 91, in _handle_exception >>> super(Continuous, self)._handle_exception(exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>> line 387, in _handle_exception >>> six.reraise(*exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>> line 439, in _handle_exception >>> super(ComponentBase, self)._handle_exception(exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>> line 387, in _handle_exception >>> six.reraise(*exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>> line 355, in __runner >>> step() >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>> line 352, in >>> step = lambda: next(self.__gen) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>> line 81, in run_generator_with_yield_from >>> six.reraise(*exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>> line 59, in run_generator_with_yield_from >>> value = gen.send(prev_value) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>> line 71, in _uninstall >>> for nothing in self._uninstaller(self.parent): >>> File >>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>> line 1409, in main >>> uninstall_check(self) >>> File >>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>> line 265, in decorated >>> func(installer) >>> File >>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>> line 1140, in uninstall_check >>> api, masters, options.ignore_disconnected_topology) >>> AttributeError: 'uninstaller(Server)' object has no attribute >>> 'ignore_disconnected_topology' >>> >>> 2015-11-30T17:14:30Z ERROR 'uninstaller(Server)' object has no attribute >>> 'ignore_disconnected_topology' >>> 2015-11-30T17:14:30Z INFO The ipa-server-install command was successful >>> >> >> Sorry I have failed horribly during option rename. Attaching patch >> that should actually work. >> > functional ACK Attaching rebased patches reflecting the recent changes in the handling of managed topology suffixes handling. -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0099.4-implement-domain-level-1-specific-topology-checks-in.patch Type: text/x-patch Size: 11263 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0098.2-extract-domain-level-1-topology-checking-code-from-i.patch Type: text/x-patch Size: 10172 bytes Desc: not available URL: From mkosek at redhat.com Tue Dec 1 13:42:54 2015 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 1 Dec 2015 14:42:54 +0100 Subject: [Freeipa-devel] [PATCH 556-557] Add option to disable setkeytab extended operations In-Reply-To: <1448977085.9040.1.camel@redhat.com> References: <1448394152.29102.10.camel@redhat.com> <1448395038.29102.11.camel@redhat.com> <1448399848.29102.13.camel@redhat.com> <56555AC7.2000908@redhat.com> <5655BF5B.6020405@redhat.com> <1448462853.29102.44.camel@redhat.com> <1448908709.3747.93.camel@redhat.com> <20151201081145.GJ9605@redhat.com> <1448977085.9040.1.camel@redhat.com> Message-ID: <565DA3DE.8000109@redhat.com> On 12/01/2015 02:38 PM, Simo Sorce wrote: > On Tue, 2015-12-01 at 10:11 +0200, Alexander Bokovoy wrote: >> On Mon, 30 Nov 2015, Simo Sorce wrote: >>> On Wed, 2015-11-25 at 09:47 -0500, Simo Sorce wrote: >>>> On Wed, 2015-11-25 at 09:02 -0500, Rob Crittenden wrote: >>>>> Jan Cholasta wrote: >>>>>> On 24.11.2015 22:17, Simo Sorce wrote: >>>>>>> On Tue, 2015-11-24 at 14:57 -0500, Simo Sorce wrote: >>>>>>>> On Tue, 2015-11-24 at 14:42 -0500, Simo Sorce wrote: >>>>>>>>> Since some time we use the getkeytab operation to fetch keytabs on >>>>>>>>> newer >>>>>>>>> clients. According to bug #232 setkeytab can be used to circumvent >>>>>>>>> password quality controls so it needs to be slowly retired. >>>>>>>>> >>>>>>>>> The attached patches implement #5485 in 2 parts. >>>>>>>>> >>>>>>>>> The first introduces the option DisableSetKeytab which globally >>>>>>>>> disables >>>>>>>>> the setkeytab extended operation. This is set to false by default for >>>>>>>>> backwards compatibility. >>>>>>>>> >>>>>>>>> The second introduces an option called DisableUserSetKeytab, which is >>>>>>>>> active by default in new installs (but not in upgraded ones), and only >>>>>>>>> disables the use of setkeytab for ipa suers, but not for >>>>>>>>> hosts/services. >>>>>>>>> This is because user's are the ones that may abuse the interface to >>>>>>>>> escape password policies and users also normally do not acquire >>>>>>>>> keytabs, >>>>>>>>> so it is a safe bet to disable just them by default in new installs. >>>>>>>>> >>>>>>>>> (Testing in progress) >>>>>>>> >>>>>>>> Tested and working as expected. >>>>>>> >>>>>>> I realized that adding options to ipaConfig require to add them in the >>>>>>> UI as well, attached patches add options in API.txt and config.py >>>>>>> Make now complain I should change API Major or Minor, but it is not >>>>>>> clear to me why given this are additional values and no real change or >>>>>>> new function is introduced. What's the recommendation ? >>>>>> >>>>>> When does make complain? It is supposed to complain only when API.txt >>>>>> does not match code. >>>>>> >>>>>> Anyway, we usually bump minor version even for backward compatible >>>>>> changes, see e.g. commit 9549a59. >>>>>> >>>>> >>>>> The point of API.txt (and the heavy client) was to save a round-trip. >>>>> Being able to pass in an invalid option would void that rule hence >>>>> having to update the API when new values are added. >>>>> >>>>> rob >>>> >>>> Ok added version change to the second patch (so we bump it only once >>>> given these are basically related changes. >>> >>> Bump, is this ok ? >> This patch is fine but please fix setkeytab use in ipa-sam before >> committing this patch. > > This patch does not disable setkeytab yet, so it can go in right away > (it blocks other patches already). We have a bug to change ipa-sam, > should we mark it blocker for 4.4 ? We also have to fix the permission to change keytab, so that it uses the new style (https://fedorahosted.org/freeipa/ticket/5487). I would actually make this ticket and the ipa-sam ticket a blocker for this patch set. Otherwise you are actually introducing a switch that breaks FreeIPA as some of it's core server functions still use the old method. From redhatrises at gmail.com Tue Dec 1 13:57:00 2015 From: redhatrises at gmail.com (Gabe Alford) Date: Tue, 1 Dec 2015 06:57:00 -0700 Subject: [Freeipa-devel] [PATCH 0065] ipa-replica-install prints incorrect error message when replica is already installed Message-ID: Sorry guys, I forgot to add a meaningful subject to this message. Ignore the previous thread start. ---------- Forwarded message ---------- From: Gabe Alford Date: Mon, Nov 30, 2015 at 7:31 PM Subject: [PATCH 0065] To: freeipa-devel Hello, Patch fix for the following tickets: https://fedorahosted.org/freeipa/ticket/5022 https://fedorahosted.org/freeipa/ticket/5320 Thanks, Gabe -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rga-0064-ipa-replica-install-prints-incorrect-error-message-w.patch Type: text/x-patch Size: 2592 bytes Desc: not available URL: From simo at redhat.com Tue Dec 1 13:59:01 2015 From: simo at redhat.com (Simo Sorce) Date: Tue, 01 Dec 2015 08:59:01 -0500 Subject: [Freeipa-devel] [PATCH 556-557] Add option to disable setkeytab extended operations In-Reply-To: <565DA3DE.8000109@redhat.com> References: <1448394152.29102.10.camel@redhat.com> <1448395038.29102.11.camel@redhat.com> <1448399848.29102.13.camel@redhat.com> <56555AC7.2000908@redhat.com> <5655BF5B.6020405@redhat.com> <1448462853.29102.44.camel@redhat.com> <1448908709.3747.93.camel@redhat.com> <20151201081145.GJ9605@redhat.com> <1448977085.9040.1.camel@redhat.com> <565DA3DE.8000109@redhat.com> Message-ID: <1448978341.9040.5.camel@redhat.com> On Tue, 2015-12-01 at 14:42 +0100, Martin Kosek wrote: > On 12/01/2015 02:38 PM, Simo Sorce wrote: > > On Tue, 2015-12-01 at 10:11 +0200, Alexander Bokovoy wrote: > >> On Mon, 30 Nov 2015, Simo Sorce wrote: > >>> On Wed, 2015-11-25 at 09:47 -0500, Simo Sorce wrote: > >>>> On Wed, 2015-11-25 at 09:02 -0500, Rob Crittenden wrote: > >>>>> Jan Cholasta wrote: > >>>>>> On 24.11.2015 22:17, Simo Sorce wrote: > >>>>>>> On Tue, 2015-11-24 at 14:57 -0500, Simo Sorce wrote: > >>>>>>>> On Tue, 2015-11-24 at 14:42 -0500, Simo Sorce wrote: > >>>>>>>>> Since some time we use the getkeytab operation to fetch keytabs on > >>>>>>>>> newer > >>>>>>>>> clients. According to bug #232 setkeytab can be used to circumvent > >>>>>>>>> password quality controls so it needs to be slowly retired. > >>>>>>>>> > >>>>>>>>> The attached patches implement #5485 in 2 parts. > >>>>>>>>> > >>>>>>>>> The first introduces the option DisableSetKeytab which globally > >>>>>>>>> disables > >>>>>>>>> the setkeytab extended operation. This is set to false by default for > >>>>>>>>> backwards compatibility. > >>>>>>>>> > >>>>>>>>> The second introduces an option called DisableUserSetKeytab, which is > >>>>>>>>> active by default in new installs (but not in upgraded ones), and only > >>>>>>>>> disables the use of setkeytab for ipa suers, but not for > >>>>>>>>> hosts/services. > >>>>>>>>> This is because user's are the ones that may abuse the interface to > >>>>>>>>> escape password policies and users also normally do not acquire > >>>>>>>>> keytabs, > >>>>>>>>> so it is a safe bet to disable just them by default in new installs. > >>>>>>>>> > >>>>>>>>> (Testing in progress) > >>>>>>>> > >>>>>>>> Tested and working as expected. > >>>>>>> > >>>>>>> I realized that adding options to ipaConfig require to add them in the > >>>>>>> UI as well, attached patches add options in API.txt and config.py > >>>>>>> Make now complain I should change API Major or Minor, but it is not > >>>>>>> clear to me why given this are additional values and no real change or > >>>>>>> new function is introduced. What's the recommendation ? > >>>>>> > >>>>>> When does make complain? It is supposed to complain only when API.txt > >>>>>> does not match code. > >>>>>> > >>>>>> Anyway, we usually bump minor version even for backward compatible > >>>>>> changes, see e.g. commit 9549a59. > >>>>>> > >>>>> > >>>>> The point of API.txt (and the heavy client) was to save a round-trip. > >>>>> Being able to pass in an invalid option would void that rule hence > >>>>> having to update the API when new values are added. > >>>>> > >>>>> rob > >>>> > >>>> Ok added version change to the second patch (so we bump it only once > >>>> given these are basically related changes. > >>> > >>> Bump, is this ok ? > >> This patch is fine but please fix setkeytab use in ipa-sam before > >> committing this patch. > > > > This patch does not disable setkeytab yet, so it can go in right away > > (it blocks other patches already). We have a bug to change ipa-sam, > > should we mark it blocker for 4.4 ? > > We also have to fix the permission to change keytab, so that it uses the new > style (https://fedorahosted.org/freeipa/ticket/5487). I would actually make > this ticket and the ipa-sam ticket a blocker for this patch set. > > Otherwise you are actually introducing a switch that breaks FreeIPA as some of > it's core server functions still use the old method. The point of this patchset is to introduce the switch early so that current server will support the off switch when newer servers down the road are ready to flip it. The defaults are still to allow these operations for services/hosts. If you flip it right now you'll break stuff, including all older clients on "Enterprise" distributions, so this switch will not be flipped soon. Simo. -- Simo Sorce * Red Hat, Inc * New York From redhatrises at gmail.com Tue Dec 1 14:00:38 2015 From: redhatrises at gmail.com (Gabe Alford) Date: Tue, 1 Dec 2015 07:00:38 -0700 Subject: [Freeipa-devel] [PATCH 0066] Migrate wget references to curl Message-ID: Hello, Fix for https://fedorahosted.org/freeipa/ticket/5458 Thanks, Gabe -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rga-0066-Migrate-wget-references-and-usage-to-curl.patch Type: text/x-patch Size: 9497 bytes Desc: not available URL: From mbabinsk at redhat.com Tue Dec 1 14:59:45 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Tue, 1 Dec 2015 15:59:45 +0100 Subject: [Freeipa-devel] [PATCH 558] Allow disabling requireing preauth by default for Service Principal Names In-Reply-To: <1448908922.3747.94.camel@redhat.com> References: <1448400058.29102.17.camel@redhat.com> <56558064.7010600@redhat.com> <1448908922.3747.94.camel@redhat.com> Message-ID: <565DB5E1.8040500@redhat.com> On 11/30/2015 07:42 PM, Simo Sorce wrote: > On Wed, 2015-11-25 at 10:33 +0100, Martin Babinsky wrote: >> On 11/24/2015 10:20 PM, Simo Sorce wrote: >>> This addresses #3860, giving admins the option to not require preauth >>> for Hosts and services. >>> >>> I did not add this option by default, although it does reduce the load >>> on the KDC as well as speed up TGT acquisition for service principal >>> accounts that acquire TGTs. >>> >>> Tested and working as expected (SPNs are not returned PREAUTH_NEEDED >>> error while normal users are). >>> >>> HTH, >>> Simo. >>> >>> >>> >> Hi Simo, >> >> I was not able to apply the patch on current master branch: >> >> """ >> git am >> ../review/ssorce/3860/freeipa-simo-558-1-Allow-admins-to-disable-preauth-for-SPNs.patch >> -3 >> >> Applying: Allow admins to disable preauth for SPNs. >> error: invalid object 100644 a6b4d4349a9ac6de453d9ad3c679ec32add4e43b >> for 'ipalib/plugins/config.py' >> fatal: git-write-tree: error building trees >> Repository lacks necessary blobs to fall back on 3-way merge. >> Cannot fall back to three-way merge. >> Patch failed at 0001 Allow admins to disable preauth for SPNs. >> """ >> >> It seems that I nedd to apply some of your other patches first (which one?) > > Sorry did not see this question earlier, it requires 556 and 557, I just > bumped that thread. > > Simo. > It seems that I need something else, patch 556-2 applies cleanly, but patch 557-3 fails with http://fpaste.org/296230/89819431/ on both master and 4-2 branch. -- Martin^3 Babinsky From slaznick at redhat.com Tue Dec 1 15:03:06 2015 From: slaznick at redhat.com (Stanislav Laznicka) Date: Tue, 1 Dec 2015 16:03:06 +0100 Subject: [Freeipa-devel] [PATCH] Removed duplicate domain name validation function In-Reply-To: <565D6116.9070203@redhat.com> References: <5655DF48.4060302@redhat.com> <56586981.2010703@redhat.com> <565869D0.4060606@redhat.com> <56586D27.4030802@redhat.com> <56587181.1070004@redhat.com> <565C9CEA.80105@redhat.com> <565D5C5E.1080707@redhat.com> <565D6116.9070203@redhat.com> Message-ID: <565DB6AA.6080306@redhat.com> Sending the patch with renamed function. Standa On 12/01/2015 09:57 AM, Jan Cholasta wrote: > On 1.12.2015 09:37, Petr Spacek wrote: >> On 30.11.2015 20:00, Martin Basti wrote: >>> >>> >>> On 27.11.2015 16:06, Stanislav Laznicka wrote: >>>> Please, see the modified patch attached. >>>> >>>> Standa >>>> >>>> On 11/27/2015 03:48 PM, Martin Basti wrote: >>>>> >>>>> >>>>> On 27.11.2015 15:33, Petr Spacek wrote: >>>>>> On 27.11.2015 15:32, Martin Basti wrote: >>>>>>> >>>>>>> On 25.11.2015 17:18, Stanislav Laznicka wrote: >>>>>>>> There were two functions for the same purpose. Removed one. >>>>>>>> >>>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> I would like to have "log" param of is_host_resolvable as optional >>>>>> Is there an immediate need for the optional param? If not, I >>>>>> would not >>>>>> clutter >>>>>> the code. >>>>>> >>>>> So at least I would like to move log param as the last param, in >>>>> case of >>>>> need it can be modified to optional parameter. >>>>> >>>>> Or log can be default as root_logger, IMO we us only root_logger >>>>> everywhere, but this need investigation. >>>>> >>>>> Martin >>>> >>> It works, but I would like to have Honza's opinion if ipalib/util.py >>> is the >>> right place for the new method and if we really need to use exceptions. > > ipalib/util.py is fine for the function, since it is not used anywhere > in ipapython. > > I would rename the function to verify_host_resolvable() though, since > the is_ prefix suggests the function returns a boolean value rather > than raises an exception or not. > >>> >>> IMO is_record_resolvable() should return only True/False and then it >>> can be >>> located in ipapython module >> >> Hmm, I do not see a reason why it should not throw an exception. IMHO >> the >> exception should contain the original error/result from resolver, so >> it is >> possible to print meaningful error message like "DNS query timed out" >> instead >> of "it does not work for some reason, trust us". >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-stlaz-0010-Removed-duplicate-domain-name-validating-function.patch Type: text/x-patch Size: 9461 bytes Desc: not available URL: From mbasti at redhat.com Tue Dec 1 15:08:03 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 1 Dec 2015 16:08:03 +0100 Subject: [Freeipa-devel] [PATCH] First part of the replica promotion tests + testplan In-Reply-To: <56587616.5030609@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> Message-ID: <565DB7D3.5040500@redhat.com> On 27.11.2015 16:26, Oleg Fayans wrote: > And patch N 16 passes lint too: > > On 11/27/2015 04:03 PM, Oleg Fayans wrote: >> Hi, >> >> On 11/27/2015 03:26 PM, Martin Basti wrote: >>> >>> >>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>> Hi Martin, >>>> >>>> All your suggestions were taken into account. Both patches are >>>> updated. Thank you for your help! >>>> >>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>> >>>>> >>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>> Hi Martin, >>>>>> >>>>>> I agree to all your points but one. please, see my comment below >>>>>> >>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>> Hi, >>>>>>> >>>>>>> 0) Note >>>>>>> Please be aware of https://fedorahosted.org/freeipa/ticket/5469 >>>>>>> during >>>>>>> KRA testing >>>>>>> >>>>>>> 1) >>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, this may >>>>>>> change >>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for domain level 0 >>>>>>> and 1 >>>>>>> >>>>>>> 2) >>>>>>> Why uninstall KRA then server, is not enough just uninstall server >>>>>>> which >>>>>>> covers KRA uninstall? >>>>>>> >>>>>>> + def teardown_method(self, method): >>>>>>> + for host in self.replicas: >>>>>>> + host.run_command(self.kra_uninstall, raiseonerr=False) >>>>>>> + tasks.uninstall_master(host) >>>>>>> >>>>>>> >>>>>>> 3) >>>>>>> Can be this function more generic? It should allow specify host >>>>>>> where >>>>>>> KRA should be installed not just master >>>>>>> >>>>>>> + def test_kra_install_master(self): >>>>>>> + self.master.run_command(self.kra_install) >>>>>>> >>>>>>> >>>>>>> 4) >>>>>>> >>>>>>> TestLevel0(Dummy): >>>>>>> Can be the test name more specific, something like >>>>>>> TestReplicaPromotionLevel0 >>>>>>> >>>>>>> >>>>>>> 5) >>>>>>> please remove this, the patch is on review and it will be pushed >>>>>>> sooner >>>>>>> than tests >>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>> >>>>>>> and as I mentioned in ticket #5455, I cannot reproduce it with >>>>>>> ipa-kra-install, so please provide steps to reproduce if you insist >>>>>>> that >>>>>>> this still does not work as expected with KRA. >>>>>>> >>>>>>> 6) This is completely wrong, it removes everything that we tried to >>>>>>> achieve with previous patches with domain level in CI >>>>>> >>>>>> Actually, being able to configure domain level per class is WAY more >>>>>> convenient, than to always have to think which domain level is >>>>>> appropriate for which particular test during jenkins job >>>>>> configuration. In fact, I should have thought about it from the very >>>>>> beginning. For example, in test_replica_promotion.py we have on >>>>>> class, >>>>>> which intiates with domain level = 1, while others - with domain >>>>>> level >>>>>> 0. With config-based approach, we would have to implement a separate >>>>>> step that raises domain level. Overall, I am against the approach, >>>>>> when you have to remember to set certain domain level in config for >>>>>> any particular test. The tests themselves should be aware of the >>>>>> domain level they need. >>>>> I do not say that we should not have something that overrides >>>>> settings >>>>> in from config in a particular test case, I say your patch is >>>>> doing it >>>>> wrong. >>>>> >>>>> I agree it is useful to have param domain_level in install_master, >>>>> and >>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by default, >>>>> because with your current patch the domain_level in config is not >>>>> used >>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>> >>>>> For example I want to achieve this goal: >>>>> test_vault.py, this test suite can run on domain level1 and on domain >>>>> level0, so with one test we can test 2 domain levels just with >>>>> putting >>>>> domain level into config file. >>>>> >>>>> I agree that with extraordinary test like replica promotion test >>>>> is, we >>>>> need something that allows override the config file. >>>>> >>>>> As I said bellow, domain_level default value should be None in >>>>> install_master and install_topo plugin. If domain level was specified >>>>> use the specified one, if not (value is None) use the domain level >>>>> from >>>>> config file. >>>> >>>> Agreed :) >>>> >>>>> >>>>> Martin >>>>>> >>>>>>> >>>>>>> [PATCH] Enabled setting domain_level per class derived from >>>>>>> TestIntegration >>>>>>> >>>>>>> When I configure domain level 0 in yaml config, how is this >>>>>>> supposed to >>>>>>> get into install methods when you removed that code? >>>>>>> >>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>> + "--domain-level=%i" % domain_level >>>>>>> >>>>>>> >>>>>>> You always use MAX_DOMAIN_LEVEL in this case or whatever is >>>>>>> specified in >>>>>>> domain_level option. >>>>>>> I suggest to use domain_level=None, and when it is None use >>>>>>> 'host.config.domain_level', if it is not None, use 'domain_level' >>>>>>> >>>>>>> With this we can specify domain level in config file for test that >>>>>>> can >>>>>>> be used for both domain levels and you can manually specify domain >>>>>>> level >>>>>>> for test that requires specific domain level. >>>>>>> >>>>>>> Also this should go away >>>>>>> >>>>>>> @classmethod >>>>>>> def install(cls, mh): >>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>> if cls.topology is None: >>>>>>> return >>>>>>> >>>>>>> I do not see reason why test should override configuration in >>>>>>> config in >>>>>>> this case. >>>>>>> >>>>>>> Martin >>>>>>> >>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Here is the updated version of the patch (more tests + fixed the >>>>>>>> issues of the first one) + patch 0017, that implements the >>>>>>>> necessary >>>>>>>> changes in the background code, i. e. patch 16 does not work >>>>>>>> without >>>>>>>> patch 17 >>>>>>>> >>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>> Hi guys, >>>>>>>>>> >>>>>>>>>> Here are first two automated testcases from this (so far >>>>>>>>>> incomplete) >>>>>>>>>> testplan: >>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>> >>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> PATCH 16: NACK >>>>>>>>> >>>>>>>>> 1) >>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>> 'domain_level' to >>>>>>>>> install_topo()? >>>>>>>>> Also it is good practise to add new option as the last parameter. >>>>>>>>> >>>>>>>>> 2) >>>>>>>>> cab you in both tests specify a domain level with constant >>>>>>>>> instead of >>>>>>>>> number literal? >>>>>>>>> >>>>>>>>> 3) >>>>>>>>> both test call install_topo with custom domain level, but it >>>>>>>>> cannot >>>>>>>>> work >>>>>>>>> because 1) (did you run the test?) >>>>>>>>> >>>>>>>>> 4) >>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>> Respectively why there is call of install_topo() that installs >>>>>>>>> replica. >>>>>>>>> As this test just tests that ipa-replica-prepare is not working >>>>>>>>> anymore, >>>>>>>>> is it worth to spend 20 minutes with installing replica and then >>>>>>>>> just no >>>>>>>>> tot use it? IMO to install master in install step is enough. >>>>>>>>> >>>>>>>>> Martin^2 >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> ./make-lint >>> ************* Module ipatests.test_integration.base >>> ipatests/test_integration/base.py:66: [E1101(no-member), >>> IntegrationTest.install] Class 'IntegrationTest' has no 'domain_level' >>> member) >>> ************* Module ipatests.test_integration.test_replica_promotion >>> ipatests/test_integration/test_replica_promotion.py:16: >>> [E1101(no-member), Dummy.install] Class 'Dummy' has no 'domain_level' >>> member) >>> ipatests/test_integration/test_replica_promotion.py:117: >>> [E1101(no-member), TestCAInstall.test_ca_install_without_replica_file] >>> Module 'ipatests.test_integration.tasks' has no 'setup_replica' member) >>> >>> >>> Is it so hard to run pylint before patch is posted for review? >> >> Sorry, my bad. Fixed. >> >> >> > 1) Why is this change in the patch? - # Clean up the test directory - host.run_command(['rm', '-rvf', host.config.test_dir]) - 2) is enough to have this check only in install_master, install_topo can pass None to install_master + if domain_level is None: + domain_level = master.config.domain_level 3) IMO replica-manage del should cleanup hosts entry, so following code should not be needed. + if cleanhost: + kinit_admin(master) + master.run_command(["ipa", "host-del", "--updatedns", replica.hostname], + raiseonerr=False) 4) This variable is not used + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] 5) Why do you need this + kra_install = ["ipa-kra-install", "-U", "-p", config.dirman_password] when you implemented tasks.install_kra that returns the exactly the same result? 6) PEP8 ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 blank lines, found 1 ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 blank lines, found 1 ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 blank lines, found 1 ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 blank lines, found 1 ./ipatests/test_integration/tasks.py:950:80: E501 line too long (80 > 79 characters) ./ipatests/test_integration/test_replica_promotion.py:29:80: E501 line too long (85 > 79 characters) ./ipatests/test_integration/test_replica_promotion.py:64:80: E501 line too long (85 > 79 characters) ./ipatests/test_integration/test_replica_promotion.py:67:80: E501 line too long (88 > 79 characters) ./ipatests/test_integration/test_replica_promotion.py:93:80: E501 line too long (80 > 79 characters) ./ipatests/test_integration/test_replica_promotion.py:94:80: E501 line too long (83 > 79 characters) ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 line too long (81 > 79 characters) ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 line too long (80 > 79 characters) ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 line too long (82 > 79 characters) ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 line too long (80 > 79 characters) 7) Why this must be stored in instance? IMO to have it stored as local variable is perfect TestKRAInstall, TestCAInstall self.replica1_filename = tasks.get_replica_filename(replica1) self.replica2_filename = tasks.get_replica_filename(replica2) From mkosek at redhat.com Tue Dec 1 15:10:04 2015 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 1 Dec 2015 16:10:04 +0100 Subject: [Freeipa-devel] [PATCH 556-557] Add option to disable setkeytab extended operations In-Reply-To: <1448978341.9040.5.camel@redhat.com> References: <1448394152.29102.10.camel@redhat.com> <1448395038.29102.11.camel@redhat.com> <1448399848.29102.13.camel@redhat.com> <56555AC7.2000908@redhat.com> <5655BF5B.6020405@redhat.com> <1448462853.29102.44.camel@redhat.com> <1448908709.3747.93.camel@redhat.com> <20151201081145.GJ9605@redhat.com> <1448977085.9040.1.camel@redhat.com> <565DA3DE.8000109@redhat.com> <1448978341.9040.5.camel@redhat.com> Message-ID: <565DB84C.2010401@redhat.com> On 12/01/2015 02:59 PM, Simo Sorce wrote: > On Tue, 2015-12-01 at 14:42 +0100, Martin Kosek wrote: >> On 12/01/2015 02:38 PM, Simo Sorce wrote: >>> On Tue, 2015-12-01 at 10:11 +0200, Alexander Bokovoy wrote: >>>> On Mon, 30 Nov 2015, Simo Sorce wrote: >>>>> On Wed, 2015-11-25 at 09:47 -0500, Simo Sorce wrote: >>>>>> On Wed, 2015-11-25 at 09:02 -0500, Rob Crittenden wrote: >>>>>>> Jan Cholasta wrote: >>>>>>>> On 24.11.2015 22:17, Simo Sorce wrote: >>>>>>>>> On Tue, 2015-11-24 at 14:57 -0500, Simo Sorce wrote: >>>>>>>>>> On Tue, 2015-11-24 at 14:42 -0500, Simo Sorce wrote: >>>>>>>>>>> Since some time we use the getkeytab operation to fetch keytabs on >>>>>>>>>>> newer >>>>>>>>>>> clients. According to bug #232 setkeytab can be used to circumvent >>>>>>>>>>> password quality controls so it needs to be slowly retired. >>>>>>>>>>> >>>>>>>>>>> The attached patches implement #5485 in 2 parts. >>>>>>>>>>> >>>>>>>>>>> The first introduces the option DisableSetKeytab which globally >>>>>>>>>>> disables >>>>>>>>>>> the setkeytab extended operation. This is set to false by default for >>>>>>>>>>> backwards compatibility. >>>>>>>>>>> >>>>>>>>>>> The second introduces an option called DisableUserSetKeytab, which is >>>>>>>>>>> active by default in new installs (but not in upgraded ones), and only >>>>>>>>>>> disables the use of setkeytab for ipa suers, but not for >>>>>>>>>>> hosts/services. >>>>>>>>>>> This is because user's are the ones that may abuse the interface to >>>>>>>>>>> escape password policies and users also normally do not acquire >>>>>>>>>>> keytabs, >>>>>>>>>>> so it is a safe bet to disable just them by default in new installs. >>>>>>>>>>> >>>>>>>>>>> (Testing in progress) >>>>>>>>>> >>>>>>>>>> Tested and working as expected. >>>>>>>>> >>>>>>>>> I realized that adding options to ipaConfig require to add them in the >>>>>>>>> UI as well, attached patches add options in API.txt and config.py >>>>>>>>> Make now complain I should change API Major or Minor, but it is not >>>>>>>>> clear to me why given this are additional values and no real change or >>>>>>>>> new function is introduced. What's the recommendation ? >>>>>>>> >>>>>>>> When does make complain? It is supposed to complain only when API.txt >>>>>>>> does not match code. >>>>>>>> >>>>>>>> Anyway, we usually bump minor version even for backward compatible >>>>>>>> changes, see e.g. commit 9549a59. >>>>>>>> >>>>>>> >>>>>>> The point of API.txt (and the heavy client) was to save a round-trip. >>>>>>> Being able to pass in an invalid option would void that rule hence >>>>>>> having to update the API when new values are added. >>>>>>> >>>>>>> rob >>>>>> >>>>>> Ok added version change to the second patch (so we bump it only once >>>>>> given these are basically related changes. >>>>> >>>>> Bump, is this ok ? >>>> This patch is fine but please fix setkeytab use in ipa-sam before >>>> committing this patch. >>> >>> This patch does not disable setkeytab yet, so it can go in right away >>> (it blocks other patches already). We have a bug to change ipa-sam, >>> should we mark it blocker for 4.4 ? >> >> We also have to fix the permission to change keytab, so that it uses the new >> style (https://fedorahosted.org/freeipa/ticket/5487). I would actually make >> this ticket and the ipa-sam ticket a blocker for this patch set. >> >> Otherwise you are actually introducing a switch that breaks FreeIPA as some of >> it's core server functions still use the old method. > > The point of this patchset is to introduce the switch early so that > current server will support the off switch when newer servers down the > road are ready to flip it. The defaults are still to allow these > operations for services/hosts. I still do not get the logic about an early switch. Now, if switch is turned on, FreeIPA server breaks on several places. I would really rather expect the switch to be introduced when the server itself supports it. Then, admin would enable it when the clients are sufficiently updated and can use the new method. Why would admin want to enable the switch early if it breaks FreeIPA some of the FreeIPA servers? Permission can be upgraded in newer version and get replicated, that's fine. But ipa-sam would be still broken on this old FreeIPA server. > If you flip it right now you'll break stuff, including all older clients > on "Enterprise" distributions, so this switch will not be flipped soon. I definitely do not want to flip it now. From tbabej at redhat.com Tue Dec 1 15:19:23 2015 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 1 Dec 2015 16:19:23 +0100 Subject: [Freeipa-devel] [PATCH 0385] replicainstall: Add possiblity to install client in one In-Reply-To: <565D4A1F.30907@redhat.com> References: <5652FD85.3050105@redhat.com> <5653343B.6020200@redhat.com> <56547AB1.5030405@redhat.com> <56556AC2.1090302@redhat.com> <5656FC28.6000009@redhat.com> <56570AEF.7040607@redhat.com> <565C0806.5090103@redhat.com> <565C3236.5010703@redhat.com> <1448907476.3747.89.camel@redhat.com> <565D4A1F.30907@redhat.com> Message-ID: <565DBA7B.6020006@redhat.com> On 12/01/2015 08:19 AM, Jan Cholasta wrote: > On 30.11.2015 19:17, Simo Sorce wrote: >> On Mon, 2015-11-30 at 12:25 +0100, Tomas Babej wrote: >>> + # Perform only if we have the necessary options >>> + if not any([installer.admin_password, installer.keytab]): >>> + sys.exit("IPA client is not configured on this system.\n" >>> + "You must use a replica file or join the system " >>> + "either by using by running 'ipa-client-install'. " >>> + "Alternatively, you may specify enrollment related >>> options " >>> + "directly, see man ipa-replica-install.") >>> + >> >> There is a typo "either by using by " >> >> Also this seem to be run in promote_check, so you should not mention >> replica files, as promotion can only be run at domain level 1 where >> replica files cannot be used. > > One more thing from me: admin password should be passed to > ipa-client-install through stdin. Apply the following changes (tested > and working) to make it so: > > args.extend(["--hostname", installer.host_name]) > > if installer.admin_password: > - args.extend(["--password", installer.admin_password]) > args.extend(["--principal", installer.principal or "admin"]) > if installer.keytab: > args.extend(["--keytab", installer.keytab]) > @@ -792,7 +791,13 @@ def ensure_enrolled(installer): > args.append("--no-sshd") > if installer.mkhomedir: > args.append("--mkhomedir") > - ipautil.run(args) > + > + if installer.admin_password: > + stdin = installer.admin_password > + else: > + stdin = None > + > + ipautil.run(args, stdin=stdin) > except Exception as e: > sys.exit("Configuration of client side components failed!\n" > "ipa-client-install returned: " + str(e)) > Both Simo's and Jan's suggestions make sense, thanks. Updated patch attached. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0385-6-replicainstall-Add-possiblity-to-install-client-in-o.patch Type: text/x-patch Size: 6739 bytes Desc: not available URL: From abokovoy at redhat.com Tue Dec 1 15:20:42 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 1 Dec 2015 17:20:42 +0200 Subject: [Freeipa-devel] [PATCH 556-557] Add option to disable setkeytab extended operations In-Reply-To: <565DB84C.2010401@redhat.com> References: <1448399848.29102.13.camel@redhat.com> <56555AC7.2000908@redhat.com> <5655BF5B.6020405@redhat.com> <1448462853.29102.44.camel@redhat.com> <1448908709.3747.93.camel@redhat.com> <20151201081145.GJ9605@redhat.com> <1448977085.9040.1.camel@redhat.com> <565DA3DE.8000109@redhat.com> <1448978341.9040.5.camel@redhat.com> <565DB84C.2010401@redhat.com> Message-ID: <20151201152042.GQ9605@redhat.com> On Tue, 01 Dec 2015, Martin Kosek wrote: >On 12/01/2015 02:59 PM, Simo Sorce wrote: >> On Tue, 2015-12-01 at 14:42 +0100, Martin Kosek wrote: >>> On 12/01/2015 02:38 PM, Simo Sorce wrote: >>>> On Tue, 2015-12-01 at 10:11 +0200, Alexander Bokovoy wrote: >>>>> On Mon, 30 Nov 2015, Simo Sorce wrote: >>>>>> On Wed, 2015-11-25 at 09:47 -0500, Simo Sorce wrote: >>>>>>> On Wed, 2015-11-25 at 09:02 -0500, Rob Crittenden wrote: >>>>>>>> Jan Cholasta wrote: >>>>>>>>> On 24.11.2015 22:17, Simo Sorce wrote: >>>>>>>>>> On Tue, 2015-11-24 at 14:57 -0500, Simo Sorce wrote: >>>>>>>>>>> On Tue, 2015-11-24 at 14:42 -0500, Simo Sorce wrote: >>>>>>>>>>>> Since some time we use the getkeytab operation to fetch keytabs on >>>>>>>>>>>> newer >>>>>>>>>>>> clients. According to bug #232 setkeytab can be used to circumvent >>>>>>>>>>>> password quality controls so it needs to be slowly retired. >>>>>>>>>>>> >>>>>>>>>>>> The attached patches implement #5485 in 2 parts. >>>>>>>>>>>> >>>>>>>>>>>> The first introduces the option DisableSetKeytab which globally >>>>>>>>>>>> disables >>>>>>>>>>>> the setkeytab extended operation. This is set to false by default for >>>>>>>>>>>> backwards compatibility. >>>>>>>>>>>> >>>>>>>>>>>> The second introduces an option called DisableUserSetKeytab, which is >>>>>>>>>>>> active by default in new installs (but not in upgraded ones), and only >>>>>>>>>>>> disables the use of setkeytab for ipa suers, but not for >>>>>>>>>>>> hosts/services. >>>>>>>>>>>> This is because user's are the ones that may abuse the interface to >>>>>>>>>>>> escape password policies and users also normally do not acquire >>>>>>>>>>>> keytabs, >>>>>>>>>>>> so it is a safe bet to disable just them by default in new installs. >>>>>>>>>>>> >>>>>>>>>>>> (Testing in progress) >>>>>>>>>>> >>>>>>>>>>> Tested and working as expected. >>>>>>>>>> >>>>>>>>>> I realized that adding options to ipaConfig require to add them in the >>>>>>>>>> UI as well, attached patches add options in API.txt and config.py >>>>>>>>>> Make now complain I should change API Major or Minor, but it is not >>>>>>>>>> clear to me why given this are additional values and no real change or >>>>>>>>>> new function is introduced. What's the recommendation ? >>>>>>>>> >>>>>>>>> When does make complain? It is supposed to complain only when API.txt >>>>>>>>> does not match code. >>>>>>>>> >>>>>>>>> Anyway, we usually bump minor version even for backward compatible >>>>>>>>> changes, see e.g. commit 9549a59. >>>>>>>>> >>>>>>>> >>>>>>>> The point of API.txt (and the heavy client) was to save a round-trip. >>>>>>>> Being able to pass in an invalid option would void that rule hence >>>>>>>> having to update the API when new values are added. >>>>>>>> >>>>>>>> rob >>>>>>> >>>>>>> Ok added version change to the second patch (so we bump it only once >>>>>>> given these are basically related changes. >>>>>> >>>>>> Bump, is this ok ? >>>>> This patch is fine but please fix setkeytab use in ipa-sam before >>>>> committing this patch. >>>> >>>> This patch does not disable setkeytab yet, so it can go in right away >>>> (it blocks other patches already). We have a bug to change ipa-sam, >>>> should we mark it blocker for 4.4 ? >>> >>> We also have to fix the permission to change keytab, so that it uses the new >>> style (https://fedorahosted.org/freeipa/ticket/5487). I would actually make >>> this ticket and the ipa-sam ticket a blocker for this patch set. >>> >>> Otherwise you are actually introducing a switch that breaks FreeIPA as some of >>> it's core server functions still use the old method. >> >> The point of this patchset is to introduce the switch early so that >> current server will support the off switch when newer servers down the >> road are ready to flip it. The defaults are still to allow these >> operations for services/hosts. > >I still do not get the logic about an early switch. Now, if switch is turned >on, FreeIPA server breaks on several places. I would really rather expect the >switch to be introduced when the server itself supports it. Then, admin would >enable it when the clients are sufficiently updated and can use the new method. > >Why would admin want to enable the switch early if it breaks FreeIPA some of >the FreeIPA servers? Permission can be upgraded in newer version and get >replicated, that's fine. But ipa-sam would be still broken on this old FreeIPA >server. Old server is not a problem here: ipa-sam only talks to the localhost-based server as we always use ldapi protocol. So if server is running old-behavior FreeIPA, ipa-sam on the same server will work against it. What I don't want to have is a situation where setkeytab is disabled and a new server obeys it but ipa-sam on this new server is not updated and will expectedly fail. We are not that forced to do the change right now in 4.3, given that the default will still be to keep setkeytab, thus we can wait with this patch until ipa-sam is fixed and then push both patches into the closest release, be it 4.3.x (x>=0) or whatever is the next one. -- / Alexander Bokovoy From jcholast at redhat.com Tue Dec 1 15:33:29 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 1 Dec 2015 16:33:29 +0100 Subject: [Freeipa-devel] [PATCH 0385] replicainstall: Add possiblity to install client in one In-Reply-To: <565DBA7B.6020006@redhat.com> References: <5652FD85.3050105@redhat.com> <5653343B.6020200@redhat.com> <56547AB1.5030405@redhat.com> <56556AC2.1090302@redhat.com> <5656FC28.6000009@redhat.com> <56570AEF.7040607@redhat.com> <565C0806.5090103@redhat.com> <565C3236.5010703@redhat.com> <1448907476.3747.89.camel@redhat.com> <565D4A1F.30907@redhat.com> <565DBA7B.6020006@redhat.com> Message-ID: <565DBDC9.7040905@redhat.com> On 1.12.2015 16:19, Tomas Babej wrote: > > > On 12/01/2015 08:19 AM, Jan Cholasta wrote: >> On 30.11.2015 19:17, Simo Sorce wrote: >>> On Mon, 2015-11-30 at 12:25 +0100, Tomas Babej wrote: >>>> + # Perform only if we have the necessary options >>>> + if not any([installer.admin_password, installer.keytab]): >>>> + sys.exit("IPA client is not configured on this system.\n" >>>> + "You must use a replica file or join the system " >>>> + "either by using by running 'ipa-client-install'. " >>>> + "Alternatively, you may specify enrollment related >>>> options " >>>> + "directly, see man ipa-replica-install.") >>>> + >>> >>> There is a typo "either by using by " >>> >>> Also this seem to be run in promote_check, so you should not mention >>> replica files, as promotion can only be run at domain level 1 where >>> replica files cannot be used. >> >> One more thing from me: admin password should be passed to >> ipa-client-install through stdin. Apply the following changes (tested >> and working) to make it so: >> >> args.extend(["--hostname", installer.host_name]) >> >> if installer.admin_password: >> - args.extend(["--password", installer.admin_password]) >> args.extend(["--principal", installer.principal or "admin"]) >> if installer.keytab: >> args.extend(["--keytab", installer.keytab]) >> @@ -792,7 +791,13 @@ def ensure_enrolled(installer): >> args.append("--no-sshd") >> if installer.mkhomedir: >> args.append("--mkhomedir") >> - ipautil.run(args) >> + >> + if installer.admin_password: >> + stdin = installer.admin_password >> + else: >> + stdin = None >> + >> + ipautil.run(args, stdin=stdin) >> except Exception as e: >> sys.exit("Configuration of client side components failed!\n" >> "ipa-client-install returned: " + str(e)) >> > > Both Simo's and Jan's suggestions make sense, thanks. > > Updated patch attached. Thank you, ACK. Pushed to master: 034e76062fd897dc67b5a395735a5471257bfc8b -- Jan Cholasta From pvoborni at redhat.com Tue Dec 1 15:44:27 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 1 Dec 2015 16:44:27 +0100 Subject: [Freeipa-devel] [PATCH 556-557] Add option to disable setkeytab extended operations In-Reply-To: <20151201152042.GQ9605@redhat.com> References: <1448399848.29102.13.camel@redhat.com> <56555AC7.2000908@redhat.com> <5655BF5B.6020405@redhat.com> <1448462853.29102.44.camel@redhat.com> <1448908709.3747.93.camel@redhat.com> <20151201081145.GJ9605@redhat.com> <1448977085.9040.1.camel@redhat.com> <565DA3DE.8000109@redhat.com> <1448978341.9040.5.camel@redhat.com> <565DB84C.2010401@redhat.com> <20151201152042.GQ9605@redhat.com> Message-ID: <565DC05B.1050804@redhat.com> On 12/01/2015 04:20 PM, Alexander Bokovoy wrote: > On Tue, 01 Dec 2015, Martin Kosek wrote: >> On 12/01/2015 02:59 PM, Simo Sorce wrote: >>> On Tue, 2015-12-01 at 14:42 +0100, Martin Kosek wrote: >>>> On 12/01/2015 02:38 PM, Simo Sorce wrote: >>>>> On Tue, 2015-12-01 at 10:11 +0200, Alexander Bokovoy wrote: >>>>>> On Mon, 30 Nov 2015, Simo Sorce wrote: >>>>>>> On Wed, 2015-11-25 at 09:47 -0500, Simo Sorce wrote: >>>>>>>> On Wed, 2015-11-25 at 09:02 -0500, Rob Crittenden wrote: >>>>>>>>> Jan Cholasta wrote: >>>>>>>>>> On 24.11.2015 22:17, Simo Sorce wrote: >>>>>>>>>>> On Tue, 2015-11-24 at 14:57 -0500, Simo Sorce wrote: >>>>>>>>>>>> On Tue, 2015-11-24 at 14:42 -0500, Simo Sorce wrote: >>>>>>>>>>>>> Since some time we use the getkeytab operation to fetch >>>>>>>>>>>>> keytabs on >>>>>>>>>>>>> newer >>>>>>>>>>>>> clients. According to bug #232 setkeytab can be used to >>>>>>>>>>>>> circumvent >>>>>>>>>>>>> password quality controls so it needs to be slowly retired. >>>>>>>>>>>>> >>>>>>>>>>>>> The attached patches implement #5485 in 2 parts. >>>>>>>>>>>>> >>>>>>>>>>>>> The first introduces the option DisableSetKeytab which >>>>>>>>>>>>> globally >>>>>>>>>>>>> disables >>>>>>>>>>>>> the setkeytab extended operation. This is set to false by >>>>>>>>>>>>> default for >>>>>>>>>>>>> backwards compatibility. >>>>>>>>>>>>> >>>>>>>>>>>>> The second introduces an option called >>>>>>>>>>>>> DisableUserSetKeytab, which is >>>>>>>>>>>>> active by default in new installs (but not in upgraded >>>>>>>>>>>>> ones), and only >>>>>>>>>>>>> disables the use of setkeytab for ipa suers, but not for >>>>>>>>>>>>> hosts/services. >>>>>>>>>>>>> This is because user's are the ones that may abuse the >>>>>>>>>>>>> interface to >>>>>>>>>>>>> escape password policies and users also normally do not >>>>>>>>>>>>> acquire >>>>>>>>>>>>> keytabs, >>>>>>>>>>>>> so it is a safe bet to disable just them by default in new >>>>>>>>>>>>> installs. >>>>>>>>>>>>> >>>>>>>>>>>>> (Testing in progress) >>>>>>>>>>>> >>>>>>>>>>>> Tested and working as expected. >>>>>>>>>>> >>>>>>>>>>> I realized that adding options to ipaConfig require to add >>>>>>>>>>> them in the >>>>>>>>>>> UI as well, attached patches add options in API.txt and >>>>>>>>>>> config.py >>>>>>>>>>> Make now complain I should change API Major or Minor, but it >>>>>>>>>>> is not >>>>>>>>>>> clear to me why given this are additional values and no real >>>>>>>>>>> change or >>>>>>>>>>> new function is introduced. What's the recommendation ? >>>>>>>>>> >>>>>>>>>> When does make complain? It is supposed to complain only when >>>>>>>>>> API.txt >>>>>>>>>> does not match code. >>>>>>>>>> >>>>>>>>>> Anyway, we usually bump minor version even for backward >>>>>>>>>> compatible >>>>>>>>>> changes, see e.g. commit 9549a59. >>>>>>>>>> >>>>>>>>> >>>>>>>>> The point of API.txt (and the heavy client) was to save a >>>>>>>>> round-trip. >>>>>>>>> Being able to pass in an invalid option would void that rule hence >>>>>>>>> having to update the API when new values are added. >>>>>>>>> >>>>>>>>> rob >>>>>>>> >>>>>>>> Ok added version change to the second patch (so we bump it only >>>>>>>> once >>>>>>>> given these are basically related changes. >>>>>>> >>>>>>> Bump, is this ok ? >>>>>> This patch is fine but please fix setkeytab use in ipa-sam before >>>>>> committing this patch. >>>>> >>>>> This patch does not disable setkeytab yet, so it can go in right away >>>>> (it blocks other patches already). We have a bug to change ipa-sam, >>>>> should we mark it blocker for 4.4 ? >>>> >>>> We also have to fix the permission to change keytab, so that it uses >>>> the new >>>> style (https://fedorahosted.org/freeipa/ticket/5487). I would >>>> actually make >>>> this ticket and the ipa-sam ticket a blocker for this patch set. >>>> >>>> Otherwise you are actually introducing a switch that breaks FreeIPA >>>> as some of >>>> it's core server functions still use the old method. >>> >>> The point of this patchset is to introduce the switch early so that >>> current server will support the off switch when newer servers down the >>> road are ready to flip it. The defaults are still to allow these >>> operations for services/hosts. >> >> I still do not get the logic about an early switch. Now, if switch is >> turned >> on, FreeIPA server breaks on several places. I would really rather >> expect the >> switch to be introduced when the server itself supports it. Then, >> admin would >> enable it when the clients are sufficiently updated and can use the >> new method. >> >> Why would admin want to enable the switch early if it breaks FreeIPA >> some of >> the FreeIPA servers? Permission can be upgraded in newer version and get >> replicated, that's fine. But ipa-sam would be still broken on this old >> FreeIPA >> server. > Old server is not a problem here: ipa-sam only talks to the > localhost-based server as we always use ldapi protocol. So if server is > running old-behavior FreeIPA, ipa-sam on the same server will work > against it. > > What I don't want to have is a situation where setkeytab is disabled and > a new server obeys it but ipa-sam on this new server is not updated and > will expectedly fail. We are not that forced to do the change right now > in 4.3, given that the default will still be to keep setkeytab, thus we > can wait with this patch until ipa-sam is fixed and then push both > patches into the closest release, be it 4.3.x (x>=0) or whatever is the > next one. > +1, fixing ipa-sam would be then a release blocker for 4.3 which is not desired. Also what about adding support for "ipaProtectedoperation check" for user principals? I'm afraid that forbidding getting user principal might be regarded as a regression which might cause that admins won't set DisableSetKeytab. -- Petr Vobornik From cheimes at redhat.com Tue Dec 1 17:17:04 2015 From: cheimes at redhat.com (Christian Heimes) Date: Tue, 1 Dec 2015 18:17:04 +0100 Subject: [Freeipa-devel] [PATCH 25] Improve error logging for Dogtag subsystem installation Message-ID: <565DD610.5080608@redhat.com> In the case of a failed installation or uninstallation of a Dogtag subsystem, the error output of pkispawn / pkidestroyed are now shown to the user. It makes it more obvious what went wrong and makes it easier to debug a problem. The error handler also attempts to get the full name of the installation / uninstallation log file from stdout. pkispawn and pkidestroy print the absolute name as 'Log file: /path/to/file.log'. The user no longer has to guess the right log file. Example: [1/8]: configuring KRA instance Failed to configure KRA instance: Command ''/usr/sbin/pkispawn' '-s' 'KRA' '-f' '/tmp/tmp1UpbwF'' returned non-zero exit status 1 pkispawn : ERROR ....... PKI subsystem 'KRA' for instance 'pki-tomcat' already exists! See the installation logs and the following files/directories for more information: /var/log/pki/pki-tomcat /var/log/pki/pki-kra-spawn.20151201151735.log [error] RuntimeError: KRA configuration failed. The patch also changes a couple of modules that were using the CalledProcessError exception object from subprocess instead of ipautil. -------------- next part -------------- A non-text attachment was scrubbed... Name: .freeipa-cheimes-0025-Improve-error-logging-for-Dogtag-subsystem-installat.patch.swp Type: application/octet-stream Size: 20480 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From tbabej at redhat.com Tue Dec 1 17:27:29 2015 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 1 Dec 2015 18:27:29 +0100 Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <20151130163256.GA7759@mail.corp.redhat.com> References: <565C3C6F.4040901@redhat.com> <20151130163256.GA7759@mail.corp.redhat.com> Message-ID: <565DD881.60401@redhat.com> On 11/30/2015 05:32 PM, Lukas Slebodnik wrote: > On (30/11/15 13:09), Tomas Babej wrote: >> Hi, >> >> IPA sudo tests worked under the assumption that the clients that >> are executing the sudo commands have their IPs assigned within >> 255.255.255.0 hostmask. >> >> Removes this (invalid) assumption and adds a dynamic detection of >> the hostmask of the IPA client. >> >> https://fedorahosted.org/freeipa/ticket/5501 > >>From e6f1846f0d7d17303e5b30b1643651ba739b2b6c Mon Sep 17 00:00:00 2001 >> From: Tomas Babej >> Date: Mon, 30 Nov 2015 12:53:39 +0100 >> Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on >> hostmask >> >> IPA sudo tests worked under the assumption that the clients that >> are executing the sudo commands have their IPs assigned within >> 255.255.255.0 hostmask. >> >> Removes this (invalid) assumption and adds a dynamic detection of >> the hostmask of the IPA client. >> Thanks, updated patch attached. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0388-2-tests-Add-hostmask-detection-for-sudo-rules-validati.patch Type: text/x-patch Size: 2664 bytes Desc: not available URL: From cheimes at redhat.com Tue Dec 1 17:42:54 2015 From: cheimes at redhat.com (Christian Heimes) Date: Tue, 1 Dec 2015 18:42:54 +0100 Subject: [Freeipa-devel] [PATCH 25] Improve error logging for Dogtag subsystem installation In-Reply-To: <565DD610.5080608@redhat.com> References: <565DD610.5080608@redhat.com> Message-ID: <565DDC1E.5000008@redhat.com> Now the correct patch file instead of a vim swap file... -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-cheimes-0025-Improve-error-logging-for-Dogtag-subsystem-installat.patch Type: text/x-patch Size: 10692 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From pvoborni at redhat.com Tue Dec 1 18:01:14 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 1 Dec 2015 19:01:14 +0100 Subject: [Freeipa-devel] [PATCH] 938 rename topology suffixes to "domain" and "ca" In-Reply-To: <565D54A1.1050509@redhat.com> References: <565C35ED.6090701@redhat.com> <565D54A1.1050509@redhat.com> Message-ID: <565DE06A.9010602@redhat.com> On 12/01/2015 09:04 AM, Jan Cholasta wrote: > On 30.11.2015 12:41, Petr Vobornik wrote: >> see >> https://www.redhat.com/archives/freeipa-devel/2015-November/msg00485.html > > LGTM, but I would s/_SUFFIX/_SUFFIX_NAME/. > Updated patch attached. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0938-1-rename-topology-suffixes-to-domain-and-ca.patch Type: text/x-patch Size: 8494 bytes Desc: not available URL: From simo at redhat.com Tue Dec 1 21:07:32 2015 From: simo at redhat.com (Simo Sorce) Date: Tue, 01 Dec 2015 16:07:32 -0500 Subject: [Freeipa-devel] [PATCH 556-557] Add option to disable setkeytab extended operations In-Reply-To: <20151201152042.GQ9605@redhat.com> References: <1448399848.29102.13.camel@redhat.com> <56555AC7.2000908@redhat.com> <5655BF5B.6020405@redhat.com> <1448462853.29102.44.camel@redhat.com> <1448908709.3747.93.camel@redhat.com> <20151201081145.GJ9605@redhat.com> <1448977085.9040.1.camel@redhat.com> <565DA3DE.8000109@redhat.com> <1448978341.9040.5.camel@redhat.com> <565DB84C.2010401@redhat.com> <20151201152042.GQ9605@redhat.com> Message-ID: <1449004052.9040.17.camel@redhat.com> On Tue, 2015-12-01 at 17:20 +0200, Alexander Bokovoy wrote: > On Tue, 01 Dec 2015, Martin Kosek wrote: > >On 12/01/2015 02:59 PM, Simo Sorce wrote: > >> On Tue, 2015-12-01 at 14:42 +0100, Martin Kosek wrote: > >>> On 12/01/2015 02:38 PM, Simo Sorce wrote: > >>>> On Tue, 2015-12-01 at 10:11 +0200, Alexander Bokovoy wrote: > >>>>> On Mon, 30 Nov 2015, Simo Sorce wrote: > >>>>>> On Wed, 2015-11-25 at 09:47 -0500, Simo Sorce wrote: > >>>>>>> On Wed, 2015-11-25 at 09:02 -0500, Rob Crittenden wrote: > >>>>>>>> Jan Cholasta wrote: > >>>>>>>>> On 24.11.2015 22:17, Simo Sorce wrote: > >>>>>>>>>> On Tue, 2015-11-24 at 14:57 -0500, Simo Sorce wrote: > >>>>>>>>>>> On Tue, 2015-11-24 at 14:42 -0500, Simo Sorce wrote: > >>>>>>>>>>>> Since some time we use the getkeytab operation to fetch keytabs on > >>>>>>>>>>>> newer > >>>>>>>>>>>> clients. According to bug #232 setkeytab can be used to circumvent > >>>>>>>>>>>> password quality controls so it needs to be slowly retired. > >>>>>>>>>>>> > >>>>>>>>>>>> The attached patches implement #5485 in 2 parts. > >>>>>>>>>>>> > >>>>>>>>>>>> The first introduces the option DisableSetKeytab which globally > >>>>>>>>>>>> disables > >>>>>>>>>>>> the setkeytab extended operation. This is set to false by default for > >>>>>>>>>>>> backwards compatibility. > >>>>>>>>>>>> > >>>>>>>>>>>> The second introduces an option called DisableUserSetKeytab, which is > >>>>>>>>>>>> active by default in new installs (but not in upgraded ones), and only > >>>>>>>>>>>> disables the use of setkeytab for ipa suers, but not for > >>>>>>>>>>>> hosts/services. > >>>>>>>>>>>> This is because user's are the ones that may abuse the interface to > >>>>>>>>>>>> escape password policies and users also normally do not acquire > >>>>>>>>>>>> keytabs, > >>>>>>>>>>>> so it is a safe bet to disable just them by default in new installs. > >>>>>>>>>>>> > >>>>>>>>>>>> (Testing in progress) > >>>>>>>>>>> > >>>>>>>>>>> Tested and working as expected. > >>>>>>>>>> > >>>>>>>>>> I realized that adding options to ipaConfig require to add them in the > >>>>>>>>>> UI as well, attached patches add options in API.txt and config.py > >>>>>>>>>> Make now complain I should change API Major or Minor, but it is not > >>>>>>>>>> clear to me why given this are additional values and no real change or > >>>>>>>>>> new function is introduced. What's the recommendation ? > >>>>>>>>> > >>>>>>>>> When does make complain? It is supposed to complain only when API.txt > >>>>>>>>> does not match code. > >>>>>>>>> > >>>>>>>>> Anyway, we usually bump minor version even for backward compatible > >>>>>>>>> changes, see e.g. commit 9549a59. > >>>>>>>>> > >>>>>>>> > >>>>>>>> The point of API.txt (and the heavy client) was to save a round-trip. > >>>>>>>> Being able to pass in an invalid option would void that rule hence > >>>>>>>> having to update the API when new values are added. > >>>>>>>> > >>>>>>>> rob > >>>>>>> > >>>>>>> Ok added version change to the second patch (so we bump it only once > >>>>>>> given these are basically related changes. > >>>>>> > >>>>>> Bump, is this ok ? > >>>>> This patch is fine but please fix setkeytab use in ipa-sam before > >>>>> committing this patch. > >>>> > >>>> This patch does not disable setkeytab yet, so it can go in right away > >>>> (it blocks other patches already). We have a bug to change ipa-sam, > >>>> should we mark it blocker for 4.4 ? > >>> > >>> We also have to fix the permission to change keytab, so that it uses the new > >>> style (https://fedorahosted.org/freeipa/ticket/5487). I would actually make > >>> this ticket and the ipa-sam ticket a blocker for this patch set. > >>> > >>> Otherwise you are actually introducing a switch that breaks FreeIPA as some of > >>> it's core server functions still use the old method. > >> > >> The point of this patchset is to introduce the switch early so that > >> current server will support the off switch when newer servers down the > >> road are ready to flip it. The defaults are still to allow these > >> operations for services/hosts. > > > >I still do not get the logic about an early switch. Now, if switch is turned > >on, FreeIPA server breaks on several places. I would really rather expect the > >switch to be introduced when the server itself supports it. Then, admin would > >enable it when the clients are sufficiently updated and can use the new method. > > > >Why would admin want to enable the switch early if it breaks FreeIPA some of > >the FreeIPA servers? Permission can be upgraded in newer version and get > >replicated, that's fine. But ipa-sam would be still broken on this old FreeIPA > >server. > Old server is not a problem here: ipa-sam only talks to the > localhost-based server as we always use ldapi protocol. So if server is > running old-behavior FreeIPA, ipa-sam on the same server will work against it. > > What I don't want to have is a situation where setkeytab is disabled and > a new server obeys it but ipa-sam on this new server is not updated and > will expectedly fail. We are not that forced to do the change right now > in 4.3, given that the default will still be to keep setkeytab, thus we > can wait with this patch until ipa-sam is fixed and then push both > patches into the closest release, be it 4.3.x (x>=0) or whatever is the > next one. > Ok I have a prototype patch for ipasam, if it works for Alexander tomorrow I'll send it to the list. Meanwhile rebased 556 and 557 as they were conflicting with master where VERSION has changed. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-556-3-Introduce-option-to-disable-the-SetKeytab-exop.patch Type: text/x-patch Size: 5183 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-557-4-Disable-User-s-ability-to-use-the-setkeytab-exop.patch Type: text/x-patch Size: 7556 bytes Desc: not available URL: From simo at redhat.com Tue Dec 1 21:08:57 2015 From: simo at redhat.com (Simo Sorce) Date: Tue, 01 Dec 2015 16:08:57 -0500 Subject: [Freeipa-devel] [PATCH 558] Allow disabling requireing preauth by default for Service Principal Names In-Reply-To: <565DB5E1.8040500@redhat.com> References: <1448400058.29102.17.camel@redhat.com> <56558064.7010600@redhat.com> <1448908922.3747.94.camel@redhat.com> <565DB5E1.8040500@redhat.com> Message-ID: <1449004137.9040.18.camel@redhat.com> On Tue, 2015-12-01 at 15:59 +0100, Martin Babinsky wrote: > On 11/30/2015 07:42 PM, Simo Sorce wrote: > > On Wed, 2015-11-25 at 10:33 +0100, Martin Babinsky wrote: > >> On 11/24/2015 10:20 PM, Simo Sorce wrote: > >>> This addresses #3860, giving admins the option to not require preauth > >>> for Hosts and services. > >>> > >>> I did not add this option by default, although it does reduce the load > >>> on the KDC as well as speed up TGT acquisition for service principal > >>> accounts that acquire TGTs. > >>> > >>> Tested and working as expected (SPNs are not returned PREAUTH_NEEDED > >>> error while normal users are). > >>> > >>> HTH, > >>> Simo. > >>> > >>> > >>> > >> Hi Simo, > >> > >> I was not able to apply the patch on current master branch: > >> > >> """ > >> git am > >> ../review/ssorce/3860/freeipa-simo-558-1-Allow-admins-to-disable-preauth-for-SPNs.patch > >> -3 > >> > >> Applying: Allow admins to disable preauth for SPNs. > >> error: invalid object 100644 a6b4d4349a9ac6de453d9ad3c679ec32add4e43b > >> for 'ipalib/plugins/config.py' > >> fatal: git-write-tree: error building trees > >> Repository lacks necessary blobs to fall back on 3-way merge. > >> Cannot fall back to three-way merge. > >> Patch failed at 0001 Allow admins to disable preauth for SPNs. > >> """ > >> > >> It seems that I nedd to apply some of your other patches first (which one?) > > > > Sorry did not see this question earlier, it requires 556 and 557, I just > > bumped that thread. > > > > Simo. > > > It seems that I need something else, patch 556-2 applies cleanly, but > patch 557-3 fails with http://fpaste.org/296230/89819431/ on both master > and 4-2 branch. > Rebased 556,557 in their thread, and here is the rebase for 558 on top of them. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-558-2-Allow-admins-to-disable-preauth-for-SPNs.patch Type: text/x-patch Size: 6579 bytes Desc: not available URL: From akasurde at redhat.com Wed Dec 2 06:31:32 2015 From: akasurde at redhat.com (Abhijeet Kasurde) Date: Wed, 2 Dec 2015 12:01:32 +0530 Subject: [Freeipa-devel] [PATCH] Fixed typo in ipa stage-user documentation Message-ID: <565E9044.9050401@redhat.com> Hi All, Please find a small fix in 'ipa stage-user' documentation. Thanks, Abhijeet Kasurde -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-akasurde-0001-Fixed-small-typo-in-stage-user-documentation-ipa-4-2.patch Type: text/x-patch Size: 2252 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-akasurde-0001-Fixed-small-typo-in-stage-user-documentation.patch Type: text/x-patch Size: 2252 bytes Desc: not available URL: From jcholast at redhat.com Wed Dec 2 06:58:51 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 2 Dec 2015 07:58:51 +0100 Subject: [Freeipa-devel] [PATCH 0069] ipa-replica-install support caless install with promotion. In-Reply-To: <565DA040.4000509@redhat.com> References: <565710A8.1020306@redhat.com> <56571376.6020209@redhat.com> <565715BB.9020200@redhat.com> <5657FEC2.3080503@redhat.com> <565C7858.1030702@redhat.com> <565DA040.4000509@redhat.com> Message-ID: <565E96AB.8090402@redhat.com> On 1.12.2015 14:27, David Kupka wrote: > On 30/11/15 17:24, Jan Cholasta wrote: >> Hi, >> >> On 27.11.2015 07:57, David Kupka wrote: >>> On 26/11/15 15:22, David Kupka wrote: >>>> On 26/11/15 15:13, David Kupka wrote: >>>>> On 26/11/15 15:01, David Kupka wrote: >>>>>> https://fedorahosted.org/freeipa/ticket/5441 >>>>>> >>>>>> >>>>> Replaced accidentally inserted tabs. >>>>> >>>>> >>>>> >>>> Fixed indentation I screwed up when replacing tabs :-/ >> >> 1) The deprecated --*_pkcs12 and --*_pin aliases should not be supported >> in ipa-replica-install. In ServerCA, inherit the knobs from BaseServerCA rather than BaseServer.ca. The "#pylint: disable=no-member" will no longer be necessary. In ipa-server-install help, there are 2 "certificate system" option groups. This is a shortcoming in the installer framework, which will be addressed in the future. For now, please inherit *all* knobs of BaseServerCA in ServerCA as a workaround. >> >> >> 2) This check from ipa-replica-prepare should be added to >> Replica.__init__() as well: >> >> # If any of the PKCS#12 options are selected, all are required. >> cert_file_req = (options.dirsrv_cert_files, >> options.http_cert_files) >> cert_file_opt = (options.pkinit_cert_files,) >> if any(cert_file_req + cert_file_opt) and not >> all(cert_file_req): >> self.option_parser.error( >> "--dirsrv-cert-file and --http-cert-file are required >> if any " >> "PKCS#12 options are used.") The check is done when replica file is specified in the patch, but it should be done only when replica file is *not* specified. >> 6) Please make the ca_is_enabled argument of install_replica_ds() and >> install_http() mandatory and fill as appropriate when called, it will >> make the code more readable. This bit in install_http() is redundant now: + if ca_is_configured is None: + ca_is_configured = ipautil.file_exists(config.dir + "/cacert.p12") >> >> >> 7) >> >> $ git diff -U0 | pep8 --diff >> ./ipaserver/install/server/replicainstall.py:99:80: E501 line too long >> (82 > 79 characters) >> ./ipaserver/install/server/replicainstall.py:161:80: E501 line too long >> (82 > 79 characters) >> ./ipaserver/install/server/replicainstall.py:1289:13: E265 block comment >> should start with '# ' >> ./ipaserver/install/server/replicainstall.py:1291:17: E125 continuation >> line with same indent as next logical line >> ./ipaserver/install/server/replicainstall.py:1291:17: E128 continuation >> line under-indented for visual indent $ git diff -U0 | pep8 --diff ./ipaserver/install/server/install.py:1142:1: E302 expected 2 blank lines, found 1 ./ipaserver/install/server/install.py:1143:5: E265 block comment should start with '# ' ./ipaserver/install/server/install.py:1160:17: E222 multiple spaces after operator ./ipaserver/install/server/install.py:1288:9: E265 block comment should start with '# ' ./ipaserver/install/server/replicainstall.py:100:80: E501 line too long (82 > 79 characters) ./ipaserver/install/server/replicainstall.py:162:80: E501 line too long (82 > 79 characters) ./ipaserver/install/server/replicainstall.py:697:41: E251 unexpected spaces around keyword / parameter equals ./ipaserver/install/server/replicainstall.py:697:43: E251 unexpected spaces around keyword / parameter equals ./ipaserver/install/server/replicainstall.py:922:9: E129 visually indented line with same indent as next logical line ./ipaserver/install/server/replicainstall.py:925:14: E131 continuation line unaligned for hanging indent ./ipaserver/install/server/replicainstall.py:1345:9: E265 block comment should start with '# ' ./ipaserver/install/server/replicainstall.py:1389:21: E128 continuation line under-indented for visual indent -- Jan Cholasta From tbabej at redhat.com Wed Dec 2 07:27:32 2015 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 2 Dec 2015 08:27:32 +0100 Subject: [Freeipa-devel] [PATCH] Fixed typo in ipa stage-user documentation In-Reply-To: <565E9044.9050401@redhat.com> References: <565E9044.9050401@redhat.com> Message-ID: <565E9D64.3020903@redhat.com> On 12/02/2015 07:31 AM, Abhijeet Kasurde wrote: > Hi All, > > Please find a small fix in 'ipa stage-user' documentation. > > Thanks, > Abhijeet Kasurde > > ACK, Thanks. From tbabej at redhat.com Wed Dec 2 07:29:46 2015 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 2 Dec 2015 08:29:46 +0100 Subject: [Freeipa-devel] [PATCH] Fixed typo in ipa stage-user documentation In-Reply-To: <565E9D64.3020903@redhat.com> References: <565E9044.9050401@redhat.com> <565E9D64.3020903@redhat.com> Message-ID: <565E9DEA.9040804@redhat.com> On 12/02/2015 08:27 AM, Tomas Babej wrote: > On 12/02/2015 07:31 AM, Abhijeet Kasurde wrote: >> Hi All, >> >> Please find a small fix in 'ipa stage-user' documentation. >> >> Thanks, >> Abhijeet Kasurde >> >> > > ACK, Thanks. > Pushed to master: 9a73c20763da42b331b73d7b716a1ea38f00a680 Pushed to ipa-4-2: d96b840730dc30d873a325d2a6790c3c24d4f414 From pspacek at redhat.com Wed Dec 2 07:37:19 2015 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 2 Dec 2015 08:37:19 +0100 Subject: [Freeipa-devel] [PATCH 25] Improve error logging for Dogtag subsystem installation In-Reply-To: <565DDC1E.5000008@redhat.com> References: <565DD610.5080608@redhat.com> <565DDC1E.5000008@redhat.com> Message-ID: <565E9FAF.5030805@redhat.com> On 1.12.2015 18:42, Christian Heimes wrote: > From 33be1f56a64e53d261a1058c4606a7e48c0aac52 Mon Sep 17 00:00:00 2001 > From: Christian Heimes > Date: Tue, 1 Dec 2015 15:49:53 +0100 > Subject: [PATCH 25] Improve error logging for Dogtag subsystem installation > > In the case of a failed installation or uninstallation of a Dogtag > subsystem, the error output of pkispawn / pkidestroyed are now shown to > the user. It makes it more obvious what went wrong and makes it easier > to debug a problem. > > The error handler also attempts to get the full name of the installation > / uninstallation log file from stdout. pkispawn and pkidestroy print the > absolute name as 'Log file: /path/to/file.log'. The user no longer has > to guess the right log file. > > Example: > [1/8]: configuring KRA instance > Failed to configure KRA instance: Command ''/usr/sbin/pkispawn' '-s' > 'KRA' '-f' '/tmp/tmp1UpbwF'' returned non-zero exit status 1 > pkispawn : ERROR ....... PKI subsystem 'KRA' for instance > 'pki-tomcat' already exists! > See the installation logs and the following files/directories for more > information: > /var/log/pki/pki-tomcat > /var/log/pki/pki-kra-spawn.20151201151735.log > [error] RuntimeError: KRA configuration failed. > > The patch also changes a couple of modules that were using > the CalledProcessError exception object from subprocess instead of > ipautil. I'm wondering if ipautil.run() can log stdout and stderr on log level ERROR when return code is non-zero (and log on level DEBUG as usual when return code is zero). IMHO it would be nicer, universal, and does not require any changes in places calling ipautil.run(). What do you think? -- Petr^2 Spacek From pspacek at redhat.com Wed Dec 2 07:40:37 2015 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 2 Dec 2015 08:40:37 +0100 Subject: [Freeipa-devel] [PATCH] Add option to disable setkeytab extended operations In-Reply-To: <20151201110057.GP9605@redhat.com> References: <1448394152.29102.10.camel@redhat.com> <565D5735.7080405@redhat.com> <20151201082128.GL9605@redhat.com> <565D5BA2.7080608@redhat.com> <20151201084720.GM9605@redhat.com> <565D685E.9080704@redhat.com> <20151201095348.GO9605@redhat.com> <20151201110057.GP9605@redhat.com> Message-ID: <565EA075.7010406@redhat.com> On 1.12.2015 12:00, Alexander Bokovoy wrote: > On Tue, 01 Dec 2015, Alexander Bokovoy wrote: >> On Tue, 01 Dec 2015, Petr Spacek wrote: >>> On 1.12.2015 09:47, Alexander Bokovoy wrote: >>>> On Tue, 01 Dec 2015, Petr Spacek wrote: >>>>> On 1.12.2015 09:21, Alexander Bokovoy wrote: >>>>>> On Tue, 01 Dec 2015, Petr Spacek wrote: >>>>>>> On 24.11.2015 20:42, Simo Sorce wrote: >>>>>>>> Since some time we use the getkeytab operation to fetch keytabs on newer >>>>>>>> clients. According to bug #232 setkeytab can be used to circumvent >>>>>>>> password quality controls so it needs to be slowly retired. >>>>>>>> >>>>>>>> The attached patches implement #5485 in 2 parts. >>>>>>>> >>>>>>>> The first introduces the option DisableSetKeytab which globally disables >>>>>>>> the setkeytab extended operation. This is set to false by default for >>>>>>>> backwards compatibility. >>>>>>>> >>>>>>>> The second introduces an option called DisableUserSetKeytab, which is >>>>>>>> active by default in new installs (but not in upgraded ones), and only >>>>>>>> disables the use of setkeytab for ipa suers, but not for hosts/services. >>>>>>>> This is because user's are the ones that may abuse the interface to >>>>>>>> escape password policies and users also normally do not acquire keytabs, >>>>>>>> so it is a safe bet to disable just them by default in new installs. >>>>>>> >>>>>>> On a related note, how this works with plain kadmin & kpasswd protocols? >>>>>> It is unrelated. We don't support principal manipulation via kadmin >>>>>> protocol. >>>>> >>>>> Sure, I know that, but I'm trying to find out if we re-invented the wheel or >>>>> if our wheel has some additional features which cannot be incorporated to >>>>> the >>>>> original wheel :-) >>>> There is no need to incorporate something specific into kadmin protocol, >>>> the problem is with the fact that we don't have access controls within >>>> our KDC driver. It always connects to LDAP server over ldapi as root and >>>> thus maps to cn=Directory Manager which bypasses LDAP ACIs. As such, >>>> this means we'll need to have some access control added to KDC DAL >>>> driver before we can allow kadmin operations on principals. >>>> >>>> Adding those access controls is not an easy feat. >>> >>> Hmm, why is that? We could use LDAP Proxy control (RFC 4370) and let DS to >>> evaluate ACIs as usual. The change in kadmin would be adding LDAP Proxy >>> control with proper DN (i.e. mapping from principal name to DN) to the LDAP >>> requests. >> Not sure how does it help: >> $ ldapsearch -Dcn=Directory\ Manager -W -e >> \!authzid=dn:uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li -b >> uid=admin,cn=users,cn=accounts,dc=vda,dc=li userPassword >> Enter LDAP Password: # extended LDIF >> # >> # LDAPv3 >> # base with scope subtree >> # filter: (objectclass=*) >> # requesting: userPassword # >> >> # admin, users, accounts, vda.li >> dn: uid=admin,cn=users,cn=accounts,dc=vda,dc=li >> userPassword:: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX= >> = >> >> # search result >> search: 2 >> result: 0 Success >> >> # numResponses: 2 >> # numEntries: 1 >> $ ldapsearch -Y GSSAPI -e >> \!authzid=dn:uid=admin,cn=users,cn=accounts,dc=vda,dc=li -b >> uid=admin,cn=users,cn=accounts,dc=vda,dc=li userPassword >> SASL/GSSAPI authentication started >> SASL username: abokovoy at VDA.LI >> SASL SSF: 56 >> SASL data security layer installed. >> # extended LDIF >> # >> # LDAPv3 >> # base with scope subtree >> # filter: (objectclass=*) >> # requesting: userPassword # >> >> # search result >> search: 4 >> result: 0 Success >> >> # numResponses: 1 >> >> As you can see, posing different authzid does not allow to get through >> ACLs unless you were already a directory manager. Which means the >> proxyauth control is not really useful here: >> >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=0 BIND dn="cn=Directory Manager" >> method=128 version=3 >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=0 RESULT err=0 tag=97 nentries=0 >> etime=0 dn="cn=directory manager" >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=1 SRCH >> base="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" scope=2 >> filter="(objectClass=*)" attrs="userPassword" >> authzid="uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li" >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=1 RESULT err=0 tag=101 nentries=1 >> etime=0 notes=U >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=2 UNBIND >> >> and >> >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=0 BIND dn="" method=sasl >> version=3 mech=GSSAPI >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=0 RESULT err=14 tag=97 nentries=0 >> etime=0, SASL bind in progress >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=1 BIND dn="" method=sasl >> version=3 mech=GSSAPI >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=1 RESULT err=14 tag=97 nentries=0 >> etime=0, SASL bind in progress >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=2 BIND dn="" method=sasl >> version=3 mech=GSSAPI >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=2 RESULT err=0 tag=97 nentries=0 >> etime=0 dn="uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li" >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=3 SRCH >> base="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" scope=2 >> filter="(objectClass=*)" attrs="userPassword" >> authzid="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=3 RESULT err=0 tag=101 nentries=0 >> etime=0 notes=U >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=4 UNBIND >> > Ludwig confirmed that ACL plugin short-circuits DM access before > proxy auth checked. I've filed a bug about it: > https://fedorahosted.org/389/ticket/48366 But now back to the original question: Do kadmin/kpasswd support password policies? If it is the case, why it works with kadmin/kpasswd and does not work with our setkeytab? -- Petr^2 Spacek From jcholast at redhat.com Wed Dec 2 07:59:54 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 2 Dec 2015 08:59:54 +0100 Subject: [Freeipa-devel] [PATCH] 0748 Handle encoding for ipautil.run In-Reply-To: <565D83CC.8020802@redhat.com> References: <5638B8F9.6010703@redhat.com> <564D0FD9.4070505@redhat.com> <5652B5AC.2040508@redhat.com> <5652E174.50301@redhat.com> <56548E73.1090105@redhat.com> <565587AE.2070303@redhat.com> <5655CA08.8090701@redhat.com> <565C01FD.7050906@redhat.com> <565D83CC.8020802@redhat.com> Message-ID: <565EA4FA.3020307@redhat.com> On 1.12.2015 12:26, Petr Viktorin wrote: > On 11/30/2015 08:59 AM, Jan Cholasta wrote: >> On 25.11.2015 15:47, Petr Viktorin wrote: >>> On 11/25/2015 11:04 AM, Jan Cholasta wrote: >>>> On 24.11.2015 17:21, Petr Viktorin wrote: >>>>> On 11/23/2015 10:50 AM, Jan Cholasta wrote: >>>>>> On 23.11.2015 07:43, Jan Cholasta wrote: >>>>>>> On 19.11.2015 00:55, Petr Viktorin wrote: >>>>>>>> On 11/03/2015 02:39 PM, Petr Viktorin wrote: >>>>>>>>> Hello, >>>>>>>>> Python 3's strings are Unicode, so data coming to or leaving a >>>>>>>>> Python >>>>>>>>> program needs to be decoded/encoded if it's to be handled as a >>>>>>>>> string. >>>>>>>>> One of the boundaries where encoding is necessary is external >>>>>>>>> programs, >>>>>>>>> specifically, ipautil.run. >>>>>>>>> Unfortunately there's no one set of options that would work for all >>>>>>>>> run() invocations, so I went through them all and specified the >>>>>>>>> stdin/stdout/stderr encoding where necessary. I've also updated the >>>>>>>>> call >>>>>>>>> sites to make it clearer if the return values are used or not. >>>>>>>>> If an encoding is not set, run() will accept/return bytes. (This >>>>>>>>> is a >>>>>>>>> fail-safe setting, since it can't raise errors, and using bytes >>>>>>>>> where >>>>>>>>> strings are expected generally fails loudly in py3.) >>>>>>>>> >>>>>>>>> Note that the changes are not effective under Python 2. >>>>>>>> >>>>>>>> ping, >>>>>>>> Could someone look at this patch? >>>>>>> >>>>>>> Looking. >>>>>> >>>>>> 1) I think a better approach would be to use str for >>>>>> stdin/stdout/stderr >>>>>> by default in both Python 2 and 3, i.e. do nothing in Python 2 and >>>>>> encode/decode using locale.getpreferredencoding() in Python 3. This is >>>>>> consistent with sys.std{in,out,err} in both Python 2 and 3. Using >>>>>> bytes >>>>>> or different encoding should be opt-in. >>>>>> >>>>>> Note that different encoding should be used only if the LC_ALL or LANG >>>>>> variables are overriden in the command's environment. >>>>> >>>>> That would assume the output of *everything* run via ipautil.run can be >>>>> decoded using the locale encoding. Any stray invalid byte would make >>>>> IPA >>>>> crash, even in cases where we don't care about the output. IPA calls >>>>> too >>>>> many weird tools to assume they all output text. >>>> >>>> Such a stray invalid byte may bubble through our code and cause havoc >>>> somewhere else, which is much harder to troubleshoot than a crash in >>>> ipautil.run(), where you can see that it was a misbehaving command that >>>> caused the crash and exactly what command it was. >>> >>> The invalid byte can't bubble through code, because if you don't specify >>> an encoding you get bytes back. You'll get a crash when you try using it >>> as a string. >> >> Invalid bytes *can* bubble through our code, into configuration files >> and other external places. The fact that it has not caused much grief so >> far suggests that it should be safe to use the locale encoding for most >> commands. > > Yes, "suggests" for "most". I'm still wary of using the locale encoding > for everything by default. OK, let's keep bytes as the default, but I still think the locale encoding should be the default when decoding output to text. > >> If we do subscribe to the statement that any command can output invalid >> bytes at any time, I don't see a point in handling encoding in >> ipautil.run() at all - just always return bytes and let the caller worry >> about encoding and handling related errors. > > Right, that's actually the basic idea. > > However, the encoding is two lines of boilerplate code ("if six.PY3" and > the actual encode). I've turned this into a convenience keyword argument > to minimize the repetition. If you always return bytes in both Python 2 and 3, you can always .decode() the output, so there won't be any "if six.PY3". Bearing that in mind, I don't see much difference between adding ", stdout_encoding='ascii'" to the run() call or "stdout = stdout.decode('ascii')" on the line below, except the latter is more explicit and lets you handle decoding errors yourself. > >>> locale.getpreferredencoding() is not used consistently in all software, >>> especially if it's not written in Python. For instance, wget won't >>> magically re-encode the data it fetches for us. It's better to >>> explicitly specify the encoding every time. >> >> I would say it is used consistently in most software used by IPA. The >> wget example is obviously not relevant to this discussion, since it's >> returning external data. >> >> IMO setting the encoding to 'ascii' without also setting LC_ALL=C in the >> environment, as seen frequently in your patch, has even higher >> probability of breaking something than using the locale encoding. > > The patch should use 'ascii' with programs that don't output localized > messages: things like base64-encoded certs or the output of `ip`. > If you see a particular call where that's not the case, it's a bug in > the patch, please point it out. IMO it's better to be safe than sorry, so I would like to see either LC_ALL=C set with every use of 'ascii' encoding, or use the locale encoding instead of 'ascii'. > >>>> If we don't care about output somewhere, we should not capture it there >>>> at all. >>> >>> Then people need to remember to put "capture_output=True" everywhere >>> (except that also disables logging of the output, so we'd need an >>> additional option). >> >> capture_output=True is the default and capture_output=False should be >> set where output is not needed. > > "capture_output=True" being the default would mean that it's still > possible to leave "capture_output=False" out, but ignore the output. It > would make the wrong usage easier to type than the right one, which is > something to avoid. I'm not sure I understand what are you trying to say here. My point is that capture_output=True is currently the default (see for yourself) and it is indeed possible to leave capture_output=False but ignore the output. I believe this is wrong and that you should always have to explicitly request the output to be captured. > So I still prefer the encoding being explicit. > >> IMO the output should always be logged if possible, regardless of >> capture_output value. > > File a bug for that? > -- Jan Cholasta From tbabej at redhat.com Wed Dec 2 08:24:14 2015 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 2 Dec 2015 09:24:14 +0100 Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <565DD881.60401@redhat.com> References: <565C3C6F.4040901@redhat.com> <20151130163256.GA7759@mail.corp.redhat.com> <565DD881.60401@redhat.com> Message-ID: <565EAAAE.8020306@redhat.com> On 12/01/2015 06:27 PM, Tomas Babej wrote: > > > On 11/30/2015 05:32 PM, Lukas Slebodnik wrote: >> On (30/11/15 13:09), Tomas Babej wrote: >>> Hi, >>> >>> IPA sudo tests worked under the assumption that the clients that >>> are executing the sudo commands have their IPs assigned within >>> 255.255.255.0 hostmask. >>> >>> Removes this (invalid) assumption and adds a dynamic detection of >>> the hostmask of the IPA client. >>> >>> https://fedorahosted.org/freeipa/ticket/5501 >> >> >From e6f1846f0d7d17303e5b30b1643651ba739b2b6c Mon Sep 17 00:00:00 2001 >>> From: Tomas Babej >>> Date: Mon, 30 Nov 2015 12:53:39 +0100 >>> Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on >>> hostmask >>> >>> IPA sudo tests worked under the assumption that the clients that >>> are executing the sudo commands have their IPs assigned within >>> 255.255.255.0 hostmask. >>> >>> Removes this (invalid) assumption and adds a dynamic detection of >>> the hostmask of the IPA client. >>> > > Thanks, updated patch attached. > > Tomas > Actually, a small improvement is necessary. Updated patch attached. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0388-3-tests-Add-hostmask-detection-for-sudo-rules-validati.patch Type: text/x-patch Size: 2676 bytes Desc: not available URL: From ofayans at redhat.com Wed Dec 2 08:52:34 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Wed, 2 Dec 2015 09:52:34 +0100 Subject: [Freeipa-devel] [IPAQE][REVIEW-REQUEST][TEST PLAN] Installation tests In-Reply-To: <565878C0.7050509@redhat.com> References: <56535233.1020207@redhat.com> <56570BA8.6080002@redhat.com> <56586322.8070005@redhat.com> <565878C0.7050509@redhat.com> Message-ID: <565EB152.2010008@redhat.com> Hi all, I've updated the test plan according to your feedback. The tests for 'ca' suffix would be added to Topology Plugin testplan during this week. On 11/27/2015 04:37 PM, Martin Basti wrote: > > > On 27.11.2015 15:05, Martin Basti wrote: >> >> >> On 26.11.2015 14:39, Petr Vobornik wrote: >>> On 11/23/2015 06:51 PM, Oleg Fayans wrote: >>>> Hi all, >>>> >>>> Here is a draft of the Replica Promotion test plan >>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>> >>>> >>> >>> == Test case: Unprivileged users are not allowed to enroll and >>> promote clients == >>> User credentials are passed there through -p $principal and -w >>> $password options. It is correct atm because it is required for >>> connection check. But end goal of replica promotion is the avoid it. >>> See https://fedorahosted.org/freeipa/ticket/5497 and >>> https://fedorahosted.org/freeipa/ticket/5498 for more information. >>> >>> == Missing test cases == >>> 1. ipa-replica-install works on CA-less master with with both domain >>> levels >>> 2. ipa-server-install works with --setup-dns option with both domain >>> levels >>> 3. ipa-server-install works with externally signed CA cert with both >>> domain levels >>> 4. ipa-replica-install with options(and their combination): >>> --setup-ca --setup-dns --setup-kra works with both domain levels >>> >>> Note: Not sure if #2 and #3 belongs here, but should be tested. Maybe >>> tests for domain level 0 already exist. >> >> Many of them are already part of some tests. >> >> I wanted to crate install tests in one place with all combination of >> options, but I did not have time for it yet. >> I have a proof of concept test plan I can share it if you want. >> >> Martin >> > Proof of concept test plan: > > Master: > * ipa-server-install > * ipa-server-install --setup-dns > * caless ipa-server-install > * ipa-server-install && ipa-dns-install > * ipa-server-install && ipa-dns-install --dnssec-master > * caless ipa-server-install && ipa-dns-install > * caless ipa-server-install && ipa-ca-install > * caless ipa-server-install && ipa-ca-install && ipa-kra-install > * ipa-server-install && ipa-kra-install > > Replica: > ipa-replica-install > ipa-replica-install --setup-dns > ipa-replica-install --setup-ca > ipa-replica-install --setup-dns --setup-ca > ipa-replica-install --setup-ca --setup-kra > ipa-replica-install --setup-dns --setup-ca --setup-kra > ipa-replica-install && ipa-dns-install > ipa-replica-install && ipa-dns-install --dnssec-master > ipa-replica-install && ipa-ca-install > ipa-replica-install && ipa-ca-install && ipa-kra-install > ipa-replica-install && ipa-dns-install && ipa-ca-install && ipa-kra-install > caless ipa-replica-install && ipa-ca-install > caless ipa-replica-install --setup-dns > caless ipa-replica-install && ipa-dns-install > > These are basic tests, other specialized test like "install KRA first on > replica then on master" should be in separate test suite IMO > > I'm not sure if caless install test should be covered in basic install > tests. > Same for DNSSEC master (this is fuly covered in dnssec tests) > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From mbabinsk at redhat.com Wed Dec 2 08:53:23 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Wed, 2 Dec 2015 09:53:23 +0100 Subject: [Freeipa-devel] [PATCH 0385] replicainstall: Add possiblity to install client in one In-Reply-To: <565DBDC9.7040905@redhat.com> References: <5652FD85.3050105@redhat.com> <5653343B.6020200@redhat.com> <56547AB1.5030405@redhat.com> <56556AC2.1090302@redhat.com> <5656FC28.6000009@redhat.com> <56570AEF.7040607@redhat.com> <565C0806.5090103@redhat.com> <565C3236.5010703@redhat.com> <1448907476.3747.89.camel@redhat.com> <565D4A1F.30907@redhat.com> <565DBA7B.6020006@redhat.com> <565DBDC9.7040905@redhat.com> Message-ID: <565EB183.7010204@redhat.com> On 12/01/2015 04:33 PM, Jan Cholasta wrote: > On 1.12.2015 16:19, Tomas Babej wrote: >> >> >> On 12/01/2015 08:19 AM, Jan Cholasta wrote: >>> On 30.11.2015 19:17, Simo Sorce wrote: >>>> On Mon, 2015-11-30 at 12:25 +0100, Tomas Babej wrote: >>>>> + # Perform only if we have the necessary options >>>>> + if not any([installer.admin_password, installer.keytab]): >>>>> + sys.exit("IPA client is not configured on this system.\n" >>>>> + "You must use a replica file or join the system " >>>>> + "either by using by running 'ipa-client-install'. " >>>>> + "Alternatively, you may specify enrollment related >>>>> options " >>>>> + "directly, see man ipa-replica-install.") >>>>> + >>>> >>>> There is a typo "either by using by " >>>> >>>> Also this seem to be run in promote_check, so you should not mention >>>> replica files, as promotion can only be run at domain level 1 where >>>> replica files cannot be used. >>> >>> One more thing from me: admin password should be passed to >>> ipa-client-install through stdin. Apply the following changes (tested >>> and working) to make it so: >>> >>> args.extend(["--hostname", installer.host_name]) >>> >>> if installer.admin_password: >>> - args.extend(["--password", installer.admin_password]) >>> args.extend(["--principal", installer.principal or >>> "admin"]) >>> if installer.keytab: >>> args.extend(["--keytab", installer.keytab]) >>> @@ -792,7 +791,13 @@ def ensure_enrolled(installer): >>> args.append("--no-sshd") >>> if installer.mkhomedir: >>> args.append("--mkhomedir") >>> - ipautil.run(args) >>> + >>> + if installer.admin_password: >>> + stdin = installer.admin_password >>> + else: >>> + stdin = None >>> + >>> + ipautil.run(args, stdin=stdin) >>> except Exception as e: >>> sys.exit("Configuration of client side components failed!\n" >>> "ipa-client-install returned: " + str(e)) >>> >> >> Both Simo's and Jan's suggestions make sense, thanks. >> >> Updated patch attached. > > Thank you, ACK. > > Pushed to master: 034e76062fd897dc67b5a395735a5471257bfc8b > It would be nice if we could update ipa-replica-install manpage and document all three different ways to set up a replica (ipa-replica-install w/ replica file in domain level 0, ipa-client-install & ipa-replica-install via promotion in domain level 1, single ipa-replica-install w/ client-specific options in domain level 1). Especially in the latter case it was not obvious to me which options should I use to set up client and replica in one go without peeking in the source code. -- Martin^3 Babinsky From tbabej at redhat.com Wed Dec 2 08:58:03 2015 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 2 Dec 2015 09:58:03 +0100 Subject: [Freeipa-devel] [PATCH 0385] replicainstall: Add possiblity to install client in one In-Reply-To: <565EB183.7010204@redhat.com> References: <5652FD85.3050105@redhat.com> <5653343B.6020200@redhat.com> <56547AB1.5030405@redhat.com> <56556AC2.1090302@redhat.com> <5656FC28.6000009@redhat.com> <56570AEF.7040607@redhat.com> <565C0806.5090103@redhat.com> <565C3236.5010703@redhat.com> <1448907476.3747.89.camel@redhat.com> <565D4A1F.30907@redhat.com> <565DBA7B.6020006@redhat.com> <565DBDC9.7040905@redhat.com> <565EB183.7010204@redhat.com> Message-ID: <565EB29B.8020205@redhat.com> On 12/02/2015 09:53 AM, Martin Babinsky wrote: > On 12/01/2015 04:33 PM, Jan Cholasta wrote: >> On 1.12.2015 16:19, Tomas Babej wrote: >>> >>> >>> On 12/01/2015 08:19 AM, Jan Cholasta wrote: >>>> On 30.11.2015 19:17, Simo Sorce wrote: >>>>> On Mon, 2015-11-30 at 12:25 +0100, Tomas Babej wrote: >>>>>> + # Perform only if we have the necessary options >>>>>> + if not any([installer.admin_password, installer.keytab]): >>>>>> + sys.exit("IPA client is not configured on this system.\n" >>>>>> + "You must use a replica file or join the system " >>>>>> + "either by using by running 'ipa-client-install'. " >>>>>> + "Alternatively, you may specify enrollment related >>>>>> options " >>>>>> + "directly, see man ipa-replica-install.") >>>>>> + >>>>> >>>>> There is a typo "either by using by " >>>>> >>>>> Also this seem to be run in promote_check, so you should not mention >>>>> replica files, as promotion can only be run at domain level 1 where >>>>> replica files cannot be used. >>>> >>>> One more thing from me: admin password should be passed to >>>> ipa-client-install through stdin. Apply the following changes (tested >>>> and working) to make it so: >>>> >>>> args.extend(["--hostname", installer.host_name]) >>>> >>>> if installer.admin_password: >>>> - args.extend(["--password", installer.admin_password]) >>>> args.extend(["--principal", installer.principal or >>>> "admin"]) >>>> if installer.keytab: >>>> args.extend(["--keytab", installer.keytab]) >>>> @@ -792,7 +791,13 @@ def ensure_enrolled(installer): >>>> args.append("--no-sshd") >>>> if installer.mkhomedir: >>>> args.append("--mkhomedir") >>>> - ipautil.run(args) >>>> + >>>> + if installer.admin_password: >>>> + stdin = installer.admin_password >>>> + else: >>>> + stdin = None >>>> + >>>> + ipautil.run(args, stdin=stdin) >>>> except Exception as e: >>>> sys.exit("Configuration of client side components failed!\n" >>>> "ipa-client-install returned: " + str(e)) >>>> >>> >>> Both Simo's and Jan's suggestions make sense, thanks. >>> >>> Updated patch attached. >> >> Thank you, ACK. >> >> Pushed to master: 034e76062fd897dc67b5a395735a5471257bfc8b >> > > It would be nice if we could update ipa-replica-install manpage and > document all three different ways to set up a replica > (ipa-replica-install w/ replica file in domain level 0, > ipa-client-install & ipa-replica-install via promotion in domain level > 1, single ipa-replica-install w/ client-specific options in domain level > 1). > > Especially in the latter case it was not obvious to me which options > should I use to set up client and replica in one go without peeking in > the source code. > Yes, a man page update is long due and in the forge already. Tomas From mbabinsk at redhat.com Wed Dec 2 09:45:07 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Wed, 2 Dec 2015 10:45:07 +0100 Subject: [Freeipa-devel] [PATCH 0098-0099] domain level 1 topology checks during IPA server uninstall In-Reply-To: <565DA35D.6010804@redhat.com> References: <564E04BE.1020304@redhat.com> <56586187.5090402@redhat.com> <565C3399.5030109@redhat.com> <565C6E8C.2090900@redhat.com> <565C844B.7050901@redhat.com> <565C8A57.2000305@redhat.com> <565CA4B8.1010701@redhat.com> <565DA35D.6010804@redhat.com> Message-ID: <565EBDA3.40600@redhat.com> On 12/01/2015 02:40 PM, Martin Babinsky wrote: > On 11/30/2015 08:34 PM, Martin Basti wrote: >> >> >> On 30.11.2015 18:41, Martin Babinsky wrote: >>> On 11/30/2015 06:15 PM, Martin Basti wrote: >>>> >>>> >>>> On 30.11.2015 16:43, Martin Babinsky wrote: >>>>> On 11/30/2015 12:31 PM, Jan Cholasta wrote: >>>>>> Hi, >>>>>> >>>>>> On 27.11.2015 14:58, Martin Babinsky wrote: >>>>>>> On 11/19/2015 06:19 PM, Martin Babinsky wrote: >>>>>>>> These two patches fix the following tickets: >>>>>>>> >>>>>>>> https://fedorahosted.org/freeipa/ticket/5377 >>>>>>>> https://fedorahosted.org/freeipa/ticket/5409 >>>>>>>> >>>>>>>> I have added a new option '--ignore-disconnected-topology' which >>>>>>>> forces >>>>>>>> IPA master uninstall despite reported errors in topology. I'm not >>>>>>>> quite >>>>>>>> sure if we want to flood ipa-server-install with uninstall-specific >>>>>>>> options, maybe it is better to skip the check in unattended mode >>>>>>>> and >>>>>>>> just print a warning about disconnected topology and what to do >>>>>>>> about >>>>>>>> it. >>>>>>>> >>>>>>>> I would like to hear your opinions about this. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> Attaching rebased and updated patches. >>>>>> >>>>>> Patch 0098: LGTM >>>>>> >>>>>> >>>>>> Patch 0099: >>>>>> >>>>>> a) This check should be done in Server.__init__() rather than >>>>>> install_check(): >>>>>> >>>>>> + if options.ignore_disconnected_topology: >>>>>> + print("'--ignore-disconnected-topology' is used only >>>>>> during " >>>>>> + "uninstallation") >>>>>> + sys.exit(1) >>>>>> >>>>>> >>>>>> b) s/--ignore-disconnected-topology/--ignore-topology-disconnect/, >>>>>> for >>>>>> consistency with other options, e.g. --no-ui-redirect. >>>>>> >>>>>> Maybe even shorten it to --ignore-topology? But we probably don't >>>>>> want >>>>>> people to use this option much, so it might be better to keep it >>>>>> long? >>>>>> >>>>> I would rather leave it with the long option name, it is more apparent >>>>> what this switch should be around. >>>>>> >>>>>> c) I'm fine with uninstall options, you can remove the TODO: >>>>>> >>>>>> + # TODO: ask jcholast about uninstallation options >>>>>> >>>>>> >>>>>> Honza >>>>>> >>>>> >>>>> Attaching updated patches. >>>>> >>>> NACK >>>> >>>> ipa-server-install --uninstall >>>> >>>> 2015-11-30T17:14:30Z DEBUG Destroyed connection >>>> context.ldap2_140081152041808 >>>> 2015-11-30T17:14:30Z DEBUG Traceback (most recent call last): >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>> line 91, in _handle_exception >>>> super(Continuous, self)._handle_exception(exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 387, in _handle_exception >>>> six.reraise(*exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 439, in _handle_exception >>>> super(ComponentBase, self)._handle_exception(exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 387, in _handle_exception >>>> six.reraise(*exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 355, in __runner >>>> step() >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 352, in >>>> step = lambda: next(self.__gen) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>> line 81, in run_generator_with_yield_from >>>> six.reraise(*exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>> line 59, in run_generator_with_yield_from >>>> value = gen.send(prev_value) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>> line 71, in _uninstall >>>> for nothing in self._uninstaller(self.parent): >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>> line 1409, in main >>>> uninstall_check(self) >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>> line 265, in decorated >>>> func(installer) >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>> line 1140, in uninstall_check >>>> api, masters, options.ignore_disconnected_topology) >>>> AttributeError: 'uninstaller(Server)' object has no attribute >>>> 'ignore_disconnected_topology' >>>> >>>> 2015-11-30T17:14:30Z ERROR 'uninstaller(Server)' object has no >>>> attribute >>>> 'ignore_disconnected_topology' >>>> 2015-11-30T17:14:30Z INFO The ipa-server-install command was successful >>>> >>> >>> Sorry I have failed horribly during option rename. Attaching patch >>> that should actually work. >>> >> functional ACK > Attaching rebased patches reflecting the recent changes in the handling > of managed topology suffixes handling. > > > > Jan had some more suggestions to the patches. Attaching updated version. -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0098.3-extract-domain-level-1-topology-checking-code-from-i.patch Type: text/x-patch Size: 10180 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0099.5-implement-domain-level-1-specific-topology-checks-in.patch Type: text/x-patch Size: 11229 bytes Desc: not available URL: From dkupka at redhat.com Wed Dec 2 10:10:24 2015 From: dkupka at redhat.com (David Kupka) Date: Wed, 2 Dec 2015 11:10:24 +0100 Subject: [Freeipa-devel] [PATCH 0069] ipa-replica-install support caless install with promotion. In-Reply-To: <565E96AB.8090402@redhat.com> References: <565710A8.1020306@redhat.com> <56571376.6020209@redhat.com> <565715BB.9020200@redhat.com> <5657FEC2.3080503@redhat.com> <565C7858.1030702@redhat.com> <565DA040.4000509@redhat.com> <565E96AB.8090402@redhat.com> Message-ID: <565EC390.5060806@redhat.com> On 02/12/15 07:58, Jan Cholasta wrote: > On 1.12.2015 14:27, David Kupka wrote: >> On 30/11/15 17:24, Jan Cholasta wrote: >>> Hi, >>> >>> On 27.11.2015 07:57, David Kupka wrote: >>>> On 26/11/15 15:22, David Kupka wrote: >>>>> On 26/11/15 15:13, David Kupka wrote: >>>>>> On 26/11/15 15:01, David Kupka wrote: >>>>>>> https://fedorahosted.org/freeipa/ticket/5441 >>>>>>> >>>>>>> >>>>>> Replaced accidentally inserted tabs. >>>>>> >>>>>> >>>>>> >>>>> Fixed indentation I screwed up when replacing tabs :-/ >>> >>> 1) The deprecated --*_pkcs12 and --*_pin aliases should not be supported >>> in ipa-replica-install. > > In ServerCA, inherit the knobs from BaseServerCA rather than > BaseServer.ca. The "#pylint: disable=no-member" will no longer be > necessary. > > In ipa-server-install help, there are 2 "certificate system" option > groups. This is a shortcoming in the installer framework, which will be > addressed in the future. For now, please inherit *all* knobs of > BaseServerCA in ServerCA as a workaround. > >>> >>> >>> 2) This check from ipa-replica-prepare should be added to >>> Replica.__init__() as well: >>> >>> # If any of the PKCS#12 options are selected, all are required. >>> cert_file_req = (options.dirsrv_cert_files, >>> options.http_cert_files) >>> cert_file_opt = (options.pkinit_cert_files,) >>> if any(cert_file_req + cert_file_opt) and not >>> all(cert_file_req): >>> self.option_parser.error( >>> "--dirsrv-cert-file and --http-cert-file are required >>> if any " >>> "PKCS#12 options are used.") > > The check is done when replica file is specified in the patch, but it > should be done only when replica file is *not* specified. > >>> 6) Please make the ca_is_enabled argument of install_replica_ds() and >>> install_http() mandatory and fill as appropriate when called, it will >>> make the code more readable. > > This bit in install_http() is redundant now: > > + if ca_is_configured is None: > + ca_is_configured = ipautil.file_exists(config.dir + "/cacert.p12") > >>> >>> >>> 7) >>> >>> $ git diff -U0 | pep8 --diff >>> ./ipaserver/install/server/replicainstall.py:99:80: E501 line too long >>> (82 > 79 characters) >>> ./ipaserver/install/server/replicainstall.py:161:80: E501 line too long >>> (82 > 79 characters) >>> ./ipaserver/install/server/replicainstall.py:1289:13: E265 block comment >>> should start with '# ' >>> ./ipaserver/install/server/replicainstall.py:1291:17: E125 continuation >>> line with same indent as next logical line >>> ./ipaserver/install/server/replicainstall.py:1291:17: E128 continuation >>> line under-indented for visual indent > > $ git diff -U0 | pep8 --diff > ./ipaserver/install/server/install.py:1142:1: E302 expected 2 blank > lines, found 1 > ./ipaserver/install/server/install.py:1143:5: E265 block comment should > start with '# ' > ./ipaserver/install/server/install.py:1160:17: E222 multiple spaces > after operator > ./ipaserver/install/server/install.py:1288:9: E265 block comment should > start with '# ' > ./ipaserver/install/server/replicainstall.py:100:80: E501 line too long > (82 > 79 characters) > ./ipaserver/install/server/replicainstall.py:162:80: E501 line too long > (82 > 79 characters) > ./ipaserver/install/server/replicainstall.py:697:41: E251 unexpected > spaces around keyword / parameter equals > ./ipaserver/install/server/replicainstall.py:697:43: E251 unexpected > spaces around keyword / parameter equals > ./ipaserver/install/server/replicainstall.py:922:9: E129 visually > indented line with same indent as next logical line > ./ipaserver/install/server/replicainstall.py:925:14: E131 continuation > line unaligned for hanging indent > ./ipaserver/install/server/replicainstall.py:1345:9: E265 block comment > should start with '# ' > ./ipaserver/install/server/replicainstall.py:1389:21: E128 continuation > line under-indented for visual indent > > Thanks, updated patch attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0069.4-ipa-replica-install-support-caless-install-with-prom.patch Type: text/x-patch Size: 21379 bytes Desc: not available URL: From tbabej at redhat.com Wed Dec 2 10:30:31 2015 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 2 Dec 2015 11:30:31 +0100 Subject: [Freeipa-devel] [PATCH 0389] translations: Update ipa.pot file Message-ID: <565EC847.1030406@redhat.com> Hi, the attached patch updates the translations in the master branch. There aren't much string changes between 4.2 and master, most of the bulk of the patch are changed references to the string locations. Thanks, Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0389-translations-Update-ipa.pot-file.patch Type: text/x-patch Size: 403325 bytes Desc: not available URL: From mbasti at redhat.com Wed Dec 2 10:44:36 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 2 Dec 2015 11:44:36 +0100 Subject: [Freeipa-devel] [IPAQE][REVIEW-REQUEST][TEST PLAN] Installation tests In-Reply-To: <565EB152.2010008@redhat.com> References: <56535233.1020207@redhat.com> <56570BA8.6080002@redhat.com> <56586322.8070005@redhat.com> <565878C0.7050509@redhat.com> <565EB152.2010008@redhat.com> Message-ID: <565ECB94.90202@redhat.com> On 02.12.2015 09:52, Oleg Fayans wrote: > Hi all, > > I've updated the test plan according to your feedback. The tests for > 'ca' suffix would be added to Topology Plugin testplan during this week. > > > > On 11/27/2015 04:37 PM, Martin Basti wrote: >> >> >> On 27.11.2015 15:05, Martin Basti wrote: >>> >>> >>> On 26.11.2015 14:39, Petr Vobornik wrote: >>>> On 11/23/2015 06:51 PM, Oleg Fayans wrote: >>>>> Hi all, >>>>> >>>>> Here is a draft of the Replica Promotion test plan >>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>> >>>>> >>>> >>>> == Test case: Unprivileged users are not allowed to enroll and >>>> promote clients == >>>> User credentials are passed there through -p $principal and -w >>>> $password options. It is correct atm because it is required for >>>> connection check. But end goal of replica promotion is the avoid it. >>>> See https://fedorahosted.org/freeipa/ticket/5497 and >>>> https://fedorahosted.org/freeipa/ticket/5498 for more information. >>>> >>>> == Missing test cases == >>>> 1. ipa-replica-install works on CA-less master with with both domain >>>> levels >>>> 2. ipa-server-install works with --setup-dns option with both domain >>>> levels >>>> 3. ipa-server-install works with externally signed CA cert with both >>>> domain levels >>>> 4. ipa-replica-install with options(and their combination): >>>> --setup-ca --setup-dns --setup-kra works with both domain levels >>>> >>>> Note: Not sure if #2 and #3 belongs here, but should be tested. Maybe >>>> tests for domain level 0 already exist. >>> >>> Many of them are already part of some tests. >>> >>> I wanted to crate install tests in one place with all combination of >>> options, but I did not have time for it yet. >>> I have a proof of concept test plan I can share it if you want. >>> >>> Martin >>> >> Proof of concept test plan: >> >> Master: >> * ipa-server-install >> * ipa-server-install --setup-dns >> * caless ipa-server-install >> * ipa-server-install && ipa-dns-install >> * ipa-server-install && ipa-dns-install --dnssec-master >> * caless ipa-server-install && ipa-dns-install >> * caless ipa-server-install && ipa-ca-install >> * caless ipa-server-install && ipa-ca-install && ipa-kra-install >> * ipa-server-install && ipa-kra-install >> >> Replica: >> ipa-replica-install >> ipa-replica-install --setup-dns >> ipa-replica-install --setup-ca >> ipa-replica-install --setup-dns --setup-ca >> ipa-replica-install --setup-ca --setup-kra >> ipa-replica-install --setup-dns --setup-ca --setup-kra >> ipa-replica-install && ipa-dns-install >> ipa-replica-install && ipa-dns-install --dnssec-master >> ipa-replica-install && ipa-ca-install >> ipa-replica-install && ipa-ca-install && ipa-kra-install >> ipa-replica-install && ipa-dns-install && ipa-ca-install && >> ipa-kra-install >> caless ipa-replica-install && ipa-ca-install >> caless ipa-replica-install --setup-dns >> caless ipa-replica-install && ipa-dns-install >> >> These are basic tests, other specialized test like "install KRA first on >> replica then on master" should be in separate test suite IMO >> >> I'm not sure if caless install test should be covered in basic install >> tests. >> Same for DNSSEC master (this is fuly covered in dnssec tests) >> > Hello, as I wrote above, the install tests should be in *separated* test plan, and should cover only ability to be installed, not any other functional tests, we have functional tests separated (vault, dnssec, ...) Also is need to cover multiple variation of options, not just install it with all options (--setup-dns, --setup-ca, --setup-kra) Your: Test case: ipa-server-install works with all supported options under domain level 0 Test case: ipa-server-install works with all supported options under domain level 1 Should not be part of replica promotion testing. Also I do not think that you should test DNSSEC in these tests, DNSSEC test are almost fully covered in DNSSEC integration test, it is just enough to run it on both domain levels. If you disagree to have separate "Installation test suite" please write the reason why it should be a part of replica pro motion test plan. Test case: ipa-ca install and ipa-kra-install work on master and replica under domain level 0 In this test, to be able install CA on master server, master has to be installed as CA-less. Shouldn't be there: expecting failure as result? In case we will create basic install test we may exclude testing ipa-ca-install and ipa-kra-install from this testsuite. Also I do not see reason why DNSSEC should be tested there. Shouldn't be ipa-kra-install and uninstall a separate test case? Martin From mbasti at redhat.com Wed Dec 2 11:20:56 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 2 Dec 2015 12:20:56 +0100 Subject: [Freeipa-devel] Testing replication topologies Message-ID: <565ED418.4010006@redhat.com> Hello all, due to recent bug I found https://fedorahosted.org/freeipa/ticket/5506 I realized we do not have testing of complex topologies. On the other hand, test framework is ready and allows to testing 5 different topologies, can we create test which will test those 5 topologies with for example 4-5 replicas? (read test_topologies.py there are examples of topologies) Martin From mkosek at redhat.com Wed Dec 2 11:24:56 2015 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 2 Dec 2015 12:24:56 +0100 Subject: [Freeipa-devel] [PATCH] 494 Update Contributors.txt Message-ID: <565ED508.2050308@redhat.com> Update .mailmap with misconfigured patch authors since the last feature release. Based on the git history, add new Development contributors. -- Martin Kosek Associate Manager, Software Engineering - Identity Management Team Red Hat, Inc. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-494-update-contributors.txt.patch Type: text/x-patch Size: 3114 bytes Desc: not available URL: From pvoborni at redhat.com Wed Dec 2 11:34:38 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 2 Dec 2015 12:34:38 +0100 Subject: [Freeipa-devel] [PATCH] 494 Update Contributors.txt In-Reply-To: <565ED508.2050308@redhat.com> References: <565ED508.2050308@redhat.com> Message-ID: <565ED74E.2010803@redhat.com> On 12/02/2015 12:24 PM, Martin Kosek wrote: > Update .mailmap with misconfigured patch authors since the last > feature release. Based on the git history, add new Development > contributors. > > > ACK pushed to master 4a75a5f7ffd4f5060e84d04e5806b84e5605ddec -- Petr Vobornik From mbasti at redhat.com Wed Dec 2 11:35:52 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 2 Dec 2015 12:35:52 +0100 Subject: [Freeipa-devel] [PATCH] 494 Update Contributors.txt In-Reply-To: <565ED508.2050308@redhat.com> References: <565ED508.2050308@redhat.com> Message-ID: <565ED798.2080109@redhat.com> On 02.12.2015 12:24, Martin Kosek wrote: > +Thierry Bordaz > +Thierry Bordaz Do we want this there? From mkosek at redhat.com Wed Dec 2 11:37:56 2015 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 2 Dec 2015 12:37:56 +0100 Subject: [Freeipa-devel] [PATCH] 494 Update Contributors.txt In-Reply-To: <565ED798.2080109@redhat.com> References: <565ED508.2050308@redhat.com> <565ED798.2080109@redhat.com> Message-ID: <565ED814.1060201@redhat.com> On 12/02/2015 12:35 PM, Martin Basti wrote: > > > On 02.12.2015 12:24, Martin Kosek wrote: >> +Thierry Bordaz >> +Thierry Bordaz > > Do we want this there? I do not want it there, but I had to add it since Author field in git was wrong when the patch was pushed. So I blame the original author and the reviewer for making me add this mail alias :-) From pvoborni at redhat.com Wed Dec 2 11:41:45 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 2 Dec 2015 12:41:45 +0100 Subject: [Freeipa-devel] [PATCH 0389] translations: Update ipa.pot file In-Reply-To: <565EC847.1030406@redhat.com> References: <565EC847.1030406@redhat.com> Message-ID: <565ED8F9.8020703@redhat.com> On 12/02/2015 11:30 AM, Tomas Babej wrote: > Hi, > > the attached patch updates the translations in the master branch. > > There aren't much string changes between 4.2 and master, most of the > bulk of the patch are changed references to the string locations. > > Thanks, > Tomas > ACK Pushed to master: f72f8c1ad04847e4d0f24b50c76a583bd6fe5a86 -- Petr Vobornik From cheimes at redhat.com Wed Dec 2 11:51:59 2015 From: cheimes at redhat.com (Christian Heimes) Date: Wed, 2 Dec 2015 12:51:59 +0100 Subject: [Freeipa-devel] [PATCH 25] Improve error logging for Dogtag subsystem installation In-Reply-To: <565E9FAF.5030805@redhat.com> References: <565DD610.5080608@redhat.com> <565DDC1E.5000008@redhat.com> <565E9FAF.5030805@redhat.com> Message-ID: <565EDB5F.4060001@redhat.com> On 2015-12-02 08:37, Petr Spacek wrote: > On 1.12.2015 18:42, Christian Heimes wrote: >> From 33be1f56a64e53d261a1058c4606a7e48c0aac52 Mon Sep 17 00:00:00 2001 >> From: Christian Heimes >> Date: Tue, 1 Dec 2015 15:49:53 +0100 >> Subject: [PATCH 25] Improve error logging for Dogtag subsystem installation >> >> In the case of a failed installation or uninstallation of a Dogtag >> subsystem, the error output of pkispawn / pkidestroyed are now shown to >> the user. It makes it more obvious what went wrong and makes it easier >> to debug a problem. >> >> The error handler also attempts to get the full name of the installation >> / uninstallation log file from stdout. pkispawn and pkidestroy print the >> absolute name as 'Log file: /path/to/file.log'. The user no longer has >> to guess the right log file. >> >> Example: >> [1/8]: configuring KRA instance >> Failed to configure KRA instance: Command ''/usr/sbin/pkispawn' '-s' >> 'KRA' '-f' '/tmp/tmp1UpbwF'' returned non-zero exit status 1 >> pkispawn : ERROR ....... PKI subsystem 'KRA' for instance >> 'pki-tomcat' already exists! >> See the installation logs and the following files/directories for more >> information: >> /var/log/pki/pki-tomcat >> /var/log/pki/pki-kra-spawn.20151201151735.log >> [error] RuntimeError: KRA configuration failed. >> >> The patch also changes a couple of modules that were using >> the CalledProcessError exception object from subprocess instead of >> ipautil. > > I'm wondering if ipautil.run() can log stdout and stderr on log level ERROR > when return code is non-zero (and log on level DEBUG as usual when return code > is zero). > > IMHO it would be nicer, universal, and does not require any changes in places > calling ipautil.run(). I think it's a bit confusing to print out stdout and stderr, because both streams are captured separately. The output is missing its chronological order. subprocess can capture stdout and stderr in the same stream, but then we can't distinguish between output and error output... In case of Dogtag stderr contains the relevant error message. In order to understand the events, that lead to the particular error, a user has to read the log file anyway -- unless you run pkispawn with '-vv' for extra verbosity. But then you get pages over pages of debug output on *stderr*. It's not helpful either. Christian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From pspacek at redhat.com Wed Dec 2 11:54:50 2015 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 2 Dec 2015 12:54:50 +0100 Subject: [Freeipa-devel] [PATCH 25] Improve error logging for Dogtag subsystem installation In-Reply-To: <565EDB5F.4060001@redhat.com> References: <565DD610.5080608@redhat.com> <565DDC1E.5000008@redhat.com> <565E9FAF.5030805@redhat.com> <565EDB5F.4060001@redhat.com> Message-ID: <565EDC0A.5020408@redhat.com> On 2.12.2015 12:51, Christian Heimes wrote: > On 2015-12-02 08:37, Petr Spacek wrote: >> On 1.12.2015 18:42, Christian Heimes wrote: >>> From 33be1f56a64e53d261a1058c4606a7e48c0aac52 Mon Sep 17 00:00:00 2001 >>> From: Christian Heimes >>> Date: Tue, 1 Dec 2015 15:49:53 +0100 >>> Subject: [PATCH 25] Improve error logging for Dogtag subsystem installation >>> >>> In the case of a failed installation or uninstallation of a Dogtag >>> subsystem, the error output of pkispawn / pkidestroyed are now shown to >>> the user. It makes it more obvious what went wrong and makes it easier >>> to debug a problem. >>> >>> The error handler also attempts to get the full name of the installation >>> / uninstallation log file from stdout. pkispawn and pkidestroy print the >>> absolute name as 'Log file: /path/to/file.log'. The user no longer has >>> to guess the right log file. >>> >>> Example: >>> [1/8]: configuring KRA instance >>> Failed to configure KRA instance: Command ''/usr/sbin/pkispawn' '-s' >>> 'KRA' '-f' '/tmp/tmp1UpbwF'' returned non-zero exit status 1 >>> pkispawn : ERROR ....... PKI subsystem 'KRA' for instance >>> 'pki-tomcat' already exists! >>> See the installation logs and the following files/directories for more >>> information: >>> /var/log/pki/pki-tomcat >>> /var/log/pki/pki-kra-spawn.20151201151735.log >>> [error] RuntimeError: KRA configuration failed. >>> >>> The patch also changes a couple of modules that were using >>> the CalledProcessError exception object from subprocess instead of >>> ipautil. >> >> I'm wondering if ipautil.run() can log stdout and stderr on log level ERROR >> when return code is non-zero (and log on level DEBUG as usual when return code >> is zero). >> >> IMHO it would be nicer, universal, and does not require any changes in places >> calling ipautil.run(). > > I think it's a bit confusing to print out stdout and stderr, because > both streams are captured separately. The output is missing its > chronological order. subprocess can capture stdout and stderr in the > same stream, but then we can't distinguish between output and error > output... I do not think it is a problem if these two are clearly marked as such: standard output: %s (if non-empty) stanrard error output: %s (if non-empty) > In case of Dogtag stderr contains the relevant error message. In order > to understand the events, that lead to the particular error, a user has > to read the log file anyway -- unless you run pkispawn with '-vv' for > extra verbosity. But then you get pages over pages of debug output on > *stderr*. It's not helpful either. Sure, I was not talking about that :-) -- Petr^2 Spacek From jcholast at redhat.com Wed Dec 2 12:23:05 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 2 Dec 2015 13:23:05 +0100 Subject: [Freeipa-devel] [PATCH 25] Improve error logging for Dogtag subsystem installation In-Reply-To: <565EDC0A.5020408@redhat.com> References: <565DD610.5080608@redhat.com> <565DDC1E.5000008@redhat.com> <565E9FAF.5030805@redhat.com> <565EDB5F.4060001@redhat.com> <565EDC0A.5020408@redhat.com> Message-ID: <565EE2A9.9000906@redhat.com> On 2.12.2015 12:54, Petr Spacek wrote: > On 2.12.2015 12:51, Christian Heimes wrote: >> On 2015-12-02 08:37, Petr Spacek wrote: >>> On 1.12.2015 18:42, Christian Heimes wrote: >>>> From 33be1f56a64e53d261a1058c4606a7e48c0aac52 Mon Sep 17 00:00:00 2001 >>>> From: Christian Heimes >>>> Date: Tue, 1 Dec 2015 15:49:53 +0100 >>>> Subject: [PATCH 25] Improve error logging for Dogtag subsystem installation >>>> >>>> In the case of a failed installation or uninstallation of a Dogtag >>>> subsystem, the error output of pkispawn / pkidestroyed are now shown to >>>> the user. It makes it more obvious what went wrong and makes it easier >>>> to debug a problem. >>>> >>>> The error handler also attempts to get the full name of the installation >>>> / uninstallation log file from stdout. pkispawn and pkidestroy print the >>>> absolute name as 'Log file: /path/to/file.log'. The user no longer has >>>> to guess the right log file. >>>> >>>> Example: >>>> [1/8]: configuring KRA instance >>>> Failed to configure KRA instance: Command ''/usr/sbin/pkispawn' '-s' >>>> 'KRA' '-f' '/tmp/tmp1UpbwF'' returned non-zero exit status 1 >>>> pkispawn : ERROR ....... PKI subsystem 'KRA' for instance >>>> 'pki-tomcat' already exists! >>>> See the installation logs and the following files/directories for more >>>> information: >>>> /var/log/pki/pki-tomcat >>>> /var/log/pki/pki-kra-spawn.20151201151735.log >>>> [error] RuntimeError: KRA configuration failed. >>>> >>>> The patch also changes a couple of modules that were using >>>> the CalledProcessError exception object from subprocess instead of >>>> ipautil. >>> >>> I'm wondering if ipautil.run() can log stdout and stderr on log level ERROR >>> when return code is non-zero (and log on level DEBUG as usual when return code >>> is zero). >>> >>> IMHO it would be nicer, universal, and does not require any changes in places >>> calling ipautil.run(). >> >> I think it's a bit confusing to print out stdout and stderr, because >> both streams are captured separately. The output is missing its >> chronological order. subprocess can capture stdout and stderr in the >> same stream, but then we can't distinguish between output and error >> output... > > I do not think it is a problem if these two are clearly marked as such: > standard output: %s (if non-empty) > stanrard error output: %s (if non-empty) We do not want to log with level ERROR by default when rc != 0, because some commands generate a *lot* of output. IMO Christian's approach is OK, because it lets the caller decide how to log (or not) stderr on failure. > >> In case of Dogtag stderr contains the relevant error message. In order >> to understand the events, that lead to the particular error, a user has >> to read the log file anyway -- unless you run pkispawn with '-vv' for >> extra verbosity. But then you get pages over pages of debug output on >> *stderr*. It's not helpful either. > > Sure, I was not talking about that :-) > -- Jan Cholasta From pviktori at redhat.com Wed Dec 2 12:38:22 2015 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 2 Dec 2015 13:38:22 +0100 Subject: [Freeipa-devel] [PATCH] 0749 Package ipapython, ipalib, ipaplatform, ipatests for Python 3 In-Reply-To: <565DA287.6030403@redhat.com> References: <563C9810.1050604@redhat.com> <5656E487.8050703@redhat.com> <5658508D.1020105@redhat.com> <565C0FF3.8040406@redhat.com> <565D80B0.9000500@redhat.com> <565DA287.6030403@redhat.com> Message-ID: <565EE63E.6070207@redhat.com> On 12/01/2015 02:37 PM, Jan Cholasta wrote: [...] > /etc/ipa/default.conf is managed by freeipa-client and thus should be owned by it. > > This is a common pattern in other packages (even other FreeIPA > sub-packages) and I don't see any reason not to follow it here as well. OK. After your patch is applied this won't be a problem, though. [...] >>>> - python-ipap11helper has compiled code: with this pulled out, >>>> python3-ipalib can be noarch >>> >>> This is not the goal here, but if you insist on doing it, do it for >>> Python 2 as well. >> >> It is definitely the goal of this patch to make the py3 packages as good >> as possible. That includes making them noarch. > > It is completely unnecessary for the initial py3 support. > > I would rather maintain internal consistency than make the py3 packages > "perfect". > >> On the other hand improving the py2 packages is not a goal of this >> particular patch. > > Which is exactly the reason I have provided patches for py2 packages > myself. As far as I'm concerned, the patches look good, except for consistency the package name should be "python2-ipalib". Unless the process changed, you still need them reviewed by a core FreeIPA developer. >>>> - python3-ipatests is needed if we want to start testing the py3 >>>> packages in CI >>> >>> Right. >>> >>>> >>>> As for new provides, Fedora's Python packaging guidelines say: >>>> >>>> """ >>>> Using a fictional module named "example", the subpackage containing >>>> the python2 version must provide python2-example. This is of course >>>> always the case if the subpackage is named python2-example [...] >>>> If the subpackage has some other name then then Provides: >>>> python2-example >>>> must be added explicitly (but see the %python_provide macro below). >>>> >>>> The python3 subpackage must provide python3-example. However, as the >>>> naming guidelines mandate that the python3 subpackage be named >>>> python3-example, this will happen automatically. >>>> """ >>>> >>>> so I'm now adding Provides for the top-level modules. >>> >>> The goal of this work is to add support for Python 3, not to comply with >>> Fedora packaging guidelines. FreeIPA on Fedora uses its own spec file >>> anyway. >> >> The goal of this patch is to add new packages that support Python 3. >> Yes, the Fedora spec is different, but it's heavily based on the >> upstream one, and this is a good thing. I consider the Fedora guidelines >> the standard in Python RPM packaging. If IPA uses different packaging >> guidelines, can you point me to them? > > FreeIPA never fully complied to Fedora packaging guidelines AFAIK and I > don't see any reason to start now, since nobody seemed to care so far. > Following them in just py3 sub-packages does not improve the state of > FreeIPA as a whole and only brings inconsistency into it, so there's no > benefit in doing it at all. > >>> Again, if you insist on doing this, do it for Python 2 as well. OK, when your patches are ACKed I'll send patches to both improve py2 packaging and add the new packages. -- Petr Viktorin From pspacek at redhat.com Wed Dec 2 12:44:29 2015 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 2 Dec 2015 13:44:29 +0100 Subject: [Freeipa-devel] [PATCH 25] Improve error logging for Dogtag subsystem installation In-Reply-To: <565EE2A9.9000906@redhat.com> References: <565DD610.5080608@redhat.com> <565DDC1E.5000008@redhat.com> <565E9FAF.5030805@redhat.com> <565EDB5F.4060001@redhat.com> <565EDC0A.5020408@redhat.com> <565EE2A9.9000906@redhat.com> Message-ID: <565EE7AD.7010502@redhat.com> On 2.12.2015 13:23, Jan Cholasta wrote: > On 2.12.2015 12:54, Petr Spacek wrote: >> On 2.12.2015 12:51, Christian Heimes wrote: >>> On 2015-12-02 08:37, Petr Spacek wrote: >>>> On 1.12.2015 18:42, Christian Heimes wrote: >>>>> From 33be1f56a64e53d261a1058c4606a7e48c0aac52 Mon Sep 17 00:00:00 2001 >>>>> From: Christian Heimes >>>>> Date: Tue, 1 Dec 2015 15:49:53 +0100 >>>>> Subject: [PATCH 25] Improve error logging for Dogtag subsystem installation >>>>> >>>>> In the case of a failed installation or uninstallation of a Dogtag >>>>> subsystem, the error output of pkispawn / pkidestroyed are now shown to >>>>> the user. It makes it more obvious what went wrong and makes it easier >>>>> to debug a problem. >>>>> >>>>> The error handler also attempts to get the full name of the installation >>>>> / uninstallation log file from stdout. pkispawn and pkidestroy print the >>>>> absolute name as 'Log file: /path/to/file.log'. The user no longer has >>>>> to guess the right log file. >>>>> >>>>> Example: >>>>> [1/8]: configuring KRA instance >>>>> Failed to configure KRA instance: Command ''/usr/sbin/pkispawn' '-s' >>>>> 'KRA' '-f' '/tmp/tmp1UpbwF'' returned non-zero exit status 1 >>>>> pkispawn : ERROR ....... PKI subsystem 'KRA' for instance >>>>> 'pki-tomcat' already exists! >>>>> See the installation logs and the following files/directories for more >>>>> information: >>>>> /var/log/pki/pki-tomcat >>>>> /var/log/pki/pki-kra-spawn.20151201151735.log >>>>> [error] RuntimeError: KRA configuration failed. >>>>> >>>>> The patch also changes a couple of modules that were using >>>>> the CalledProcessError exception object from subprocess instead of >>>>> ipautil. >>>> >>>> I'm wondering if ipautil.run() can log stdout and stderr on log level ERROR >>>> when return code is non-zero (and log on level DEBUG as usual when return >>>> code >>>> is zero). >>>> >>>> IMHO it would be nicer, universal, and does not require any changes in places >>>> calling ipautil.run(). >>> >>> I think it's a bit confusing to print out stdout and stderr, because >>> both streams are captured separately. The output is missing its >>> chronological order. subprocess can capture stdout and stderr in the >>> same stream, but then we can't distinguish between output and error >>> output... >> >> I do not think it is a problem if these two are clearly marked as such: >> standard output: %s (if non-empty) >> stanrard error output: %s (if non-empty) > > We do not want to log with level ERROR by default when rc != 0, because some > commands generate a *lot* of output. I do not agree, but whatever. Somebody needs to review the original Christian's patch. Petr^2 Spacek > IMO Christian's approach is OK, because it lets the caller decide how to log > (or not) stderr on failure. > >> >>> In case of Dogtag stderr contains the relevant error message. In order >>> to understand the events, that lead to the particular error, a user has >>> to read the log file anyway -- unless you run pkispawn with '-vv' for >>> extra verbosity. But then you get pages over pages of debug output on >>> *stderr*. It's not helpful either. >> >> Sure, I was not talking about that :-) From amarecek at redhat.com Wed Dec 2 12:47:56 2015 From: amarecek at redhat.com (=?utf-8?Q?Ale=C5=A1_Mare=C4=8Dek?=) Date: Wed, 2 Dec 2015 07:47:56 -0500 (EST) Subject: [Freeipa-devel] Testing replication topologies In-Reply-To: <565ED418.4010006@redhat.com> References: <565ED418.4010006@redhat.com> Message-ID: <2057444698.22156190.1449060476956.JavaMail.zimbra@redhat.com> Greetings! ----- Original Message ----- > From: "Martin Basti" > To: "freeipa-devel" , "Petr Vobornik" , "Ale? Mare?ek" > > Sent: Wednesday, December 2, 2015 12:20:56 PM > Subject: Testing replication topologies > > Hello all, > > due to recent bug I found https://fedorahosted.org/freeipa/ticket/5506 I > realized we do not have testing of complex topologies. > > On the other hand, test framework is ready and allows to testing 5 > different topologies, can we create test which will test those 5 > topologies with for example 4-5 replicas? We were talking about it in the morning with Petr. So the answer is: yes, we can. Adding Oleg... > > (read test_topologies.py there are examples of topologies) > > Martin > Have a nice day! - alich - From mbabinsk at redhat.com Wed Dec 2 13:08:55 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Wed, 2 Dec 2015 14:08:55 +0100 Subject: [Freeipa-devel] [PATCH 0098-0099] domain level 1 topology checks during IPA server uninstall In-Reply-To: <565EBDA3.40600@redhat.com> References: <564E04BE.1020304@redhat.com> <56586187.5090402@redhat.com> <565C3399.5030109@redhat.com> <565C6E8C.2090900@redhat.com> <565C844B.7050901@redhat.com> <565C8A57.2000305@redhat.com> <565CA4B8.1010701@redhat.com> <565DA35D.6010804@redhat.com> <565EBDA3.40600@redhat.com> Message-ID: <565EED67.3040405@redhat.com> On 12/02/2015 10:45 AM, Martin Babinsky wrote: > On 12/01/2015 02:40 PM, Martin Babinsky wrote: >> On 11/30/2015 08:34 PM, Martin Basti wrote: >>> >>> >>> On 30.11.2015 18:41, Martin Babinsky wrote: >>>> On 11/30/2015 06:15 PM, Martin Basti wrote: >>>>> >>>>> >>>>> On 30.11.2015 16:43, Martin Babinsky wrote: >>>>>> On 11/30/2015 12:31 PM, Jan Cholasta wrote: >>>>>>> Hi, >>>>>>> >>>>>>> On 27.11.2015 14:58, Martin Babinsky wrote: >>>>>>>> On 11/19/2015 06:19 PM, Martin Babinsky wrote: >>>>>>>>> These two patches fix the following tickets: >>>>>>>>> >>>>>>>>> https://fedorahosted.org/freeipa/ticket/5377 >>>>>>>>> https://fedorahosted.org/freeipa/ticket/5409 >>>>>>>>> >>>>>>>>> I have added a new option '--ignore-disconnected-topology' which >>>>>>>>> forces >>>>>>>>> IPA master uninstall despite reported errors in topology. I'm not >>>>>>>>> quite >>>>>>>>> sure if we want to flood ipa-server-install with >>>>>>>>> uninstall-specific >>>>>>>>> options, maybe it is better to skip the check in unattended mode >>>>>>>>> and >>>>>>>>> just print a warning about disconnected topology and what to do >>>>>>>>> about >>>>>>>>> it. >>>>>>>>> >>>>>>>>> I would like to hear your opinions about this. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> Attaching rebased and updated patches. >>>>>>> >>>>>>> Patch 0098: LGTM >>>>>>> >>>>>>> >>>>>>> Patch 0099: >>>>>>> >>>>>>> a) This check should be done in Server.__init__() rather than >>>>>>> install_check(): >>>>>>> >>>>>>> + if options.ignore_disconnected_topology: >>>>>>> + print("'--ignore-disconnected-topology' is used only >>>>>>> during " >>>>>>> + "uninstallation") >>>>>>> + sys.exit(1) >>>>>>> >>>>>>> >>>>>>> b) s/--ignore-disconnected-topology/--ignore-topology-disconnect/, >>>>>>> for >>>>>>> consistency with other options, e.g. --no-ui-redirect. >>>>>>> >>>>>>> Maybe even shorten it to --ignore-topology? But we probably don't >>>>>>> want >>>>>>> people to use this option much, so it might be better to keep it >>>>>>> long? >>>>>>> >>>>>> I would rather leave it with the long option name, it is more >>>>>> apparent >>>>>> what this switch should be around. >>>>>>> >>>>>>> c) I'm fine with uninstall options, you can remove the TODO: >>>>>>> >>>>>>> + # TODO: ask jcholast about uninstallation options >>>>>>> >>>>>>> >>>>>>> Honza >>>>>>> >>>>>> >>>>>> Attaching updated patches. >>>>>> >>>>> NACK >>>>> >>>>> ipa-server-install --uninstall >>>>> >>>>> 2015-11-30T17:14:30Z DEBUG Destroyed connection >>>>> context.ldap2_140081152041808 >>>>> 2015-11-30T17:14:30Z DEBUG Traceback (most recent call last): >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>>> line 91, in _handle_exception >>>>> super(Continuous, self)._handle_exception(exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 387, in _handle_exception >>>>> six.reraise(*exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 439, in _handle_exception >>>>> super(ComponentBase, self)._handle_exception(exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 387, in _handle_exception >>>>> six.reraise(*exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 355, in __runner >>>>> step() >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 352, in >>>>> step = lambda: next(self.__gen) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>> line 81, in run_generator_with_yield_from >>>>> six.reraise(*exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>> line 59, in run_generator_with_yield_from >>>>> value = gen.send(prev_value) >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>>> line 71, in _uninstall >>>>> for nothing in self._uninstaller(self.parent): >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>> >>>>> line 1409, in main >>>>> uninstall_check(self) >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>> >>>>> line 265, in decorated >>>>> func(installer) >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>> >>>>> line 1140, in uninstall_check >>>>> api, masters, options.ignore_disconnected_topology) >>>>> AttributeError: 'uninstaller(Server)' object has no attribute >>>>> 'ignore_disconnected_topology' >>>>> >>>>> 2015-11-30T17:14:30Z ERROR 'uninstaller(Server)' object has no >>>>> attribute >>>>> 'ignore_disconnected_topology' >>>>> 2015-11-30T17:14:30Z INFO The ipa-server-install command was >>>>> successful >>>>> >>>> >>>> Sorry I have failed horribly during option rename. Attaching patch >>>> that should actually work. >>>> >>> functional ACK >> Attaching rebased patches reflecting the recent changes in the handling >> of managed topology suffixes handling. >> >> >> >> > > Jan had some more suggestions to the patches. Attaching updated version. > > > Attaching updated patch 99 with fixed error message. -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0099.6-implement-domain-level-1-specific-topology-checks-in.patch Type: text/x-patch Size: 11234 bytes Desc: not available URL: From mbasti at redhat.com Wed Dec 2 13:10:52 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 2 Dec 2015 14:10:52 +0100 Subject: [Freeipa-devel] [PATCH 0098-0099] domain level 1 topology checks during IPA server uninstall In-Reply-To: <565EED67.3040405@redhat.com> References: <564E04BE.1020304@redhat.com> <56586187.5090402@redhat.com> <565C3399.5030109@redhat.com> <565C6E8C.2090900@redhat.com> <565C844B.7050901@redhat.com> <565C8A57.2000305@redhat.com> <565CA4B8.1010701@redhat.com> <565DA35D.6010804@redhat.com> <565EBDA3.40600@redhat.com> <565EED67.3040405@redhat.com> Message-ID: <565EEDDC.8060304@redhat.com> On 02.12.2015 14:08, Martin Babinsky wrote: > On 12/02/2015 10:45 AM, Martin Babinsky wrote: >> On 12/01/2015 02:40 PM, Martin Babinsky wrote: >>> On 11/30/2015 08:34 PM, Martin Basti wrote: >>>> >>>> >>>> On 30.11.2015 18:41, Martin Babinsky wrote: >>>>> On 11/30/2015 06:15 PM, Martin Basti wrote: >>>>>> >>>>>> >>>>>> On 30.11.2015 16:43, Martin Babinsky wrote: >>>>>>> On 11/30/2015 12:31 PM, Jan Cholasta wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> On 27.11.2015 14:58, Martin Babinsky wrote: >>>>>>>>> On 11/19/2015 06:19 PM, Martin Babinsky wrote: >>>>>>>>>> These two patches fix the following tickets: >>>>>>>>>> >>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5377 >>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5409 >>>>>>>>>> >>>>>>>>>> I have added a new option '--ignore-disconnected-topology' which >>>>>>>>>> forces >>>>>>>>>> IPA master uninstall despite reported errors in topology. I'm >>>>>>>>>> not >>>>>>>>>> quite >>>>>>>>>> sure if we want to flood ipa-server-install with >>>>>>>>>> uninstall-specific >>>>>>>>>> options, maybe it is better to skip the check in unattended mode >>>>>>>>>> and >>>>>>>>>> just print a warning about disconnected topology and what to do >>>>>>>>>> about >>>>>>>>>> it. >>>>>>>>>> >>>>>>>>>> I would like to hear your opinions about this. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> Attaching rebased and updated patches. >>>>>>>> >>>>>>>> Patch 0098: LGTM >>>>>>>> >>>>>>>> >>>>>>>> Patch 0099: >>>>>>>> >>>>>>>> a) This check should be done in Server.__init__() rather than >>>>>>>> install_check(): >>>>>>>> >>>>>>>> + if options.ignore_disconnected_topology: >>>>>>>> + print("'--ignore-disconnected-topology' is used only >>>>>>>> during " >>>>>>>> + "uninstallation") >>>>>>>> + sys.exit(1) >>>>>>>> >>>>>>>> >>>>>>>> b) s/--ignore-disconnected-topology/--ignore-topology-disconnect/, >>>>>>>> for >>>>>>>> consistency with other options, e.g. --no-ui-redirect. >>>>>>>> >>>>>>>> Maybe even shorten it to --ignore-topology? But we probably don't >>>>>>>> want >>>>>>>> people to use this option much, so it might be better to keep it >>>>>>>> long? >>>>>>>> >>>>>>> I would rather leave it with the long option name, it is more >>>>>>> apparent >>>>>>> what this switch should be around. >>>>>>>> >>>>>>>> c) I'm fine with uninstall options, you can remove the TODO: >>>>>>>> >>>>>>>> + # TODO: ask jcholast about uninstallation options >>>>>>>> >>>>>>>> >>>>>>>> Honza >>>>>>>> >>>>>>> >>>>>>> Attaching updated patches. >>>>>>> >>>>>> NACK >>>>>> >>>>>> ipa-server-install --uninstall >>>>>> >>>>>> 2015-11-30T17:14:30Z DEBUG Destroyed connection >>>>>> context.ldap2_140081152041808 >>>>>> 2015-11-30T17:14:30Z DEBUG Traceback (most recent call last): >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>>>> line 91, in _handle_exception >>>>>> super(Continuous, self)._handle_exception(exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 387, in _handle_exception >>>>>> six.reraise(*exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 439, in _handle_exception >>>>>> super(ComponentBase, self)._handle_exception(exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 387, in _handle_exception >>>>>> six.reraise(*exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 355, in __runner >>>>>> step() >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 352, in >>>>>> step = lambda: next(self.__gen) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>> line 81, in run_generator_with_yield_from >>>>>> six.reraise(*exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>> line 59, in run_generator_with_yield_from >>>>>> value = gen.send(prev_value) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>>>> line 71, in _uninstall >>>>>> for nothing in self._uninstaller(self.parent): >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>> >>>>>> >>>>>> line 1409, in main >>>>>> uninstall_check(self) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>> >>>>>> >>>>>> line 265, in decorated >>>>>> func(installer) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>> >>>>>> >>>>>> line 1140, in uninstall_check >>>>>> api, masters, options.ignore_disconnected_topology) >>>>>> AttributeError: 'uninstaller(Server)' object has no attribute >>>>>> 'ignore_disconnected_topology' >>>>>> >>>>>> 2015-11-30T17:14:30Z ERROR 'uninstaller(Server)' object has no >>>>>> attribute >>>>>> 'ignore_disconnected_topology' >>>>>> 2015-11-30T17:14:30Z INFO The ipa-server-install command was >>>>>> successful >>>>>> >>>>> >>>>> Sorry I have failed horribly during option rename. Attaching patch >>>>> that should actually work. >>>>> >>>> functional ACK >>> Attaching rebased patches reflecting the recent changes in the handling >>> of managed topology suffixes handling. >>> >>> >>> >>> >> >> Jan had some more suggestions to the patches. Attaching updated version. >> >> >> > Attaching updated patch 99 with fixed error message. > Pushed to master: b8c619a7139bd7b65caa03b68431e22791ff19bf From mbasti at redhat.com Wed Dec 2 13:19:40 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 2 Dec 2015 14:19:40 +0100 Subject: [Freeipa-devel] [PATCH 0098-0099] domain level 1 topology checks during IPA server uninstall In-Reply-To: <565EEDDC.8060304@redhat.com> References: <564E04BE.1020304@redhat.com> <56586187.5090402@redhat.com> <565C3399.5030109@redhat.com> <565C6E8C.2090900@redhat.com> <565C844B.7050901@redhat.com> <565C8A57.2000305@redhat.com> <565CA4B8.1010701@redhat.com> <565DA35D.6010804@redhat.com> <565EBDA3.40600@redhat.com> <565EED67.3040405@redhat.com> <565EEDDC.8060304@redhat.com> Message-ID: <565EEFEC.1080703@redhat.com> On 02.12.2015 14:10, Martin Basti wrote: > > > On 02.12.2015 14:08, Martin Babinsky wrote: >> On 12/02/2015 10:45 AM, Martin Babinsky wrote: >>> On 12/01/2015 02:40 PM, Martin Babinsky wrote: >>>> On 11/30/2015 08:34 PM, Martin Basti wrote: >>>>> >>>>> >>>>> On 30.11.2015 18:41, Martin Babinsky wrote: >>>>>> On 11/30/2015 06:15 PM, Martin Basti wrote: >>>>>>> >>>>>>> >>>>>>> On 30.11.2015 16:43, Martin Babinsky wrote: >>>>>>>> On 11/30/2015 12:31 PM, Jan Cholasta wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> On 27.11.2015 14:58, Martin Babinsky wrote: >>>>>>>>>> On 11/19/2015 06:19 PM, Martin Babinsky wrote: >>>>>>>>>>> These two patches fix the following tickets: >>>>>>>>>>> >>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5377 >>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5409 >>>>>>>>>>> >>>>>>>>>>> I have added a new option '--ignore-disconnected-topology' >>>>>>>>>>> which >>>>>>>>>>> forces >>>>>>>>>>> IPA master uninstall despite reported errors in topology. >>>>>>>>>>> I'm not >>>>>>>>>>> quite >>>>>>>>>>> sure if we want to flood ipa-server-install with >>>>>>>>>>> uninstall-specific >>>>>>>>>>> options, maybe it is better to skip the check in unattended >>>>>>>>>>> mode >>>>>>>>>>> and >>>>>>>>>>> just print a warning about disconnected topology and what to do >>>>>>>>>>> about >>>>>>>>>>> it. >>>>>>>>>>> >>>>>>>>>>> I would like to hear your opinions about this. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> Attaching rebased and updated patches. >>>>>>>>> >>>>>>>>> Patch 0098: LGTM >>>>>>>>> >>>>>>>>> >>>>>>>>> Patch 0099: >>>>>>>>> >>>>>>>>> a) This check should be done in Server.__init__() rather than >>>>>>>>> install_check(): >>>>>>>>> >>>>>>>>> + if options.ignore_disconnected_topology: >>>>>>>>> + print("'--ignore-disconnected-topology' is used only >>>>>>>>> during " >>>>>>>>> + "uninstallation") >>>>>>>>> + sys.exit(1) >>>>>>>>> >>>>>>>>> >>>>>>>>> b) >>>>>>>>> s/--ignore-disconnected-topology/--ignore-topology-disconnect/, >>>>>>>>> for >>>>>>>>> consistency with other options, e.g. --no-ui-redirect. >>>>>>>>> >>>>>>>>> Maybe even shorten it to --ignore-topology? But we probably don't >>>>>>>>> want >>>>>>>>> people to use this option much, so it might be better to keep it >>>>>>>>> long? >>>>>>>>> >>>>>>>> I would rather leave it with the long option name, it is more >>>>>>>> apparent >>>>>>>> what this switch should be around. >>>>>>>>> >>>>>>>>> c) I'm fine with uninstall options, you can remove the TODO: >>>>>>>>> >>>>>>>>> + # TODO: ask jcholast about uninstallation options >>>>>>>>> >>>>>>>>> >>>>>>>>> Honza >>>>>>>>> >>>>>>>> >>>>>>>> Attaching updated patches. >>>>>>>> >>>>>>> NACK >>>>>>> >>>>>>> ipa-server-install --uninstall >>>>>>> >>>>>>> 2015-11-30T17:14:30Z DEBUG Destroyed connection >>>>>>> context.ldap2_140081152041808 >>>>>>> 2015-11-30T17:14:30Z DEBUG Traceback (most recent call last): >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>>>>> line 91, in _handle_exception >>>>>>> super(Continuous, self)._handle_exception(exc_info) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 387, in _handle_exception >>>>>>> six.reraise(*exc_info) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 439, in _handle_exception >>>>>>> super(ComponentBase, self)._handle_exception(exc_info) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 387, in _handle_exception >>>>>>> six.reraise(*exc_info) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 355, in __runner >>>>>>> step() >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 352, in >>>>>>> step = lambda: next(self.__gen) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>> line 81, in run_generator_with_yield_from >>>>>>> six.reraise(*exc_info) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>> line 59, in run_generator_with_yield_from >>>>>>> value = gen.send(prev_value) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>>>>> line 71, in _uninstall >>>>>>> for nothing in self._uninstaller(self.parent): >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>> >>>>>>> >>>>>>> line 1409, in main >>>>>>> uninstall_check(self) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>> >>>>>>> >>>>>>> line 265, in decorated >>>>>>> func(installer) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>> >>>>>>> >>>>>>> line 1140, in uninstall_check >>>>>>> api, masters, options.ignore_disconnected_topology) >>>>>>> AttributeError: 'uninstaller(Server)' object has no attribute >>>>>>> 'ignore_disconnected_topology' >>>>>>> >>>>>>> 2015-11-30T17:14:30Z ERROR 'uninstaller(Server)' object has no >>>>>>> attribute >>>>>>> 'ignore_disconnected_topology' >>>>>>> 2015-11-30T17:14:30Z INFO The ipa-server-install command was >>>>>>> successful >>>>>>> >>>>>> >>>>>> Sorry I have failed horribly during option rename. Attaching patch >>>>>> that should actually work. >>>>>> >>>>> functional ACK >>>> Attaching rebased patches reflecting the recent changes in the >>>> handling >>>> of managed topology suffixes handling. >>>> >>>> >>>> >>>> >>> >>> Jan had some more suggestions to the patches. Attaching updated >>> version. >>> >>> >>> >> Attaching updated patch 99 with fixed error message. >> > Pushed to master: b8c619a7139bd7b65caa03b68431e22791ff19bf > ACK :-) From mbabinsk at redhat.com Wed Dec 2 13:52:40 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Wed, 2 Dec 2015 14:52:40 +0100 Subject: [Freeipa-devel] [PATCH 0364, 0367] ipa-kra-install: allow first KRA to be installed on replica In-Reply-To: <565C877B.3050405@redhat.com> References: <56587E92.7060008@redhat.com> <565C4C31.4070509@redhat.com> <565C877B.3050405@redhat.com> Message-ID: <565EF7A8.7050304@redhat.com> On 11/30/2015 06:29 PM, Martin Basti wrote: > > > On 30.11.2015 14:16, Martin Babinsky wrote: >> On 11/27/2015 05:02 PM, Martin Basti wrote: >>> https://fedorahosted.org/freeipa/ticket/5460 >>> >>> I tested just master, I will test ipa-4-2 later. >>> patch attached. >>> >>> >> >> ACK for the master branch. >> > Thanks, additional patch improves error message when ipa-replica-install > --setup-ca --setup-kra is executed and KRA is not installed anywhere yet. > > I'm working on patches for ipa-4-2 branch > > Martin ACK for patch 367. -- Martin^3 Babinsky From pviktori at redhat.com Wed Dec 2 13:52:57 2015 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 2 Dec 2015 14:52:57 +0100 Subject: [Freeipa-devel] [PATCH] 0748 Handle encoding for ipautil.run In-Reply-To: <565EA4FA.3020307@redhat.com> References: <5638B8F9.6010703@redhat.com> <564D0FD9.4070505@redhat.com> <5652B5AC.2040508@redhat.com> <5652E174.50301@redhat.com> <56548E73.1090105@redhat.com> <565587AE.2070303@redhat.com> <5655CA08.8090701@redhat.com> <565C01FD.7050906@redhat.com> <565D83CC.8020802@redhat.com> <565EA4FA.3020307@redhat.com> Message-ID: <565EF7B9.6020507@redhat.com> On 12/02/2015 08:59 AM, Jan Cholasta wrote: > On 1.12.2015 12:26, Petr Viktorin wrote: >> On 11/30/2015 08:59 AM, Jan Cholasta wrote: >>> On 25.11.2015 15:47, Petr Viktorin wrote: >>>> On 11/25/2015 11:04 AM, Jan Cholasta wrote: >>>>> On 24.11.2015 17:21, Petr Viktorin wrote: >>>>>> On 11/23/2015 10:50 AM, Jan Cholasta wrote: >>>>>>> On 23.11.2015 07:43, Jan Cholasta wrote: >>>>>>>> On 19.11.2015 00:55, Petr Viktorin wrote: >>>>>>>>> On 11/03/2015 02:39 PM, Petr Viktorin wrote: >>>>>>>>>> Hello, >>>>>>>>>> Python 3's strings are Unicode, so data coming to or leaving a >>>>>>>>>> Python >>>>>>>>>> program needs to be decoded/encoded if it's to be handled as a >>>>>>>>>> string. >>>>>>>>>> One of the boundaries where encoding is necessary is external >>>>>>>>>> programs, >>>>>>>>>> specifically, ipautil.run. >>>>>>>>>> Unfortunately there's no one set of options that would work >>>>>>>>>> for all >>>>>>>>>> run() invocations, so I went through them all and specified the >>>>>>>>>> stdin/stdout/stderr encoding where necessary. I've also >>>>>>>>>> updated the >>>>>>>>>> call >>>>>>>>>> sites to make it clearer if the return values are used or not. >>>>>>>>>> If an encoding is not set, run() will accept/return bytes. (This >>>>>>>>>> is a >>>>>>>>>> fail-safe setting, since it can't raise errors, and using bytes >>>>>>>>>> where >>>>>>>>>> strings are expected generally fails loudly in py3.) >>>>>>>>>> >>>>>>>>>> Note that the changes are not effective under Python 2. >>>>>>>>> >>>>>>>>> ping, >>>>>>>>> Could someone look at this patch? >>>>>>>> >>>>>>>> Looking. >>>>>>> >>>>>>> 1) I think a better approach would be to use str for >>>>>>> stdin/stdout/stderr >>>>>>> by default in both Python 2 and 3, i.e. do nothing in Python 2 and >>>>>>> encode/decode using locale.getpreferredencoding() in Python 3. >>>>>>> This is >>>>>>> consistent with sys.std{in,out,err} in both Python 2 and 3. Using >>>>>>> bytes >>>>>>> or different encoding should be opt-in. >>>>>>> >>>>>>> Note that different encoding should be used only if the LC_ALL or >>>>>>> LANG >>>>>>> variables are overriden in the command's environment. >>>>>> >>>>>> That would assume the output of *everything* run via ipautil.run >>>>>> can be >>>>>> decoded using the locale encoding. Any stray invalid byte would make >>>>>> IPA >>>>>> crash, even in cases where we don't care about the output. IPA calls >>>>>> too >>>>>> many weird tools to assume they all output text. >>>>> >>>>> Such a stray invalid byte may bubble through our code and cause havoc >>>>> somewhere else, which is much harder to troubleshoot than a crash in >>>>> ipautil.run(), where you can see that it was a misbehaving command >>>>> that >>>>> caused the crash and exactly what command it was. >>>> >>>> The invalid byte can't bubble through code, because if you don't >>>> specify >>>> an encoding you get bytes back. You'll get a crash when you try >>>> using it >>>> as a string. >>> >>> Invalid bytes *can* bubble through our code, into configuration files >>> and other external places. The fact that it has not caused much grief so >>> far suggests that it should be safe to use the locale encoding for most >>> commands. >> >> Yes, "suggests" for "most". I'm still wary of using the locale encoding >> for everything by default. > > OK, let's keep bytes as the default, but I still think the locale > encoding should be the default when decoding output to text. > >> >>> If we do subscribe to the statement that any command can output invalid >>> bytes at any time, I don't see a point in handling encoding in >>> ipautil.run() at all - just always return bytes and let the caller worry >>> about encoding and handling related errors. >> >> Right, that's actually the basic idea. >> >> However, the encoding is two lines of boilerplate code ("if six.PY3" and >> the actual encode). I've turned this into a convenience keyword argument >> to minimize the repetition. > > If you always return bytes in both Python 2 and 3, you can always > .decode() the output, so there won't be any "if six.PY3". Bearing that > in mind, I don't see much difference between adding ", > stdout_encoding='ascii'" to the run() call or "stdout = > stdout.decode('ascii')" on the line below, except the latter is more > explicit and lets you handle decoding errors yourself. > >>>> locale.getpreferredencoding() is not used consistently in all software, >>>> especially if it's not written in Python. For instance, wget won't >>>> magically re-encode the data it fetches for us. It's better to >>>> explicitly specify the encoding every time. >>> >>> I would say it is used consistently in most software used by IPA. The >>> wget example is obviously not relevant to this discussion, since it's >>> returning external data. >>> >>> IMO setting the encoding to 'ascii' without also setting LC_ALL=C in the >>> environment, as seen frequently in your patch, has even higher >>> probability of breaking something than using the locale encoding. >> >> The patch should use 'ascii' with programs that don't output localized >> messages: things like base64-encoded certs or the output of `ip`. >> If you see a particular call where that's not the case, it's a bug in >> the patch, please point it out. > > IMO it's better to be safe than sorry, so I would like to see either > LC_ALL=C set with every use of 'ascii' encoding, or use the locale > encoding instead of 'ascii'. > >> >>>>> If we don't care about output somewhere, we should not capture it >>>>> there >>>>> at all. >>>> >>>> Then people need to remember to put "capture_output=True" everywhere >>>> (except that also disables logging of the output, so we'd need an >>>> additional option). >>> >>> capture_output=True is the default and capture_output=False should be >>> set where output is not needed. >> >> "capture_output=True" being the default would mean that it's still >> possible to leave "capture_output=False" out, but ignore the output. It >> would make the wrong usage easier to type than the right one, which is >> something to avoid. > > I'm not sure I understand what are you trying to say here. > > My point is that capture_output=True is currently the default (see for > yourself) and it is indeed possible to leave capture_output=False but > ignore the output. I believe this is wrong and that you should always > have to explicitly request the output to be captured. OK, looks like I will need to refactor ipautil.run even for Python 2, then. I seem to have have problems coming up with a solution that would match your expectations, so let me write the semantics as I understand you'd like hem: Would you be happy with the following signature? def run(args, stdin=None, raiseonerr=True, nolog=(), env=None, capture_output=False, skip_output=False, cwd=None, runas=None, timeout=None, suplementary_groups=[], capture_stderr=False, encode_stdout=True, encode_stderr=True, ): Output and error would be always logged, except if skip_output is set. But they would be only returned if capture_output/capture_stderr were set. If encode_* is False, bytes are returned, by default locale.getpreferredencoding() is used. (If the caller wants any other encoding, it can do that by itself.) stdin it encoded using locale.getpreferredencoding() if it's unicode. Also, in this case I'd like the function would return a namedtuple, so we don't have to set unused variables all the time. But I can leave that as is if you disagree. Would that be OK? -- Petr Viktorin From tbabej at redhat.com Wed Dec 2 14:41:54 2015 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 2 Dec 2015 15:41:54 +0100 Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <565EAAAE.8020306@redhat.com> References: <565C3C6F.4040901@redhat.com> <20151130163256.GA7759@mail.corp.redhat.com> <565DD881.60401@redhat.com> <565EAAAE.8020306@redhat.com> Message-ID: <565F0332.9000804@redhat.com> On 12/02/2015 09:24 AM, Tomas Babej wrote: > > > On 12/01/2015 06:27 PM, Tomas Babej wrote: >> >> >> On 11/30/2015 05:32 PM, Lukas Slebodnik wrote: >>> On (30/11/15 13:09), Tomas Babej wrote: >>>> Hi, >>>> >>>> IPA sudo tests worked under the assumption that the clients that >>>> are executing the sudo commands have their IPs assigned within >>>> 255.255.255.0 hostmask. >>>> >>>> Removes this (invalid) assumption and adds a dynamic detection of >>>> the hostmask of the IPA client. >>>> >>>> https://fedorahosted.org/freeipa/ticket/5501 >>> >>> >From e6f1846f0d7d17303e5b30b1643651ba739b2b6c Mon Sep 17 00:00:00 2001 >>>> From: Tomas Babej >>>> Date: Mon, 30 Nov 2015 12:53:39 +0100 >>>> Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on >>>> hostmask >>>> >>>> IPA sudo tests worked under the assumption that the clients that >>>> are executing the sudo commands have their IPs assigned within >>>> 255.255.255.0 hostmask. >>>> >>>> Removes this (invalid) assumption and adds a dynamic detection of >>>> the hostmask of the IPA client. >>>> >> >> Thanks, updated patch attached. >> >> Tomas >> > > Actually, a small improvement is necessary. > > Updated patch attached. > > Tomas > > > Thanks to Lukas, we found another problem with the test. Updated patch attached. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0388-4-tests-Add-hostmask-detection-for-sudo-rules-validati.patch Type: text/x-patch Size: 2705 bytes Desc: not available URL: From ofayans at redhat.com Wed Dec 2 15:18:53 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Wed, 2 Dec 2015 16:18:53 +0100 Subject: [Freeipa-devel] [PATCH] First part of the replica promotion tests + testplan In-Reply-To: <565DB7D3.5040500@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> <565DB7D3.5040500@redhat.com> Message-ID: <565F0BDD.8080206@redhat.com> Hi Martin, On 12/01/2015 04:08 PM, Martin Basti wrote: > > > On 27.11.2015 16:26, Oleg Fayans wrote: >> And patch N 16 passes lint too: >> >> On 11/27/2015 04:03 PM, Oleg Fayans wrote: >>> Hi, >>> >>> On 11/27/2015 03:26 PM, Martin Basti wrote: >>>> >>>> >>>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>>> Hi Martin, >>>>> >>>>> All your suggestions were taken into account. Both patches are >>>>> updated. Thank you for your help! >>>>> >>>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>>> >>>>>> >>>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>>> Hi Martin, >>>>>>> >>>>>>> I agree to all your points but one. please, see my comment below >>>>>>> >>>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> 0) Note >>>>>>>> Please be aware of https://fedorahosted.org/freeipa/ticket/5469 >>>>>>>> during >>>>>>>> KRA testing >>>>>>>> >>>>>>>> 1) >>>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, this may >>>>>>>> change >>>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for domain level 0 >>>>>>>> and 1 >>>>>>>> >>>>>>>> 2) >>>>>>>> Why uninstall KRA then server, is not enough just uninstall server >>>>>>>> which >>>>>>>> covers KRA uninstall? >>>>>>>> >>>>>>>> + def teardown_method(self, method): >>>>>>>> + for host in self.replicas: >>>>>>>> + host.run_command(self.kra_uninstall, raiseonerr=False) >>>>>>>> + tasks.uninstall_master(host) >>>>>>>> >>>>>>>> >>>>>>>> 3) >>>>>>>> Can be this function more generic? It should allow specify host >>>>>>>> where >>>>>>>> KRA should be installed not just master >>>>>>>> >>>>>>>> + def test_kra_install_master(self): >>>>>>>> + self.master.run_command(self.kra_install) >>>>>>>> >>>>>>>> >>>>>>>> 4) >>>>>>>> >>>>>>>> TestLevel0(Dummy): >>>>>>>> Can be the test name more specific, something like >>>>>>>> TestReplicaPromotionLevel0 >>>>>>>> >>>>>>>> >>>>>>>> 5) >>>>>>>> please remove this, the patch is on review and it will be pushed >>>>>>>> sooner >>>>>>>> than tests >>>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>>> >>>>>>>> and as I mentioned in ticket #5455, I cannot reproduce it with >>>>>>>> ipa-kra-install, so please provide steps to reproduce if you insist >>>>>>>> that >>>>>>>> this still does not work as expected with KRA. >>>>>>>> >>>>>>>> 6) This is completely wrong, it removes everything that we tried to >>>>>>>> achieve with previous patches with domain level in CI >>>>>>> >>>>>>> Actually, being able to configure domain level per class is WAY more >>>>>>> convenient, than to always have to think which domain level is >>>>>>> appropriate for which particular test during jenkins job >>>>>>> configuration. In fact, I should have thought about it from the very >>>>>>> beginning. For example, in test_replica_promotion.py we have on >>>>>>> class, >>>>>>> which intiates with domain level = 1, while others - with domain >>>>>>> level >>>>>>> 0. With config-based approach, we would have to implement a separate >>>>>>> step that raises domain level. Overall, I am against the approach, >>>>>>> when you have to remember to set certain domain level in config for >>>>>>> any particular test. The tests themselves should be aware of the >>>>>>> domain level they need. >>>>>> I do not say that we should not have something that overrides >>>>>> settings >>>>>> in from config in a particular test case, I say your patch is >>>>>> doing it >>>>>> wrong. >>>>>> >>>>>> I agree it is useful to have param domain_level in install_master, >>>>>> and >>>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by default, >>>>>> because with your current patch the domain_level in config is not >>>>>> used >>>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>>> >>>>>> For example I want to achieve this goal: >>>>>> test_vault.py, this test suite can run on domain level1 and on domain >>>>>> level0, so with one test we can test 2 domain levels just with >>>>>> putting >>>>>> domain level into config file. >>>>>> >>>>>> I agree that with extraordinary test like replica promotion test >>>>>> is, we >>>>>> need something that allows override the config file. >>>>>> >>>>>> As I said bellow, domain_level default value should be None in >>>>>> install_master and install_topo plugin. If domain level was specified >>>>>> use the specified one, if not (value is None) use the domain level >>>>>> from >>>>>> config file. >>>>> >>>>> Agreed :) >>>>> >>>>>> >>>>>> Martin >>>>>>> >>>>>>>> >>>>>>>> [PATCH] Enabled setting domain_level per class derived from >>>>>>>> TestIntegration >>>>>>>> >>>>>>>> When I configure domain level 0 in yaml config, how is this >>>>>>>> supposed to >>>>>>>> get into install methods when you removed that code? >>>>>>>> >>>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>>> + "--domain-level=%i" % domain_level >>>>>>>> >>>>>>>> >>>>>>>> You always use MAX_DOMAIN_LEVEL in this case or whatever is >>>>>>>> specified in >>>>>>>> domain_level option. >>>>>>>> I suggest to use domain_level=None, and when it is None use >>>>>>>> 'host.config.domain_level', if it is not None, use 'domain_level' >>>>>>>> >>>>>>>> With this we can specify domain level in config file for test that >>>>>>>> can >>>>>>>> be used for both domain levels and you can manually specify domain >>>>>>>> level >>>>>>>> for test that requires specific domain level. >>>>>>>> >>>>>>>> Also this should go away >>>>>>>> >>>>>>>> @classmethod >>>>>>>> def install(cls, mh): >>>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>>> if cls.topology is None: >>>>>>>> return >>>>>>>> >>>>>>>> I do not see reason why test should override configuration in >>>>>>>> config in >>>>>>>> this case. >>>>>>>> >>>>>>>> Martin >>>>>>>> >>>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Here is the updated version of the patch (more tests + fixed the >>>>>>>>> issues of the first one) + patch 0017, that implements the >>>>>>>>> necessary >>>>>>>>> changes in the background code, i. e. patch 16 does not work >>>>>>>>> without >>>>>>>>> patch 17 >>>>>>>>> >>>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>>> Hi guys, >>>>>>>>>>> >>>>>>>>>>> Here are first two automated testcases from this (so far >>>>>>>>>>> incomplete) >>>>>>>>>>> testplan: >>>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>>> >>>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> PATCH 16: NACK >>>>>>>>>> >>>>>>>>>> 1) >>>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>>> 'domain_level' to >>>>>>>>>> install_topo()? >>>>>>>>>> Also it is good practise to add new option as the last parameter. >>>>>>>>>> >>>>>>>>>> 2) >>>>>>>>>> cab you in both tests specify a domain level with constant >>>>>>>>>> instead of >>>>>>>>>> number literal? >>>>>>>>>> >>>>>>>>>> 3) >>>>>>>>>> both test call install_topo with custom domain level, but it >>>>>>>>>> cannot >>>>>>>>>> work >>>>>>>>>> because 1) (did you run the test?) >>>>>>>>>> >>>>>>>>>> 4) >>>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>>> Respectively why there is call of install_topo() that installs >>>>>>>>>> replica. >>>>>>>>>> As this test just tests that ipa-replica-prepare is not working >>>>>>>>>> anymore, >>>>>>>>>> is it worth to spend 20 minutes with installing replica and then >>>>>>>>>> just no >>>>>>>>>> tot use it? IMO to install master in install step is enough. >>>>>>>>>> >>>>>>>>>> Martin^2 >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> ./make-lint >>>> ************* Module ipatests.test_integration.base >>>> ipatests/test_integration/base.py:66: [E1101(no-member), >>>> IntegrationTest.install] Class 'IntegrationTest' has no 'domain_level' >>>> member) >>>> ************* Module ipatests.test_integration.test_replica_promotion >>>> ipatests/test_integration/test_replica_promotion.py:16: >>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no 'domain_level' >>>> member) >>>> ipatests/test_integration/test_replica_promotion.py:117: >>>> [E1101(no-member), TestCAInstall.test_ca_install_without_replica_file] >>>> Module 'ipatests.test_integration.tasks' has no 'setup_replica' member) >>>> >>>> >>>> Is it so hard to run pylint before patch is posted for review? >>> >>> Sorry, my bad. Fixed. >>> >>> >>> >> > > 1) > Why is this change in the patch? > - # Clean up the test directory > - host.run_command(['rm', '-rvf', host.config.test_dir]) Otherwise 2 out of 8 tests fail with IOError at line 78 of ipatests/test_integration/tasks.py I do not understand yet how does this happen, but if you remove ipatests folder once, it then fails to be created again. > - > > > 2) > is enough to have this check only in install_master, install_topo can > pass None to install_master > + if domain_level is None: > + domain_level = master.config.domain_level Done > > > 3) > IMO replica-manage del should cleanup hosts entry, so following code > should not be needed. > + if cleanhost: > + kinit_admin(master) > + master.run_command(["ipa", "host-del", "--updatedns", > replica.hostname], > + raiseonerr=False) Well, in fact it does not. At least the corresponding dns record stays and causes the subsequent ipa-client-install to fail. Probably it's a bug. On the other hand, if I promote an existing client to replica and then delete this replica, then, I probably want the host record (that was created during client-install) to stay in the system. So, does not look like a bug to me. > > 4) > This variable is not used > + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] Removed > > 5) > Why do you need this > + kra_install = ["ipa-kra-install", "-U", "-p", config.dirman_password] > when you implemented tasks.install_kra that returns the exactly the same > result? Right. Removed > > 6) > PEP8 > ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 blank lines, > found 1 > ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 blank lines, > found 1 > ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 blank lines, > found 1 > ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 blank lines, > found 1 > ./ipatests/test_integration/tasks.py:950:80: E501 line too long (80 > 79 > characters) > > ./ipatests/test_integration/test_replica_promotion.py:29:80: E501 line > too long (85 > 79 characters) > ./ipatests/test_integration/test_replica_promotion.py:64:80: E501 line > too long (85 > 79 characters) > ./ipatests/test_integration/test_replica_promotion.py:67:80: E501 line > too long (88 > 79 characters) > ./ipatests/test_integration/test_replica_promotion.py:93:80: E501 line > too long (80 > 79 characters) > ./ipatests/test_integration/test_replica_promotion.py:94:80: E501 line > too long (83 > 79 characters) > ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 line > too long (81 > 79 characters) > ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 line > too long (80 > 79 characters) > ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 line > too long (82 > 79 characters) > ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 line > too long (80 > 79 characters) Most of these complaints are unrelated to the current patches. It's better to create a separate patch addressing PEP8 errors. > > 7) > Why this must be stored in instance? IMO to have it stored as local > variable is perfect > TestKRAInstall, TestCAInstall > self.replica1_filename = tasks.get_replica_filename(replica1) > self.replica2_filename = tasks.get_replica_filename(replica2) Agree. Fixed -- Oleg Fayans Quality Engineer FreeIPA team RedHat. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0016.4-First-part-of-replica-promotion-tests.patch Type: text/x-patch Size: 9837 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0017.3-Enabled-setting-domain_level-per-class.patch Type: text/x-patch Size: 5136 bytes Desc: not available URL: From jcholast at redhat.com Wed Dec 2 15:23:44 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 2 Dec 2015 16:23:44 +0100 Subject: [Freeipa-devel] [PATCH 522] replica promotion: allow OTP bulk client enrollment Message-ID: <565F0D00.6050805@redhat.com> Hi, the attached patch fixes . Note that you still have to provide admin password in ipa-replica-install, either using --admin-password or interactively, because: a) Admin password is required for replica promotion. This will be fixed with . Patches are on the list: . b) Admin password is required for connection check. This will be fixed with . Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-522-replica-promotion-allow-OTP-bulk-client-enrollment.patch Type: text/x-patch Size: 4641 bytes Desc: not available URL: From amarecek at redhat.com Wed Dec 2 15:53:28 2015 From: amarecek at redhat.com (=?utf-8?Q?Ale=C5=A1_Mare=C4=8Dek?=) Date: Wed, 2 Dec 2015 10:53:28 -0500 (EST) Subject: [Freeipa-devel] [patch 0025] Separated Tracker implementations into standalone package In-Reply-To: <565D68E2.5080008@redhat.com> References: <564F31CE.5010402@redhat.com> <5652DFAE.90906@redhat.com> <5658694F.3080404@redhat.com> <56586A7F.3070004@redhat.com> <56586B5D.2040507@redhat.com> <647877470.20906478.1448902246584.JavaMail.zimbra@redhat.com> <565C91B0.2070908@redhat.com> <565D68E2.5080008@redhat.com> Message-ID: <1800049648.22290760.1449071608567.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Milan Kub?k" > To: "Martin Basti" > Cc: freeipa-devel at redhat.com, "Ale? Mare?ek" > Sent: Tuesday, December 1, 2015 10:31:14 AM > Subject: Re: [Freeipa-devel] [patch 0025] Separated Tracker implementations into standalone package > > On 11/30/2015 07:13 PM, Martin Basti wrote: > > NACK > > > > 1) > > With this patch I received this error in test_user_plugin.py > > > > E AssertionError: assert_deepequal: expected != got. > > E 0106: user_status: Query status of "tuser1" > > E expected = 1 > > E got = 2 > > E path = ('count',) > > > > I have just admin user on my system > > > > 2) > > ______________________________________________________________________________ > > ERROR collecting test_xmlrpc/test_stageuser_plugin.py > > ______________________________________________________________________________ > > test_xmlrpc/test_stageuser_plugin.py:28: in > > from ipatests.test_xmlrpc.tracker.user_plugin import UserTracker > > E ImportError: No module named tracker.user_plugin > > ______________________________________________________________________________ > > ERROR collecting test_xmlrpc/test_stageuser_plugin.py > > ______________________________________________________________________________ > > test_xmlrpc/test_stageuser_plugin.py:28: in > > from ipatests.test_xmlrpc.tracker.user_plugin import UserTracker > > E ImportError: No module named tracker.user_plugin > > > > 3) > > ________________________________________________________________________________ > > ERROR collecting test_xmlrpc/test_group_plugin.py > > ________________________________________________________________________________ > > test_xmlrpc/test_group_plugin.py:34: in > > from ipatests.test_xmlrpc.tracker.user_plugin import UserTracker > > E ImportError: No module named tracker.user_plugin > > ________________________________________________________________________________ > > ERROR collecting test_xmlrpc/test_group_plugin.py > > ________________________________________________________________________________ > > test_xmlrpc/test_group_plugin.py:34: in > > from ipatests.test_xmlrpc.tracker.user_plugin import UserTracker > > E ImportError: No module named tracker.user_plugin > > > > 4) > > ________________________________________________________________________________ > > ERROR collecting test_xmlrpc/test_host_plugin.py > > _________________________________________________________________________________ > > test_xmlrpc/test_host_plugin.py:42: in > > from ipatests.test_xmlrpc.tracker.host_plugin import HostTracker > > E ImportError: No module named tracker.host_plugin > > ________________________________________________________________________________ > > ERROR collecting test_xmlrpc/test_host_plugin.py > > _________________________________________________________________________________ > > test_xmlrpc/test_host_plugin.py:42: in > > from ipatests.test_xmlrpc.tracker.host_plugin import HostTracker > > E ImportError: No module named tracker.host_plugin > > > > 5) > > ________________________________________________________________________________ > > ERROR collecting test_xmlrpc/test_caacl_plugin.py > > ________________________________________________________________________________ > > test_xmlrpc/test_caacl_plugin.py:15: in > > from ipatests.test_xmlrpc.test_certprofile_plugin import > > default_profile > > ../_pytest/assertion/rewrite.py:171: in load_module > > py.builtin.exec_(co, mod.__dict__) > > test_xmlrpc/test_certprofile_plugin.py:17: in > > from ipatests.test_xmlrpc.tracker.certprofile_plugin import > > CertprofileTracker > > E ImportError: No module named tracker.certprofile_plugin > > ________________________________________________________________________________ > > ERROR collecting test_xmlrpc/test_caacl_plugin.py > > ________________________________________________________________________________ > > test_xmlrpc/test_caacl_plugin.py:15: in > > from ipatests.test_xmlrpc.test_certprofile_plugin import > > default_profile > > ../_pytest/assertion/rewrite.py:171: in load_module > > py.builtin.exec_(co, mod.__dict__) > > test_xmlrpc/test_certprofile_plugin.py:17: in > > from ipatests.test_xmlrpc.tracker.certprofile_plugin import > > CertprofileTracker > > E ImportError: No module named tracker.certprofile_plugin > > > > > > .... > > > > N) > > rpm -ql freeipa-tests | grep tracker > > returns nothing, IMO the tracker directory in not in RPM All failing parts PASSED. > > > > > > On 30.11.2015 17:50, Ale? Mare?ek wrote: > >> Tested with today's master, ACK. > >> - alich - > >> > >> ----- Original Message ----- > >>> From: "Milan Kub?k" > >>> To: freeipa-devel at redhat.com > >>> Sent: Friday, November 27, 2015 3:40:29 PM > >>> Subject: Re: [Freeipa-devel] [patch 0025] Separated Tracker > >>> implementations into standalone package > >>> > >>> On 11/27/2015 03:36 PM, Milan Kub?k wrote: > >>> > >>> > >>> > >>> On 11/27/2015 03:31 PM, Milan Kub?k wrote: > >>> > >>> > >>> On 11/23/2015 10:43 AM, Lenka Doudova wrote: > >>> > >>> > >>> NACK - there's a "typo" in ..../tracker/user_plugin.py, line 17-18: > >>> > >>> def get_user_dn(cn): > >>> > >>> return DN(('cn', cn), api.env.container_user, api.env.basedn) > >>> > >>> > >>> should be > >>> > >>> def get_user_dn(uid): > >>> > >>> return DN(('uid', uid), api.env.container_user, api.env.basedn) > >>> > >>> > >>> Some tests may fail because of that. > >>> Lenka > >>> > >>> > >>> On 11/20/2015 08:54 PM, Ale? Mare?ek wrote: > >>> > >>> > >>> Looks good. ACK. > >>> > >>> ----- Original Message ----- > >>> > >>> > >>> From: "Milan Kub?k" > >>> To: "freeipa-devel" > >>> Cc: "Filip Skola" , "Ales Marecek" > >>> > >>> Sent: Friday, November 20, 2015 3:44:30 PM > >>> Subject: [patch 0025] Separated Tracker implementations into standalone > >>> package > >>> > >>> Fixes https://fedorahosted.org/freeipa/ticket/5467 > >>> Patches attached. > >>> > >>> -- > >>> Milan Kubik > >>> > >>> > >>> > >>> Fixed the function and moved it into different module. > >>> Updated patches attached. > >>> > >>> > >>> > >>> Self nack, some imports missing > >>> > >>> -- > >>> Milan Kubik > >>> > >>> > >>> Patches updated. > >>> > >>> -- > >>> Milan Kubik > >>> > >>> -- > >>> Manage your subscription for the Freeipa-devel mailing list: > >>> https://www.redhat.com/mailman/listinfo/freeipa-devel > >>> Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code > > > My sincere apologies for this unspoken transgression on the packaging > dieties. > Updated patches attached. > > -- > Milan Kubik > > From pvoborni at redhat.com Wed Dec 2 15:57:13 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 2 Dec 2015 16:57:13 +0100 Subject: [Freeipa-devel] [PATCH] 939 topologysuffix: change iparepltopoconfroot API properties Message-ID: <565F14D9.9000207@redhat.com> Change CLI option, label and type to reflect that it is a only a DN of the suffix. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0939-topologysuffix-change-iparepltopoconfroot-API-proper.patch Type: text/x-patch Size: 4503 bytes Desc: not available URL: From mbasti at redhat.com Wed Dec 2 16:10:22 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 2 Dec 2015 17:10:22 +0100 Subject: [Freeipa-devel] [PATCH] First part of the replica promotion tests + testplan In-Reply-To: <565F0BDD.8080206@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> <565DB7D3.5040500@redhat.com> <565F0BDD.8080206@redhat.com> Message-ID: <565F17EE.3020701@redhat.com> On 02.12.2015 16:18, Oleg Fayans wrote: > Hi Martin, > > On 12/01/2015 04:08 PM, Martin Basti wrote: >> >> >> On 27.11.2015 16:26, Oleg Fayans wrote: >>> And patch N 16 passes lint too: >>> >>> On 11/27/2015 04:03 PM, Oleg Fayans wrote: >>>> Hi, >>>> >>>> On 11/27/2015 03:26 PM, Martin Basti wrote: >>>>> >>>>> >>>>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>>>> Hi Martin, >>>>>> >>>>>> All your suggestions were taken into account. Both patches are >>>>>> updated. Thank you for your help! >>>>>> >>>>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>>>> >>>>>>> >>>>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>>>> Hi Martin, >>>>>>>> >>>>>>>> I agree to all your points but one. please, see my comment below >>>>>>>> >>>>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> 0) Note >>>>>>>>> Please be aware of https://fedorahosted.org/freeipa/ticket/5469 >>>>>>>>> during >>>>>>>>> KRA testing >>>>>>>>> >>>>>>>>> 1) >>>>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, this may >>>>>>>>> change >>>>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for domain >>>>>>>>> level 0 >>>>>>>>> and 1 >>>>>>>>> >>>>>>>>> 2) >>>>>>>>> Why uninstall KRA then server, is not enough just uninstall >>>>>>>>> server >>>>>>>>> which >>>>>>>>> covers KRA uninstall? >>>>>>>>> >>>>>>>>> + def teardown_method(self, method): >>>>>>>>> + for host in self.replicas: >>>>>>>>> + host.run_command(self.kra_uninstall, >>>>>>>>> raiseonerr=False) >>>>>>>>> + tasks.uninstall_master(host) >>>>>>>>> >>>>>>>>> >>>>>>>>> 3) >>>>>>>>> Can be this function more generic? It should allow specify host >>>>>>>>> where >>>>>>>>> KRA should be installed not just master >>>>>>>>> >>>>>>>>> + def test_kra_install_master(self): >>>>>>>>> + self.master.run_command(self.kra_install) >>>>>>>>> >>>>>>>>> >>>>>>>>> 4) >>>>>>>>> >>>>>>>>> TestLevel0(Dummy): >>>>>>>>> Can be the test name more specific, something like >>>>>>>>> TestReplicaPromotionLevel0 >>>>>>>>> >>>>>>>>> >>>>>>>>> 5) >>>>>>>>> please remove this, the patch is on review and it will be pushed >>>>>>>>> sooner >>>>>>>>> than tests >>>>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>>>> >>>>>>>>> and as I mentioned in ticket #5455, I cannot reproduce it with >>>>>>>>> ipa-kra-install, so please provide steps to reproduce if you >>>>>>>>> insist >>>>>>>>> that >>>>>>>>> this still does not work as expected with KRA. >>>>>>>>> >>>>>>>>> 6) This is completely wrong, it removes everything that we >>>>>>>>> tried to >>>>>>>>> achieve with previous patches with domain level in CI >>>>>>>> >>>>>>>> Actually, being able to configure domain level per class is WAY >>>>>>>> more >>>>>>>> convenient, than to always have to think which domain level is >>>>>>>> appropriate for which particular test during jenkins job >>>>>>>> configuration. In fact, I should have thought about it from the >>>>>>>> very >>>>>>>> beginning. For example, in test_replica_promotion.py we have on >>>>>>>> class, >>>>>>>> which intiates with domain level = 1, while others - with domain >>>>>>>> level >>>>>>>> 0. With config-based approach, we would have to implement a >>>>>>>> separate >>>>>>>> step that raises domain level. Overall, I am against the approach, >>>>>>>> when you have to remember to set certain domain level in config >>>>>>>> for >>>>>>>> any particular test. The tests themselves should be aware of the >>>>>>>> domain level they need. >>>>>>> I do not say that we should not have something that overrides >>>>>>> settings >>>>>>> in from config in a particular test case, I say your patch is >>>>>>> doing it >>>>>>> wrong. >>>>>>> >>>>>>> I agree it is useful to have param domain_level in install_master, >>>>>>> and >>>>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by default, >>>>>>> because with your current patch the domain_level in config is not >>>>>>> used >>>>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>>>> >>>>>>> For example I want to achieve this goal: >>>>>>> test_vault.py, this test suite can run on domain level1 and on >>>>>>> domain >>>>>>> level0, so with one test we can test 2 domain levels just with >>>>>>> putting >>>>>>> domain level into config file. >>>>>>> >>>>>>> I agree that with extraordinary test like replica promotion test >>>>>>> is, we >>>>>>> need something that allows override the config file. >>>>>>> >>>>>>> As I said bellow, domain_level default value should be None in >>>>>>> install_master and install_topo plugin. If domain level was >>>>>>> specified >>>>>>> use the specified one, if not (value is None) use the domain level >>>>>>> from >>>>>>> config file. >>>>>> >>>>>> Agreed :) >>>>>> >>>>>>> >>>>>>> Martin >>>>>>>> >>>>>>>>> >>>>>>>>> [PATCH] Enabled setting domain_level per class derived from >>>>>>>>> TestIntegration >>>>>>>>> >>>>>>>>> When I configure domain level 0 in yaml config, how is this >>>>>>>>> supposed to >>>>>>>>> get into install methods when you removed that code? >>>>>>>>> >>>>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>>>> + "--domain-level=%i" % domain_level >>>>>>>>> >>>>>>>>> >>>>>>>>> You always use MAX_DOMAIN_LEVEL in this case or whatever is >>>>>>>>> specified in >>>>>>>>> domain_level option. >>>>>>>>> I suggest to use domain_level=None, and when it is None use >>>>>>>>> 'host.config.domain_level', if it is not None, use 'domain_level' >>>>>>>>> >>>>>>>>> With this we can specify domain level in config file for test >>>>>>>>> that >>>>>>>>> can >>>>>>>>> be used for both domain levels and you can manually specify >>>>>>>>> domain >>>>>>>>> level >>>>>>>>> for test that requires specific domain level. >>>>>>>>> >>>>>>>>> Also this should go away >>>>>>>>> >>>>>>>>> @classmethod >>>>>>>>> def install(cls, mh): >>>>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>>>> if cls.topology is None: >>>>>>>>> return >>>>>>>>> >>>>>>>>> I do not see reason why test should override configuration in >>>>>>>>> config in >>>>>>>>> this case. >>>>>>>>> >>>>>>>>> Martin >>>>>>>>> >>>>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Here is the updated version of the patch (more tests + fixed the >>>>>>>>>> issues of the first one) + patch 0017, that implements the >>>>>>>>>> necessary >>>>>>>>>> changes in the background code, i. e. patch 16 does not work >>>>>>>>>> without >>>>>>>>>> patch 17 >>>>>>>>>> >>>>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>>>> Hi guys, >>>>>>>>>>>> >>>>>>>>>>>> Here are first two automated testcases from this (so far >>>>>>>>>>>> incomplete) >>>>>>>>>>>> testplan: >>>>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>>>> >>>>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> PATCH 16: NACK >>>>>>>>>>> >>>>>>>>>>> 1) >>>>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>>>> 'domain_level' to >>>>>>>>>>> install_topo()? >>>>>>>>>>> Also it is good practise to add new option as the last >>>>>>>>>>> parameter. >>>>>>>>>>> >>>>>>>>>>> 2) >>>>>>>>>>> cab you in both tests specify a domain level with constant >>>>>>>>>>> instead of >>>>>>>>>>> number literal? >>>>>>>>>>> >>>>>>>>>>> 3) >>>>>>>>>>> both test call install_topo with custom domain level, but it >>>>>>>>>>> cannot >>>>>>>>>>> work >>>>>>>>>>> because 1) (did you run the test?) >>>>>>>>>>> >>>>>>>>>>> 4) >>>>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>>>> Respectively why there is call of install_topo() that installs >>>>>>>>>>> replica. >>>>>>>>>>> As this test just tests that ipa-replica-prepare is not working >>>>>>>>>>> anymore, >>>>>>>>>>> is it worth to spend 20 minutes with installing replica and >>>>>>>>>>> then >>>>>>>>>>> just no >>>>>>>>>>> tot use it? IMO to install master in install step is enough. >>>>>>>>>>> >>>>>>>>>>> Martin^2 >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> ./make-lint >>>>> ************* Module ipatests.test_integration.base >>>>> ipatests/test_integration/base.py:66: [E1101(no-member), >>>>> IntegrationTest.install] Class 'IntegrationTest' has no >>>>> 'domain_level' >>>>> member) >>>>> ************* Module ipatests.test_integration.test_replica_promotion >>>>> ipatests/test_integration/test_replica_promotion.py:16: >>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no 'domain_level' >>>>> member) >>>>> ipatests/test_integration/test_replica_promotion.py:117: >>>>> [E1101(no-member), >>>>> TestCAInstall.test_ca_install_without_replica_file] >>>>> Module 'ipatests.test_integration.tasks' has no 'setup_replica' >>>>> member) >>>>> >>>>> >>>>> Is it so hard to run pylint before patch is posted for review? >>>> >>>> Sorry, my bad. Fixed. >>>> >>>> >>>> >>> >> >> 1) >> Why is this change in the patch? >> - # Clean up the test directory >> - host.run_command(['rm', '-rvf', host.config.test_dir]) > > Otherwise 2 out of 8 tests fail with IOError at line 78 of > ipatests/test_integration/tasks.py > > I do not understand yet how does this happen, but if you remove > ipatests folder once, it then fails to be created again. > So this should be in separated patch and investigated properly. >> - >> >> >> 2) >> is enough to have this check only in install_master, install_topo can >> pass None to install_master >> + if domain_level is None: >> + domain_level = master.config.domain_level > > Done > >> >> >> 3) >> IMO replica-manage del should cleanup hosts entry, so following code >> should not be needed. >> + if cleanhost: >> + kinit_admin(master) >> + master.run_command(["ipa", "host-del", "--updatedns", >> replica.hostname], >> + raiseonerr=False) > > Well, in fact it does not. At least the corresponding dns record stays > and causes the subsequent ipa-client-install to fail. Probably it's a > bug. On the other hand, if I promote an existing client to replica and > then delete this replica, then, I probably want the host record (that > was created during client-install) to stay in the system. So, does not > look like a bug to me. No you don't, because replica uninstallation also removes the client. I tried it with ipa42, ipa-replica-manage del removes host entry, and DNS A records, only SSHFP records stay there (I'm not sure if it is bug or feature) Also I received this message """ Failed to cleanup replica1.ipa.test DNS entries: no matching entry found You may need to manually remove them from the tree """ But, A record has been removed, so this is probably false positive and it needs to have a ticket. > >> >> 4) >> This variable is not used >> + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] > > Removed > >> >> 5) >> Why do you need this >> + kra_install = ["ipa-kra-install", "-U", "-p", >> config.dirman_password] >> when you implemented tasks.install_kra that returns the exactly the same >> result? > > Right. Removed > >> >> 6) >> PEP8 >> ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 blank lines, >> found 1 >> ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 blank lines, >> found 1 >> ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 blank lines, >> found 1 >> ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 blank lines, >> found 1 >> ./ipatests/test_integration/tasks.py:950:80: E501 line too long (80 > 79 >> characters) >> >> ./ipatests/test_integration/test_replica_promotion.py:29:80: E501 line >> too long (85 > 79 characters) >> ./ipatests/test_integration/test_replica_promotion.py:64:80: E501 line >> too long (85 > 79 characters) >> ./ipatests/test_integration/test_replica_promotion.py:67:80: E501 line >> too long (88 > 79 characters) >> ./ipatests/test_integration/test_replica_promotion.py:93:80: E501 line >> too long (80 > 79 characters) >> ./ipatests/test_integration/test_replica_promotion.py:94:80: E501 line >> too long (83 > 79 characters) >> ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 line >> too long (81 > 79 characters) >> ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 line >> too long (80 > 79 characters) >> ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 line >> too long (82 > 79 characters) >> ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 line >> too long (80 > 79 characters) > > Most of these complaints are unrelated to the current patches. > It's better to create a separate patch addressing PEP8 errors. > I beg for your pardon, those PEP8 errors have been introduced by your patches. >> >> 7) >> Why this must be stored in instance? IMO to have it stored as local >> variable is perfect >> TestKRAInstall, TestCAInstall >> self.replica1_filename = tasks.get_replica_filename(replica1) >> self.replica2_filename = tasks.get_replica_filename(replica2) > > Agree. Fixed > From mbasti at redhat.com Wed Dec 2 16:18:31 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 2 Dec 2015 17:18:31 +0100 Subject: [Freeipa-devel] [patch 0025] Separated Tracker implementations into standalone package In-Reply-To: <1800049648.22290760.1449071608567.JavaMail.zimbra@redhat.com> References: <564F31CE.5010402@redhat.com> <5652DFAE.90906@redhat.com> <5658694F.3080404@redhat.com> <56586A7F.3070004@redhat.com> <56586B5D.2040507@redhat.com> <647877470.20906478.1448902246584.JavaMail.zimbra@redhat.com> <565C91B0.2070908@redhat.com> <565D68E2.5080008@redhat.com> <1800049648.22290760.1449071608567.JavaMail.zimbra@redhat.com> Message-ID: <565F19D7.5030400@redhat.com> On 02.12.2015 16:53, Ale? Mare?ek wrote: > > ----- Original Message ----- >> From: "Milan Kub?k" >> To: "Martin Basti" >> Cc: freeipa-devel at redhat.com, "Ale? Mare?ek" >> Sent: Tuesday, December 1, 2015 10:31:14 AM >> Subject: Re: [Freeipa-devel] [patch 0025] Separated Tracker implementations into standalone package >> >> On 11/30/2015 07:13 PM, Martin Basti wrote: >>> NACK >>> >>> 1) >>> With this patch I received this error in test_user_plugin.py >>> >>> E AssertionError: assert_deepequal: expected != got. >>> E 0106: user_status: Query status of "tuser1" >>> E expected = 1 >>> E got = 2 >>> E path = ('count',) >>> >>> I have just admin user on my system >>> >>> 2) >>> ______________________________________________________________________________ >>> ERROR collecting test_xmlrpc/test_stageuser_plugin.py >>> ______________________________________________________________________________ >>> test_xmlrpc/test_stageuser_plugin.py:28: in >>> from ipatests.test_xmlrpc.tracker.user_plugin import UserTracker >>> E ImportError: No module named tracker.user_plugin >>> ______________________________________________________________________________ >>> ERROR collecting test_xmlrpc/test_stageuser_plugin.py >>> ______________________________________________________________________________ >>> test_xmlrpc/test_stageuser_plugin.py:28: in >>> from ipatests.test_xmlrpc.tracker.user_plugin import UserTracker >>> E ImportError: No module named tracker.user_plugin >>> >>> 3) >>> ________________________________________________________________________________ >>> ERROR collecting test_xmlrpc/test_group_plugin.py >>> ________________________________________________________________________________ >>> test_xmlrpc/test_group_plugin.py:34: in >>> from ipatests.test_xmlrpc.tracker.user_plugin import UserTracker >>> E ImportError: No module named tracker.user_plugin >>> ________________________________________________________________________________ >>> ERROR collecting test_xmlrpc/test_group_plugin.py >>> ________________________________________________________________________________ >>> test_xmlrpc/test_group_plugin.py:34: in >>> from ipatests.test_xmlrpc.tracker.user_plugin import UserTracker >>> E ImportError: No module named tracker.user_plugin >>> >>> 4) >>> ________________________________________________________________________________ >>> ERROR collecting test_xmlrpc/test_host_plugin.py >>> _________________________________________________________________________________ >>> test_xmlrpc/test_host_plugin.py:42: in >>> from ipatests.test_xmlrpc.tracker.host_plugin import HostTracker >>> E ImportError: No module named tracker.host_plugin >>> ________________________________________________________________________________ >>> ERROR collecting test_xmlrpc/test_host_plugin.py >>> _________________________________________________________________________________ >>> test_xmlrpc/test_host_plugin.py:42: in >>> from ipatests.test_xmlrpc.tracker.host_plugin import HostTracker >>> E ImportError: No module named tracker.host_plugin >>> >>> 5) >>> ________________________________________________________________________________ >>> ERROR collecting test_xmlrpc/test_caacl_plugin.py >>> ________________________________________________________________________________ >>> test_xmlrpc/test_caacl_plugin.py:15: in >>> from ipatests.test_xmlrpc.test_certprofile_plugin import >>> default_profile >>> ../_pytest/assertion/rewrite.py:171: in load_module >>> py.builtin.exec_(co, mod.__dict__) >>> test_xmlrpc/test_certprofile_plugin.py:17: in >>> from ipatests.test_xmlrpc.tracker.certprofile_plugin import >>> CertprofileTracker >>> E ImportError: No module named tracker.certprofile_plugin >>> ________________________________________________________________________________ >>> ERROR collecting test_xmlrpc/test_caacl_plugin.py >>> ________________________________________________________________________________ >>> test_xmlrpc/test_caacl_plugin.py:15: in >>> from ipatests.test_xmlrpc.test_certprofile_plugin import >>> default_profile >>> ../_pytest/assertion/rewrite.py:171: in load_module >>> py.builtin.exec_(co, mod.__dict__) >>> test_xmlrpc/test_certprofile_plugin.py:17: in >>> from ipatests.test_xmlrpc.tracker.certprofile_plugin import >>> CertprofileTracker >>> E ImportError: No module named tracker.certprofile_plugin >>> >>> >>> .... >>> >>> N) >>> rpm -ql freeipa-tests | grep tracker >>> returns nothing, IMO the tracker directory in not in RPM > All failing parts PASSED. > >>> >>> On 30.11.2015 17:50, Ale? Mare?ek wrote: >>>> Tested with today's master, ACK. >>>> - alich - >>>> >>>> ----- Original Message ----- >>>>> From: "Milan Kub?k" >>>>> To: freeipa-devel at redhat.com >>>>> Sent: Friday, November 27, 2015 3:40:29 PM >>>>> Subject: Re: [Freeipa-devel] [patch 0025] Separated Tracker >>>>> implementations into standalone package >>>>> >>>>> On 11/27/2015 03:36 PM, Milan Kub?k wrote: >>>>> >>>>> >>>>> >>>>> On 11/27/2015 03:31 PM, Milan Kub?k wrote: >>>>> >>>>> >>>>> On 11/23/2015 10:43 AM, Lenka Doudova wrote: >>>>> >>>>> >>>>> NACK - there's a "typo" in ..../tracker/user_plugin.py, line 17-18: >>>>> >>>>> def get_user_dn(cn): >>>>> >>>>> return DN(('cn', cn), api.env.container_user, api.env.basedn) >>>>> >>>>> >>>>> should be >>>>> >>>>> def get_user_dn(uid): >>>>> >>>>> return DN(('uid', uid), api.env.container_user, api.env.basedn) >>>>> >>>>> >>>>> Some tests may fail because of that. >>>>> Lenka >>>>> >>>>> >>>>> On 11/20/2015 08:54 PM, Ale? Mare?ek wrote: >>>>> >>>>> >>>>> Looks good. ACK. >>>>> >>>>> ----- Original Message ----- >>>>> >>>>> >>>>> From: "Milan Kub?k" >>>>> To: "freeipa-devel" >>>>> Cc: "Filip Skola" , "Ales Marecek" >>>>> >>>>> Sent: Friday, November 20, 2015 3:44:30 PM >>>>> Subject: [patch 0025] Separated Tracker implementations into standalone >>>>> package >>>>> >>>>> Fixes https://fedorahosted.org/freeipa/ticket/5467 >>>>> Patches attached. >>>>> >>>>> -- >>>>> Milan Kubik >>>>> >>>>> >>>>> >>>>> Fixed the function and moved it into different module. >>>>> Updated patches attached. >>>>> >>>>> >>>>> >>>>> Self nack, some imports missing >>>>> >>>>> -- >>>>> Milan Kubik >>>>> >>>>> >>>>> Patches updated. >>>>> >>>>> -- >>>>> Milan Kubik >>>>> >>>>> -- >>>>> Manage your subscription for the Freeipa-devel mailing list: >>>>> https://www.redhat.com/mailman/listinfo/freeipa-devel >>>>> Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code >> My sincere apologies for this unspoken transgression on the packaging >> dieties. >> Updated patches attached. >> >> -- >> Milan Kubik >> >> Pushed to master: 17f9ca154b47f1e21797d25435e25676fdca284c I did not push patch to ipa-4-2 because ticket milestone is 4.3, if anybody disagree, say it now or be silent forever. From mbasti at redhat.com Wed Dec 2 16:24:27 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 2 Dec 2015 17:24:27 +0100 Subject: [Freeipa-devel] [PATCH 0364, 0367] ipa-kra-install: allow first KRA to be installed on replica In-Reply-To: <565EF7A8.7050304@redhat.com> References: <56587E92.7060008@redhat.com> <565C4C31.4070509@redhat.com> <565C877B.3050405@redhat.com> <565EF7A8.7050304@redhat.com> Message-ID: <565F1B3B.5080905@redhat.com> On 02.12.2015 14:52, Martin Babinsky wrote: > On 11/30/2015 06:29 PM, Martin Basti wrote: >> >> >> On 30.11.2015 14:16, Martin Babinsky wrote: >>> On 11/27/2015 05:02 PM, Martin Basti wrote: >>>> https://fedorahosted.org/freeipa/ticket/5460 >>>> >>>> I tested just master, I will test ipa-4-2 later. >>>> patch attached. >>>> >>>> >>> >>> ACK for the master branch. >>> >> Thanks, additional patch improves error message when ipa-replica-install >> --setup-ca --setup-kra is executed and KRA is not installed anywhere >> yet. >> >> I'm working on patches for ipa-4-2 branch >> >> Martin > > ACK for patch 367. > Pushed to master: bbbe411f357b7fbad533b5211a90bb0558b1abbe IPA 4.2 patches attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ipa42-mbasti-0364-ipa-kra-install-allow-to-install-first-KRA-on-replic.patch Type: text/x-patch Size: 2002 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ipa42-mbasti-0367-Modify-error-message-to-install-first-instance-of-KR.patch Type: text/x-patch Size: 1197 bytes Desc: not available URL: From lslebodn at redhat.com Wed Dec 2 16:25:20 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Wed, 2 Dec 2015 17:25:20 +0100 Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <565F0332.9000804@redhat.com> References: <565C3C6F.4040901@redhat.com> <20151130163256.GA7759@mail.corp.redhat.com> <565DD881.60401@redhat.com> <565EAAAE.8020306@redhat.com> <565F0332.9000804@redhat.com> Message-ID: <20151202162519.GA19962@mail.corp.redhat.com> On (02/12/15 15:41), Tomas Babej wrote: > > >On 12/02/2015 09:24 AM, Tomas Babej wrote: >> >> >> On 12/01/2015 06:27 PM, Tomas Babej wrote: >>> >>> >>> On 11/30/2015 05:32 PM, Lukas Slebodnik wrote: >>>> On (30/11/15 13:09), Tomas Babej wrote: >>>>> Hi, >>>>> >>>>> IPA sudo tests worked under the assumption that the clients that >>>>> are executing the sudo commands have their IPs assigned within >>>>> 255.255.255.0 hostmask. >>>>> >>>>> Removes this (invalid) assumption and adds a dynamic detection of >>>>> the hostmask of the IPA client. >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/5501 >>>> >>>> >From e6f1846f0d7d17303e5b30b1643651ba739b2b6c Mon Sep 17 00:00:00 2001 >>>>> From: Tomas Babej >>>>> Date: Mon, 30 Nov 2015 12:53:39 +0100 >>>>> Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on >>>>> hostmask >>>>> >>>>> IPA sudo tests worked under the assumption that the clients that >>>>> are executing the sudo commands have their IPs assigned within >>>>> 255.255.255.0 hostmask. >>>>> >>>>> Removes this (invalid) assumption and adds a dynamic detection of >>>>> the hostmask of the IPA client. >>>>> >>> >>> Thanks, updated patch attached. >>> >>> Tomas >>> >> >> Actually, a small improvement is necessary. >> >> Updated patch attached. >> >> Tomas >> >> >> > >Thanks to Lukas, we found another problem with the test. > >Updated patch attached. > Thank you for 4th revision of patch but there is still one issue. =================================== FAILURES =================================== _________ TestSudo.test_sudo_rule_restricted_to_one_hostmask_negative __________ self = def test_sudo_rule_restricted_to_one_hostmask_negative(self): result1 = self.list_sudo_commands("testuser1") > assert result1.returncode != 0 E assert 0 != 0 E + where 0 = .returncode test_integration/test_sudo.py:323: AssertionError ==================== 1 failed, 74 passed in 807.35 seconds ===================== LS From mbasti at redhat.com Wed Dec 2 16:28:57 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 2 Dec 2015 17:28:57 +0100 Subject: [Freeipa-devel] [PATCH] Removed duplicate domain name validation function In-Reply-To: <565DB6AA.6080306@redhat.com> References: <5655DF48.4060302@redhat.com> <56586981.2010703@redhat.com> <565869D0.4060606@redhat.com> <56586D27.4030802@redhat.com> <56587181.1070004@redhat.com> <565C9CEA.80105@redhat.com> <565D5C5E.1080707@redhat.com> <565D6116.9070203@redhat.com> <565DB6AA.6080306@redhat.com> Message-ID: <565F1C49.5000201@redhat.com> On 01.12.2015 16:03, Stanislav Laznicka wrote: > Sending the patch with renamed function. > > Standa > > On 12/01/2015 09:57 AM, Jan Cholasta wrote: >> On 1.12.2015 09:37, Petr Spacek wrote: >>> On 30.11.2015 20:00, Martin Basti wrote: >>>> >>>> >>>> On 27.11.2015 16:06, Stanislav Laznicka wrote: >>>>> Please, see the modified patch attached. >>>>> >>>>> Standa >>>>> >>>>> On 11/27/2015 03:48 PM, Martin Basti wrote: >>>>>> >>>>>> >>>>>> On 27.11.2015 15:33, Petr Spacek wrote: >>>>>>> On 27.11.2015 15:32, Martin Basti wrote: >>>>>>>> >>>>>>>> On 25.11.2015 17:18, Stanislav Laznicka wrote: >>>>>>>>> There were two functions for the same purpose. Removed one. >>>>>>>>> >>>>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> I would like to have "log" param of is_host_resolvable as optional >>>>>>> Is there an immediate need for the optional param? If not, I >>>>>>> would not >>>>>>> clutter >>>>>>> the code. >>>>>>> >>>>>> So at least I would like to move log param as the last param, in >>>>>> case of >>>>>> need it can be modified to optional parameter. >>>>>> >>>>>> Or log can be default as root_logger, IMO we us only root_logger >>>>>> everywhere, but this need investigation. >>>>>> >>>>>> Martin >>>>> >>>> It works, but I would like to have Honza's opinion if >>>> ipalib/util.py is the >>>> right place for the new method and if we really need to use >>>> exceptions. >> >> ipalib/util.py is fine for the function, since it is not used >> anywhere in ipapython. >> >> I would rename the function to verify_host_resolvable() though, since >> the is_ prefix suggests the function returns a boolean value rather >> than raises an exception or not. >> >>>> >>>> IMO is_record_resolvable() should return only True/False and then >>>> it can be >>>> located in ipapython module >>> >>> Hmm, I do not see a reason why it should not throw an exception. >>> IMHO the >>> exception should contain the original error/result from resolver, so >>> it is >>> possible to print meaningful error message like "DNS query timed >>> out" instead >>> of "it does not work for some reason, trust us". >>> >> >> > Pushed to master: 498471e4aed1367b72cd74d15811d0584a6ee268 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbasti at redhat.com Wed Dec 2 16:30:18 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 2 Dec 2015 17:30:18 +0100 Subject: [Freeipa-devel] [PATCH] Removed duplicate domain name validation function In-Reply-To: <565F1C49.5000201@redhat.com> References: <5655DF48.4060302@redhat.com> <56586981.2010703@redhat.com> <565869D0.4060606@redhat.com> <56586D27.4030802@redhat.com> <56587181.1070004@redhat.com> <565C9CEA.80105@redhat.com> <565D5C5E.1080707@redhat.com> <565D6116.9070203@redhat.com> <565DB6AA.6080306@redhat.com> <565F1C49.5000201@redhat.com> Message-ID: <565F1C9A.5000603@redhat.com> On 02.12.2015 17:28, Martin Basti wrote: > > > On 01.12.2015 16:03, Stanislav Laznicka wrote: >> Sending the patch with renamed function. >> >> Standa >> >> On 12/01/2015 09:57 AM, Jan Cholasta wrote: >>> On 1.12.2015 09:37, Petr Spacek wrote: >>>> On 30.11.2015 20:00, Martin Basti wrote: >>>>> >>>>> >>>>> On 27.11.2015 16:06, Stanislav Laznicka wrote: >>>>>> Please, see the modified patch attached. >>>>>> >>>>>> Standa >>>>>> >>>>>> On 11/27/2015 03:48 PM, Martin Basti wrote: >>>>>>> >>>>>>> >>>>>>> On 27.11.2015 15:33, Petr Spacek wrote: >>>>>>>> On 27.11.2015 15:32, Martin Basti wrote: >>>>>>>>> >>>>>>>>> On 25.11.2015 17:18, Stanislav Laznicka wrote: >>>>>>>>>> There were two functions for the same purpose. Removed one. >>>>>>>>>> >>>>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> I would like to have "log" param of is_host_resolvable as >>>>>>>>> optional >>>>>>>> Is there an immediate need for the optional param? If not, I >>>>>>>> would not >>>>>>>> clutter >>>>>>>> the code. >>>>>>>> >>>>>>> So at least I would like to move log param as the last param, in >>>>>>> case of >>>>>>> need it can be modified to optional parameter. >>>>>>> >>>>>>> Or log can be default as root_logger, IMO we us only root_logger >>>>>>> everywhere, but this need investigation. >>>>>>> >>>>>>> Martin >>>>>> >>>>> It works, but I would like to have Honza's opinion if >>>>> ipalib/util.py is the >>>>> right place for the new method and if we really need to use >>>>> exceptions. >>> >>> ipalib/util.py is fine for the function, since it is not used >>> anywhere in ipapython. >>> >>> I would rename the function to verify_host_resolvable() though, >>> since the is_ prefix suggests the function returns a boolean value >>> rather than raises an exception or not. >>> >>>>> >>>>> IMO is_record_resolvable() should return only True/False and then >>>>> it can be >>>>> located in ipapython module >>>> >>>> Hmm, I do not see a reason why it should not throw an exception. >>>> IMHO the >>>> exception should contain the original error/result from resolver, >>>> so it is >>>> possible to print meaningful error message like "DNS query timed >>>> out" instead >>>> of "it does not work for some reason, trust us". >>>> >>> >>> >> > Pushed to master: 498471e4aed1367b72cd74d15811d0584a6ee268 > > > Of course I forgot to add ACK -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbasti at redhat.com Wed Dec 2 16:34:04 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 2 Dec 2015 17:34:04 +0100 Subject: [Freeipa-devel] [PATCH 0368] FIX DNS tests Message-ID: <565F1D7C.8070407@redhat.com> Test needs to be updated due deprecation of dns-resolve command. Patch attached -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0368-Fix-DNS-tests-dns-resolve-returns-warning.patch Type: text/x-patch Size: 2113 bytes Desc: not available URL: From rcritten at redhat.com Wed Dec 2 18:14:21 2015 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 2 Dec 2015 13:14:21 -0500 Subject: [Freeipa-devel] patch acceptance criteria Message-ID: <565F34FD.5090204@redhat.com> Is it still mandatory that tests pass the unit tests before acceptance? I've seen a number of cases over the past couple of months where a change goes through then shortly afterward a patch to fix the tests. IMHO this should be caught in advance. Things slip through and goodness knows I've acked more than a few patches without running the full suite. I just have a feeling it has become more frequent lately. rob From simo at redhat.com Wed Dec 2 18:16:53 2015 From: simo at redhat.com (Simo Sorce) Date: Wed, 02 Dec 2015 13:16:53 -0500 Subject: [Freeipa-devel] [PATCH 556-557] Add option to disable setkeytab extended operations In-Reply-To: <565DC05B.1050804@redhat.com> References: <1448399848.29102.13.camel@redhat.com> <56555AC7.2000908@redhat.com> <5655BF5B.6020405@redhat.com> <1448462853.29102.44.camel@redhat.com> <1448908709.3747.93.camel@redhat.com> <20151201081145.GJ9605@redhat.com> <1448977085.9040.1.camel@redhat.com> <565DA3DE.8000109@redhat.com> <1448978341.9040.5.camel@redhat.com> <565DB84C.2010401@redhat.com> <20151201152042.GQ9605@redhat.com> <565DC05B.1050804@redhat.com> Message-ID: <1449080213.9040.57.camel@redhat.com> On Tue, 2015-12-01 at 16:44 +0100, Petr Vobornik wrote: > On 12/01/2015 04:20 PM, Alexander Bokovoy wrote: > > On Tue, 01 Dec 2015, Martin Kosek wrote: > >> On 12/01/2015 02:59 PM, Simo Sorce wrote: > >>> On Tue, 2015-12-01 at 14:42 +0100, Martin Kosek wrote: > >>>> On 12/01/2015 02:38 PM, Simo Sorce wrote: > >>>>> On Tue, 2015-12-01 at 10:11 +0200, Alexander Bokovoy wrote: > >>>>>> On Mon, 30 Nov 2015, Simo Sorce wrote: > >>>>>>> On Wed, 2015-11-25 at 09:47 -0500, Simo Sorce wrote: > >>>>>>>> On Wed, 2015-11-25 at 09:02 -0500, Rob Crittenden wrote: > >>>>>>>>> Jan Cholasta wrote: > >>>>>>>>>> On 24.11.2015 22:17, Simo Sorce wrote: > >>>>>>>>>>> On Tue, 2015-11-24 at 14:57 -0500, Simo Sorce wrote: > >>>>>>>>>>>> On Tue, 2015-11-24 at 14:42 -0500, Simo Sorce wrote: > >>>>>>>>>>>>> Since some time we use the getkeytab operation to fetch > >>>>>>>>>>>>> keytabs on > >>>>>>>>>>>>> newer > >>>>>>>>>>>>> clients. According to bug #232 setkeytab can be used to > >>>>>>>>>>>>> circumvent > >>>>>>>>>>>>> password quality controls so it needs to be slowly retired. > >>>>>>>>>>>>> > >>>>>>>>>>>>> The attached patches implement #5485 in 2 parts. > >>>>>>>>>>>>> > >>>>>>>>>>>>> The first introduces the option DisableSetKeytab which > >>>>>>>>>>>>> globally > >>>>>>>>>>>>> disables > >>>>>>>>>>>>> the setkeytab extended operation. This is set to false by > >>>>>>>>>>>>> default for > >>>>>>>>>>>>> backwards compatibility. > >>>>>>>>>>>>> > >>>>>>>>>>>>> The second introduces an option called > >>>>>>>>>>>>> DisableUserSetKeytab, which is > >>>>>>>>>>>>> active by default in new installs (but not in upgraded > >>>>>>>>>>>>> ones), and only > >>>>>>>>>>>>> disables the use of setkeytab for ipa suers, but not for > >>>>>>>>>>>>> hosts/services. > >>>>>>>>>>>>> This is because user's are the ones that may abuse the > >>>>>>>>>>>>> interface to > >>>>>>>>>>>>> escape password policies and users also normally do not > >>>>>>>>>>>>> acquire > >>>>>>>>>>>>> keytabs, > >>>>>>>>>>>>> so it is a safe bet to disable just them by default in new > >>>>>>>>>>>>> installs. > >>>>>>>>>>>>> > >>>>>>>>>>>>> (Testing in progress) > >>>>>>>>>>>> > >>>>>>>>>>>> Tested and working as expected. > >>>>>>>>>>> > >>>>>>>>>>> I realized that adding options to ipaConfig require to add > >>>>>>>>>>> them in the > >>>>>>>>>>> UI as well, attached patches add options in API.txt and > >>>>>>>>>>> config.py > >>>>>>>>>>> Make now complain I should change API Major or Minor, but it > >>>>>>>>>>> is not > >>>>>>>>>>> clear to me why given this are additional values and no real > >>>>>>>>>>> change or > >>>>>>>>>>> new function is introduced. What's the recommendation ? > >>>>>>>>>> > >>>>>>>>>> When does make complain? It is supposed to complain only when > >>>>>>>>>> API.txt > >>>>>>>>>> does not match code. > >>>>>>>>>> > >>>>>>>>>> Anyway, we usually bump minor version even for backward > >>>>>>>>>> compatible > >>>>>>>>>> changes, see e.g. commit 9549a59. > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> The point of API.txt (and the heavy client) was to save a > >>>>>>>>> round-trip. > >>>>>>>>> Being able to pass in an invalid option would void that rule hence > >>>>>>>>> having to update the API when new values are added. > >>>>>>>>> > >>>>>>>>> rob > >>>>>>>> > >>>>>>>> Ok added version change to the second patch (so we bump it only > >>>>>>>> once > >>>>>>>> given these are basically related changes. > >>>>>>> > >>>>>>> Bump, is this ok ? > >>>>>> This patch is fine but please fix setkeytab use in ipa-sam before > >>>>>> committing this patch. > >>>>> > >>>>> This patch does not disable setkeytab yet, so it can go in right away > >>>>> (it blocks other patches already). We have a bug to change ipa-sam, > >>>>> should we mark it blocker for 4.4 ? > >>>> > >>>> We also have to fix the permission to change keytab, so that it uses > >>>> the new > >>>> style (https://fedorahosted.org/freeipa/ticket/5487). I would > >>>> actually make > >>>> this ticket and the ipa-sam ticket a blocker for this patch set. > >>>> > >>>> Otherwise you are actually introducing a switch that breaks FreeIPA > >>>> as some of > >>>> it's core server functions still use the old method. > >>> > >>> The point of this patchset is to introduce the switch early so that > >>> current server will support the off switch when newer servers down the > >>> road are ready to flip it. The defaults are still to allow these > >>> operations for services/hosts. > >> > >> I still do not get the logic about an early switch. Now, if switch is > >> turned > >> on, FreeIPA server breaks on several places. I would really rather > >> expect the > >> switch to be introduced when the server itself supports it. Then, > >> admin would > >> enable it when the clients are sufficiently updated and can use the > >> new method. > >> > >> Why would admin want to enable the switch early if it breaks FreeIPA > >> some of > >> the FreeIPA servers? Permission can be upgraded in newer version and get > >> replicated, that's fine. But ipa-sam would be still broken on this old > >> FreeIPA > >> server. > > Old server is not a problem here: ipa-sam only talks to the > > localhost-based server as we always use ldapi protocol. So if server is > > running old-behavior FreeIPA, ipa-sam on the same server will work > > against it. > > > > What I don't want to have is a situation where setkeytab is disabled and > > a new server obeys it but ipa-sam on this new server is not updated and > > will expectedly fail. We are not that forced to do the change right now > > in 4.3, given that the default will still be to keep setkeytab, thus we > > can wait with this patch until ipa-sam is fixed and then push both > > patches into the closest release, be it 4.3.x (x>=0) or whatever is the > > next one. > > > > +1, fixing ipa-sam would be then a release blocker for 4.3 which is not > desired. > > Also what about adding support for "ipaProtectedoperation check" for > user principals? > > I'm afraid that forbidding getting user principal might be regarded as a > regression which might cause that admins won't set DisableSetKeytab. One thing at a time. We have bugs open for all these things, but I see no reason not to add an *optional* setting just because some things break when it is turned on. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Wed Dec 2 18:25:51 2015 From: simo at redhat.com (Simo Sorce) Date: Wed, 02 Dec 2015 13:25:51 -0500 Subject: [Freeipa-devel] [PATCH] Add option to disable setkeytab extended operations In-Reply-To: <565EA075.7010406@redhat.com> References: <1448394152.29102.10.camel@redhat.com> <565D5735.7080405@redhat.com> <20151201082128.GL9605@redhat.com> <565D5BA2.7080608@redhat.com> <20151201084720.GM9605@redhat.com> <565D685E.9080704@redhat.com> <20151201095348.GO9605@redhat.com> <20151201110057.GP9605@redhat.com> <565EA075.7010406@redhat.com> Message-ID: <1449080751.9040.62.camel@redhat.com> On Wed, 2015-12-02 at 08:40 +0100, Petr Spacek wrote: > On 1.12.2015 12:00, Alexander Bokovoy wrote: > > On Tue, 01 Dec 2015, Alexander Bokovoy wrote: > >> On Tue, 01 Dec 2015, Petr Spacek wrote: > >>> On 1.12.2015 09:47, Alexander Bokovoy wrote: > >>>> On Tue, 01 Dec 2015, Petr Spacek wrote: > >>>>> On 1.12.2015 09:21, Alexander Bokovoy wrote: > >>>>>> On Tue, 01 Dec 2015, Petr Spacek wrote: > >>>>>>> On 24.11.2015 20:42, Simo Sorce wrote: > >>>>>>>> Since some time we use the getkeytab operation to fetch keytabs on newer > >>>>>>>> clients. According to bug #232 setkeytab can be used to circumvent > >>>>>>>> password quality controls so it needs to be slowly retired. > >>>>>>>> > >>>>>>>> The attached patches implement #5485 in 2 parts. > >>>>>>>> > >>>>>>>> The first introduces the option DisableSetKeytab which globally disables > >>>>>>>> the setkeytab extended operation. This is set to false by default for > >>>>>>>> backwards compatibility. > >>>>>>>> > >>>>>>>> The second introduces an option called DisableUserSetKeytab, which is > >>>>>>>> active by default in new installs (but not in upgraded ones), and only > >>>>>>>> disables the use of setkeytab for ipa suers, but not for hosts/services. > >>>>>>>> This is because user's are the ones that may abuse the interface to > >>>>>>>> escape password policies and users also normally do not acquire keytabs, > >>>>>>>> so it is a safe bet to disable just them by default in new installs. > >>>>>>> > >>>>>>> On a related note, how this works with plain kadmin & kpasswd protocols? > >>>>>> It is unrelated. We don't support principal manipulation via kadmin > >>>>>> protocol. > >>>>> > >>>>> Sure, I know that, but I'm trying to find out if we re-invented the wheel or > >>>>> if our wheel has some additional features which cannot be incorporated to > >>>>> the > >>>>> original wheel :-) > >>>> There is no need to incorporate something specific into kadmin protocol, > >>>> the problem is with the fact that we don't have access controls within > >>>> our KDC driver. It always connects to LDAP server over ldapi as root and > >>>> thus maps to cn=Directory Manager which bypasses LDAP ACIs. As such, > >>>> this means we'll need to have some access control added to KDC DAL > >>>> driver before we can allow kadmin operations on principals. > >>>> > >>>> Adding those access controls is not an easy feat. > >>> > >>> Hmm, why is that? We could use LDAP Proxy control (RFC 4370) and let DS to > >>> evaluate ACIs as usual. The change in kadmin would be adding LDAP Proxy > >>> control with proper DN (i.e. mapping from principal name to DN) to the LDAP > >>> requests. > >> Not sure how does it help: > >> $ ldapsearch -Dcn=Directory\ Manager -W -e > >> \!authzid=dn:uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li -b > >> uid=admin,cn=users,cn=accounts,dc=vda,dc=li userPassword > >> Enter LDAP Password: # extended LDIF > >> # > >> # LDAPv3 > >> # base with scope subtree > >> # filter: (objectclass=*) > >> # requesting: userPassword # > >> > >> # admin, users, accounts, vda.li > >> dn: uid=admin,cn=users,cn=accounts,dc=vda,dc=li > >> userPassword:: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX= > >> = > >> > >> # search result > >> search: 2 > >> result: 0 Success > >> > >> # numResponses: 2 > >> # numEntries: 1 > >> $ ldapsearch -Y GSSAPI -e > >> \!authzid=dn:uid=admin,cn=users,cn=accounts,dc=vda,dc=li -b > >> uid=admin,cn=users,cn=accounts,dc=vda,dc=li userPassword > >> SASL/GSSAPI authentication started > >> SASL username: abokovoy at VDA.LI > >> SASL SSF: 56 > >> SASL data security layer installed. > >> # extended LDIF > >> # > >> # LDAPv3 > >> # base with scope subtree > >> # filter: (objectclass=*) > >> # requesting: userPassword # > >> > >> # search result > >> search: 4 > >> result: 0 Success > >> > >> # numResponses: 1 > >> > >> As you can see, posing different authzid does not allow to get through > >> ACLs unless you were already a directory manager. Which means the > >> proxyauth control is not really useful here: > >> > >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=0 BIND dn="cn=Directory Manager" > >> method=128 version=3 > >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=0 RESULT err=0 tag=97 nentries=0 > >> etime=0 dn="cn=directory manager" > >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=1 SRCH > >> base="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" scope=2 > >> filter="(objectClass=*)" attrs="userPassword" > >> authzid="uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li" > >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=1 RESULT err=0 tag=101 nentries=1 > >> etime=0 notes=U > >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=2 UNBIND > >> > >> and > >> > >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=0 BIND dn="" method=sasl > >> version=3 mech=GSSAPI > >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=0 RESULT err=14 tag=97 nentries=0 > >> etime=0, SASL bind in progress > >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=1 BIND dn="" method=sasl > >> version=3 mech=GSSAPI > >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=1 RESULT err=14 tag=97 nentries=0 > >> etime=0, SASL bind in progress > >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=2 BIND dn="" method=sasl > >> version=3 mech=GSSAPI > >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=2 RESULT err=0 tag=97 nentries=0 > >> etime=0 dn="uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li" > >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=3 SRCH > >> base="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" scope=2 > >> filter="(objectClass=*)" attrs="userPassword" > >> authzid="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" > >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=3 RESULT err=0 tag=101 nentries=0 > >> etime=0 notes=U > >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=4 UNBIND > >> > > Ludwig confirmed that ACL plugin short-circuits DM access before > > proxy auth checked. I've filed a bug about it: > > https://fedorahosted.org/389/ticket/48366 Note that the way kadmin is written, the proxy control won't help. The DAL driver layer simply have *no* knowledge of what principal connected to kadmin when kadmin makes requests, and we do not reconnect at each request anyway. We asked MIT to provide that information down to the DAL layer so we *can* try to use the proxy control already. It will take soime time though because the code paths in kadmin need to be change to allow something like that. > But now back to the original question: > Do kadmin/kpasswd support password policies? Yes, but we implement our own in the DAL driver. > If it is the case, why it works with kadmin/kpasswd and does not work with our > setkeytab? Because I was stupid when I invented setkeytab and did not think that it could be used to circumvent password policies. So I had to add a new control called getkeytab that works as needed and also adds the ability to retrieve an existing key, not just set one. So we fixed a bug and gained new functionality, we just need to deprecate the old control. Once we'll be able to have kadmin authorize properly based on DS ACLs, we can allow the use of the kadmin interface too, but that will take time. Simo. -- Simo Sorce * Red Hat, Inc * New York From pspacek at redhat.com Thu Dec 3 08:00:09 2015 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 3 Dec 2015 09:00:09 +0100 Subject: [Freeipa-devel] [PATCH] 494 Update Contributors.txt In-Reply-To: <565ED814.1060201@redhat.com> References: <565ED508.2050308@redhat.com> <565ED798.2080109@redhat.com> <565ED814.1060201@redhat.com> Message-ID: <565FF689.5050303@redhat.com> On 2.12.2015 12:37, Martin Kosek wrote: > On 12/02/2015 12:35 PM, Martin Basti wrote: >> >> >> On 02.12.2015 12:24, Martin Kosek wrote: >>> +Thierry Bordaz >>> +Thierry Bordaz >> >> Do we want this there? > > I do not want it there, but I had to add it since Author field in git was wrong > when the patch was pushed. > > So I blame the original author and the reviewer for making me add this mail > alias :-) +1 @Martin^2: Again, DNS names are not secret. Really. If they were we would have a bigger problem: Your e-mail headers contain DNS names and IP addresses of internal mail servers and of course the machine you used to send the e-mail. Hello, dhcp129-150.brq.redhat.com aka 10.34.129.150! :-) -- Petr^2 Spacek From pspacek at redhat.com Thu Dec 3 08:08:59 2015 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 3 Dec 2015 09:08:59 +0100 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <565F34FD.5090204@redhat.com> References: <565F34FD.5090204@redhat.com> Message-ID: <565FF89B.6090309@redhat.com> On 2.12.2015 19:14, Rob Crittenden wrote: > Is it still mandatory that tests pass the unit tests before acceptance? > I've seen a number of cases over the past couple of months where a > change goes through then shortly afterward a patch to fix the tests. > IMHO this should be caught in advance. > > Things slip through and goodness knows I've acked more than a few > patches without running the full suite. I just have a feeling it has > become more frequent lately. When we are at it... An automated thingy which accepts URL to a Git repo, does all the test magic, and spits out test results without user interaction would be an awesome Christmas present! Bonus points if we can get Github integration so I can just push and have it tested automatically so I cannot forget to do that before sending the patch for review. -- Petr^2 Spacek From mkosek at redhat.com Thu Dec 3 08:18:34 2015 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 3 Dec 2015 09:18:34 +0100 Subject: [Freeipa-devel] [PATCH 556-557] Add option to disable setkeytab extended operations In-Reply-To: <1449080213.9040.57.camel@redhat.com> References: <1448399848.29102.13.camel@redhat.com> <56555AC7.2000908@redhat.com> <5655BF5B.6020405@redhat.com> <1448462853.29102.44.camel@redhat.com> <1448908709.3747.93.camel@redhat.com> <20151201081145.GJ9605@redhat.com> <1448977085.9040.1.camel@redhat.com> <565DA3DE.8000109@redhat.com> <1448978341.9040.5.camel@redhat.com> <565DB84C.2010401@redhat.com> <20151201152042.GQ9605@redhat.com> <565DC05B.1050804@redhat.com> <1449080213.9040.57.camel@redhat.com> Message-ID: <565FFADA.5070607@redhat.com> On 12/02/2015 07:16 PM, Simo Sorce wrote: > On Tue, 2015-12-01 at 16:44 +0100, Petr Vobornik wrote: >> On 12/01/2015 04:20 PM, Alexander Bokovoy wrote: >>> On Tue, 01 Dec 2015, Martin Kosek wrote: >>>> On 12/01/2015 02:59 PM, Simo Sorce wrote: >>>>> On Tue, 2015-12-01 at 14:42 +0100, Martin Kosek wrote: >>>>>> On 12/01/2015 02:38 PM, Simo Sorce wrote: >>>>>>> On Tue, 2015-12-01 at 10:11 +0200, Alexander Bokovoy wrote: >>>>>>>> On Mon, 30 Nov 2015, Simo Sorce wrote: >>>>>>>>> On Wed, 2015-11-25 at 09:47 -0500, Simo Sorce wrote: >>>>>>>>>> On Wed, 2015-11-25 at 09:02 -0500, Rob Crittenden wrote: >>>>>>>>>>> Jan Cholasta wrote: >>>>>>>>>>>> On 24.11.2015 22:17, Simo Sorce wrote: >>>>>>>>>>>>> On Tue, 2015-11-24 at 14:57 -0500, Simo Sorce wrote: >>>>>>>>>>>>>> On Tue, 2015-11-24 at 14:42 -0500, Simo Sorce wrote: >>>>>>>>>>>>>>> Since some time we use the getkeytab operation to fetch >>>>>>>>>>>>>>> keytabs on >>>>>>>>>>>>>>> newer >>>>>>>>>>>>>>> clients. According to bug #232 setkeytab can be used to >>>>>>>>>>>>>>> circumvent >>>>>>>>>>>>>>> password quality controls so it needs to be slowly retired. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The attached patches implement #5485 in 2 parts. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The first introduces the option DisableSetKeytab which >>>>>>>>>>>>>>> globally >>>>>>>>>>>>>>> disables >>>>>>>>>>>>>>> the setkeytab extended operation. This is set to false by >>>>>>>>>>>>>>> default for >>>>>>>>>>>>>>> backwards compatibility. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The second introduces an option called >>>>>>>>>>>>>>> DisableUserSetKeytab, which is >>>>>>>>>>>>>>> active by default in new installs (but not in upgraded >>>>>>>>>>>>>>> ones), and only >>>>>>>>>>>>>>> disables the use of setkeytab for ipa suers, but not for >>>>>>>>>>>>>>> hosts/services. >>>>>>>>>>>>>>> This is because user's are the ones that may abuse the >>>>>>>>>>>>>>> interface to >>>>>>>>>>>>>>> escape password policies and users also normally do not >>>>>>>>>>>>>>> acquire >>>>>>>>>>>>>>> keytabs, >>>>>>>>>>>>>>> so it is a safe bet to disable just them by default in new >>>>>>>>>>>>>>> installs. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> (Testing in progress) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Tested and working as expected. >>>>>>>>>>>>> >>>>>>>>>>>>> I realized that adding options to ipaConfig require to add >>>>>>>>>>>>> them in the >>>>>>>>>>>>> UI as well, attached patches add options in API.txt and >>>>>>>>>>>>> config.py >>>>>>>>>>>>> Make now complain I should change API Major or Minor, but it >>>>>>>>>>>>> is not >>>>>>>>>>>>> clear to me why given this are additional values and no real >>>>>>>>>>>>> change or >>>>>>>>>>>>> new function is introduced. What's the recommendation ? >>>>>>>>>>>> >>>>>>>>>>>> When does make complain? It is supposed to complain only when >>>>>>>>>>>> API.txt >>>>>>>>>>>> does not match code. >>>>>>>>>>>> >>>>>>>>>>>> Anyway, we usually bump minor version even for backward >>>>>>>>>>>> compatible >>>>>>>>>>>> changes, see e.g. commit 9549a59. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> The point of API.txt (and the heavy client) was to save a >>>>>>>>>>> round-trip. >>>>>>>>>>> Being able to pass in an invalid option would void that rule hence >>>>>>>>>>> having to update the API when new values are added. >>>>>>>>>>> >>>>>>>>>>> rob >>>>>>>>>> >>>>>>>>>> Ok added version change to the second patch (so we bump it only >>>>>>>>>> once >>>>>>>>>> given these are basically related changes. >>>>>>>>> >>>>>>>>> Bump, is this ok ? >>>>>>>> This patch is fine but please fix setkeytab use in ipa-sam before >>>>>>>> committing this patch. >>>>>>> >>>>>>> This patch does not disable setkeytab yet, so it can go in right away >>>>>>> (it blocks other patches already). We have a bug to change ipa-sam, >>>>>>> should we mark it blocker for 4.4 ? >>>>>> >>>>>> We also have to fix the permission to change keytab, so that it uses >>>>>> the new >>>>>> style (https://fedorahosted.org/freeipa/ticket/5487). I would >>>>>> actually make >>>>>> this ticket and the ipa-sam ticket a blocker for this patch set. >>>>>> >>>>>> Otherwise you are actually introducing a switch that breaks FreeIPA >>>>>> as some of >>>>>> it's core server functions still use the old method. >>>>> >>>>> The point of this patchset is to introduce the switch early so that >>>>> current server will support the off switch when newer servers down the >>>>> road are ready to flip it. The defaults are still to allow these >>>>> operations for services/hosts. >>>> >>>> I still do not get the logic about an early switch. Now, if switch is >>>> turned >>>> on, FreeIPA server breaks on several places. I would really rather >>>> expect the >>>> switch to be introduced when the server itself supports it. Then, >>>> admin would >>>> enable it when the clients are sufficiently updated and can use the >>>> new method. >>>> >>>> Why would admin want to enable the switch early if it breaks FreeIPA >>>> some of >>>> the FreeIPA servers? Permission can be upgraded in newer version and get >>>> replicated, that's fine. But ipa-sam would be still broken on this old >>>> FreeIPA >>>> server. >>> Old server is not a problem here: ipa-sam only talks to the >>> localhost-based server as we always use ldapi protocol. So if server is >>> running old-behavior FreeIPA, ipa-sam on the same server will work >>> against it. >>> >>> What I don't want to have is a situation where setkeytab is disabled and >>> a new server obeys it but ipa-sam on this new server is not updated and >>> will expectedly fail. We are not that forced to do the change right now >>> in 4.3, given that the default will still be to keep setkeytab, thus we >>> can wait with this patch until ipa-sam is fixed and then push both >>> patches into the closest release, be it 4.3.x (x>=0) or whatever is the >>> next one. >>> >> >> +1, fixing ipa-sam would be then a release blocker for 4.3 which is not >> desired. >> >> Also what about adding support for "ipaProtectedoperation check" for >> user principals? >> >> I'm afraid that forbidding getting user principal might be regarded as a >> regression which might cause that admins won't set DisableSetKeytab. > > One thing at a time. > We have bugs open for all these things, but I see no reason not to add > an *optional* setting just because some things break when it is turned > on. One thing at a time, yes - but at the right order. I would only agree with you if you add a proper label for this optional option or rename it to DisableUserSetKeytabAndBreakIPASamAndBreakKeytabRenewalToo :-) Martin From mkosek at redhat.com Thu Dec 3 08:22:13 2015 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 3 Dec 2015 09:22:13 +0100 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <565FF89B.6090309@redhat.com> References: <565F34FD.5090204@redhat.com> <565FF89B.6090309@redhat.com> Message-ID: <565FFBB5.1030804@redhat.com> On 12/03/2015 09:08 AM, Petr Spacek wrote: > On 2.12.2015 19:14, Rob Crittenden wrote: >> Is it still mandatory that tests pass the unit tests before acceptance? >> I've seen a number of cases over the past couple of months where a >> change goes through then shortly afterward a patch to fix the tests. >> IMHO this should be caught in advance. >> >> Things slip through and goodness knows I've acked more than a few >> patches without running the full suite. I just have a feeling it has >> become more frequent lately. > > When we are at it... An automated thingy which accepts URL to a Git repo, does > all the test magic, and spits out test results without user interaction would > be an awesome Christmas present! > > Bonus points if we can get Github integration so I can just push and have it > tested automatically so I cannot forget to do that before sending the patch > for review. +1. Having basic CI test suite run on top of a Pull Request would be awesome. From jcholast at redhat.com Thu Dec 3 08:32:57 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 3 Dec 2015 09:32:57 +0100 Subject: [Freeipa-devel] [PATCH 0069] ipa-replica-install support caless install with promotion. In-Reply-To: <565EC390.5060806@redhat.com> References: <565710A8.1020306@redhat.com> <56571376.6020209@redhat.com> <565715BB.9020200@redhat.com> <5657FEC2.3080503@redhat.com> <565C7858.1030702@redhat.com> <565DA040.4000509@redhat.com> <565E96AB.8090402@redhat.com> <565EC390.5060806@redhat.com> Message-ID: <565FFE39.60902@redhat.com> On 2.12.2015 11:10, David Kupka wrote: > On 02/12/15 07:58, Jan Cholasta wrote: >> On 1.12.2015 14:27, David Kupka wrote: >>> On 30/11/15 17:24, Jan Cholasta wrote: >>>> Hi, >>>> >>>> On 27.11.2015 07:57, David Kupka wrote: >>>>> On 26/11/15 15:22, David Kupka wrote: >>>>>> On 26/11/15 15:13, David Kupka wrote: >>>>>>> On 26/11/15 15:01, David Kupka wrote: >>>>>>>> https://fedorahosted.org/freeipa/ticket/5441 >>>>>>>> >>>>>>>> >>>>>>> Replaced accidentally inserted tabs. >>>>>>> >>>>>>> >>>>>>> >>>>>> Fixed indentation I screwed up when replacing tabs :-/ >>>> >>>> 1) The deprecated --*_pkcs12 and --*_pin aliases should not be >>>> supported >>>> in ipa-replica-install. >> >> In ServerCA, inherit the knobs from BaseServerCA rather than >> BaseServer.ca. The "#pylint: disable=no-member" will no longer be >> necessary. >> >> In ipa-server-install help, there are 2 "certificate system" option >> groups. This is a shortcoming in the installer framework, which will be >> addressed in the future. For now, please inherit *all* knobs of >> BaseServerCA in ServerCA as a workaround. >> >>>> >>>> >>>> 2) This check from ipa-replica-prepare should be added to >>>> Replica.__init__() as well: >>>> >>>> # If any of the PKCS#12 options are selected, all are >>>> required. >>>> cert_file_req = (options.dirsrv_cert_files, >>>> options.http_cert_files) >>>> cert_file_opt = (options.pkinit_cert_files,) >>>> if any(cert_file_req + cert_file_opt) and not >>>> all(cert_file_req): >>>> self.option_parser.error( >>>> "--dirsrv-cert-file and --http-cert-file are required >>>> if any " >>>> "PKCS#12 options are used.") >> >> The check is done when replica file is specified in the patch, but it >> should be done only when replica file is *not* specified. >> >>>> 6) Please make the ca_is_enabled argument of install_replica_ds() and >>>> install_http() mandatory and fill as appropriate when called, it will >>>> make the code more readable. >> >> This bit in install_http() is redundant now: >> >> + if ca_is_configured is None: >> + ca_is_configured = ipautil.file_exists(config.dir + >> "/cacert.p12") >> >>>> >>>> >>>> 7) >>>> >>>> $ git diff -U0 | pep8 --diff >>>> ./ipaserver/install/server/replicainstall.py:99:80: E501 line too long >>>> (82 > 79 characters) >>>> ./ipaserver/install/server/replicainstall.py:161:80: E501 line too long >>>> (82 > 79 characters) >>>> ./ipaserver/install/server/replicainstall.py:1289:13: E265 block >>>> comment >>>> should start with '# ' >>>> ./ipaserver/install/server/replicainstall.py:1291:17: E125 continuation >>>> line with same indent as next logical line >>>> ./ipaserver/install/server/replicainstall.py:1291:17: E128 continuation >>>> line under-indented for visual indent >> >> $ git diff -U0 | pep8 --diff >> ./ipaserver/install/server/install.py:1142:1: E302 expected 2 blank >> lines, found 1 >> ./ipaserver/install/server/install.py:1143:5: E265 block comment should >> start with '# ' >> ./ipaserver/install/server/install.py:1160:17: E222 multiple spaces >> after operator >> ./ipaserver/install/server/install.py:1288:9: E265 block comment should >> start with '# ' >> ./ipaserver/install/server/replicainstall.py:100:80: E501 line too long >> (82 > 79 characters) >> ./ipaserver/install/server/replicainstall.py:162:80: E501 line too long >> (82 > 79 characters) >> ./ipaserver/install/server/replicainstall.py:697:41: E251 unexpected >> spaces around keyword / parameter equals >> ./ipaserver/install/server/replicainstall.py:697:43: E251 unexpected >> spaces around keyword / parameter equals >> ./ipaserver/install/server/replicainstall.py:922:9: E129 visually >> indented line with same indent as next logical line >> ./ipaserver/install/server/replicainstall.py:925:14: E131 continuation >> line unaligned for hanging indent >> ./ipaserver/install/server/replicainstall.py:1345:9: E265 block comment >> should start with '# ' >> ./ipaserver/install/server/replicainstall.py:1389:21: E128 continuation >> line under-indented for visual indent >> >> > Thanks, updated patch attached. Thanks, ACK. The patch needed a rebase, see attachment. Pushed to master: 2f51f0dce2e804bc4661441f97c04dc84b84fa21 -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0069.5-ipa-replica-install-support-caless-install-with-prom.patch Type: text/x-patch Size: 21255 bytes Desc: not available URL: From pvoborni at redhat.com Thu Dec 3 08:31:58 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 3 Dec 2015 09:31:58 +0100 Subject: [Freeipa-devel] [PATCH 556-557] Add option to disable setkeytab extended operations In-Reply-To: <1449080213.9040.57.camel@redhat.com> References: <1448399848.29102.13.camel@redhat.com> <56555AC7.2000908@redhat.com> <5655BF5B.6020405@redhat.com> <1448462853.29102.44.camel@redhat.com> <1448908709.3747.93.camel@redhat.com> <20151201081145.GJ9605@redhat.com> <1448977085.9040.1.camel@redhat.com> <565DA3DE.8000109@redhat.com> <1448978341.9040.5.camel@redhat.com> <565DB84C.2010401@redhat.com> <20151201152042.GQ9605@redhat.com> <565DC05B.1050804@redhat.com> <1449080213.9040.57.camel@redhat.com> Message-ID: <565FFDFE.5010604@redhat.com> On 12/02/2015 07:16 PM, Simo Sorce wrote: > On Tue, 2015-12-01 at 16:44 +0100, Petr Vobornik wrote: >> On 12/01/2015 04:20 PM, Alexander Bokovoy wrote: >>> On Tue, 01 Dec 2015, Martin Kosek wrote: >>>> On 12/01/2015 02:59 PM, Simo Sorce wrote: >>>>> On Tue, 2015-12-01 at 14:42 +0100, Martin Kosek wrote: >>>>>> On 12/01/2015 02:38 PM, Simo Sorce wrote: >>>>>>> On Tue, 2015-12-01 at 10:11 +0200, Alexander Bokovoy wrote: >>>>>>>> On Mon, 30 Nov 2015, Simo Sorce wrote: >>>>>>>>> On Wed, 2015-11-25 at 09:47 -0500, Simo Sorce wrote: >>>>>>>>>> On Wed, 2015-11-25 at 09:02 -0500, Rob Crittenden wrote: >>>>>>>>>>> Jan Cholasta wrote: >>>>>>>>>>>> On 24.11.2015 22:17, Simo Sorce wrote: >>>>>>>>>>>>> On Tue, 2015-11-24 at 14:57 -0500, Simo Sorce wrote: >>>>>>>>>>>>>> On Tue, 2015-11-24 at 14:42 -0500, Simo Sorce wrote: >>>>>>>>>>>>>>> Since some time we use the getkeytab operation to fetch >>>>>>>>>>>>>>> keytabs on >>>>>>>>>>>>>>> newer >>>>>>>>>>>>>>> clients. According to bug #232 setkeytab can be used to >>>>>>>>>>>>>>> circumvent >>>>>>>>>>>>>>> password quality controls so it needs to be slowly retired. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The attached patches implement #5485 in 2 parts. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The first introduces the option DisableSetKeytab which >>>>>>>>>>>>>>> globally >>>>>>>>>>>>>>> disables >>>>>>>>>>>>>>> the setkeytab extended operation. This is set to false by >>>>>>>>>>>>>>> default for >>>>>>>>>>>>>>> backwards compatibility. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The second introduces an option called >>>>>>>>>>>>>>> DisableUserSetKeytab, which is >>>>>>>>>>>>>>> active by default in new installs (but not in upgraded >>>>>>>>>>>>>>> ones), and only >>>>>>>>>>>>>>> disables the use of setkeytab for ipa suers, but not for >>>>>>>>>>>>>>> hosts/services. >>>>>>>>>>>>>>> This is because user's are the ones that may abuse the >>>>>>>>>>>>>>> interface to >>>>>>>>>>>>>>> escape password policies and users also normally do not >>>>>>>>>>>>>>> acquire >>>>>>>>>>>>>>> keytabs, >>>>>>>>>>>>>>> so it is a safe bet to disable just them by default in new >>>>>>>>>>>>>>> installs. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> (Testing in progress) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Tested and working as expected. >>>>>>>>>>>>> >>>>>>>>>>>>> I realized that adding options to ipaConfig require to add >>>>>>>>>>>>> them in the >>>>>>>>>>>>> UI as well, attached patches add options in API.txt and >>>>>>>>>>>>> config.py >>>>>>>>>>>>> Make now complain I should change API Major or Minor, but it >>>>>>>>>>>>> is not >>>>>>>>>>>>> clear to me why given this are additional values and no real >>>>>>>>>>>>> change or >>>>>>>>>>>>> new function is introduced. What's the recommendation ? >>>>>>>>>>>> >>>>>>>>>>>> When does make complain? It is supposed to complain only when >>>>>>>>>>>> API.txt >>>>>>>>>>>> does not match code. >>>>>>>>>>>> >>>>>>>>>>>> Anyway, we usually bump minor version even for backward >>>>>>>>>>>> compatible >>>>>>>>>>>> changes, see e.g. commit 9549a59. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> The point of API.txt (and the heavy client) was to save a >>>>>>>>>>> round-trip. >>>>>>>>>>> Being able to pass in an invalid option would void that rule hence >>>>>>>>>>> having to update the API when new values are added. >>>>>>>>>>> >>>>>>>>>>> rob >>>>>>>>>> >>>>>>>>>> Ok added version change to the second patch (so we bump it only >>>>>>>>>> once >>>>>>>>>> given these are basically related changes. >>>>>>>>> >>>>>>>>> Bump, is this ok ? >>>>>>>> This patch is fine but please fix setkeytab use in ipa-sam before >>>>>>>> committing this patch. >>>>>>> >>>>>>> This patch does not disable setkeytab yet, so it can go in right away >>>>>>> (it blocks other patches already). We have a bug to change ipa-sam, >>>>>>> should we mark it blocker for 4.4 ? >>>>>> >>>>>> We also have to fix the permission to change keytab, so that it uses >>>>>> the new >>>>>> style (https://fedorahosted.org/freeipa/ticket/5487). I would >>>>>> actually make >>>>>> this ticket and the ipa-sam ticket a blocker for this patch set. >>>>>> >>>>>> Otherwise you are actually introducing a switch that breaks FreeIPA >>>>>> as some of >>>>>> it's core server functions still use the old method. >>>>> >>>>> The point of this patchset is to introduce the switch early so that >>>>> current server will support the off switch when newer servers down the >>>>> road are ready to flip it. The defaults are still to allow these >>>>> operations for services/hosts. >>>> >>>> I still do not get the logic about an early switch. Now, if switch is >>>> turned >>>> on, FreeIPA server breaks on several places. I would really rather >>>> expect the >>>> switch to be introduced when the server itself supports it. Then, >>>> admin would >>>> enable it when the clients are sufficiently updated and can use the >>>> new method. >>>> >>>> Why would admin want to enable the switch early if it breaks FreeIPA >>>> some of >>>> the FreeIPA servers? Permission can be upgraded in newer version and get >>>> replicated, that's fine. But ipa-sam would be still broken on this old >>>> FreeIPA >>>> server. >>> Old server is not a problem here: ipa-sam only talks to the >>> localhost-based server as we always use ldapi protocol. So if server is >>> running old-behavior FreeIPA, ipa-sam on the same server will work >>> against it. >>> >>> What I don't want to have is a situation where setkeytab is disabled and >>> a new server obeys it but ipa-sam on this new server is not updated and >>> will expectedly fail. We are not that forced to do the change right now >>> in 4.3, given that the default will still be to keep setkeytab, thus we >>> can wait with this patch until ipa-sam is fixed and then push both >>> patches into the closest release, be it 4.3.x (x>=0) or whatever is the >>> next one. >>> >> >> +1, fixing ipa-sam would be then a release blocker for 4.3 which is not >> desired. >> >> Also what about adding support for "ipaProtectedoperation check" for >> user principals? >> >> I'm afraid that forbidding getting user principal might be regarded as a >> regression which might cause that admins won't set DisableSetKeytab. > > One thing at a time. > We have bugs open for all these things, but I see no reason not to add > an *optional* setting just because some things break when it is turned > on. > > Simo. > I would agree if the optional setting(DisableUserSetKeytab) was not enabled by default. -- Petr Vobornik From mbasti at redhat.com Thu Dec 3 08:40:54 2015 From: mbasti at redhat.com (Martin Basti) Date: Thu, 3 Dec 2015 09:40:54 +0100 Subject: [Freeipa-devel] [PATCH] 494 Update Contributors.txt In-Reply-To: <565FF689.5050303@redhat.com> References: <565ED508.2050308@redhat.com> <565ED798.2080109@redhat.com> <565ED814.1060201@redhat.com> <565FF689.5050303@redhat.com> Message-ID: <56600016.8030508@redhat.com> On 03.12.2015 09:00, Petr Spacek wrote: > On 2.12.2015 12:37, Martin Kosek wrote: >> On 12/02/2015 12:35 PM, Martin Basti wrote: >>> >>> On 02.12.2015 12:24, Martin Kosek wrote: >>>> +Thierry Bordaz >>>> +Thierry Bordaz >>> Do we want this there? >> I do not want it there, but I had to add it since Author field in git was wrong >> when the patch was pushed. >> >> So I blame the original author and the reviewer for making me add this mail >> alias :-) > +1 > > @Martin^2: Again, DNS names are not secret. Really. > > If they were we would have a bigger problem: Your e-mail headers contain DNS > names and IP addresses of internal mail servers and of course the machine you > used to send the e-mail. > > Hello, dhcp129-150.brq.redhat.com aka 10.34.129.150! :-) > I know, I'm not afraid of security in this case, it is just mess. From lslebodn at redhat.com Thu Dec 3 09:09:54 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Thu, 3 Dec 2015 10:09:54 +0100 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <565F34FD.5090204@redhat.com> References: <565F34FD.5090204@redhat.com> Message-ID: <20151203090954.GB5209@mail.corp.redhat.com> On (02/12/15 13:14), Rob Crittenden wrote: >Is it still mandatory that tests pass the unit tests before acceptance? Unit test could be executed as part of "%check" phase in spec files. I recently added C-base unit tests there. I was not bale to run "make tests" there because many tests failed. If they require real IPA server for execution ( or lite server) then they are not unit test but integration tests. One solution would be to skip them or to usw cwrap[1] + lite server. So it can be run also in mock/koji which has many restrictions. Also lint should be also part of "%check" phase and not part of ordinary build. BTW I could not see a lint[2] in fedora build at all. So I'm not sure if it is executed with upstream spec file. If someone wants to comply that it would take long time to build rpms then you might take a look how long glibc is built. http://koji.fedoraproject.org/koji/buildinfo?buildID=697271 (half an hour on x86_64) >I've seen a number of cases over the past couple of months where a >change goes through then shortly afterward a patch to fix the tests. >IMHO this should be caught in advance. > >Things slip through and goodness knows I've acked more than a few >patches without running the full suite. I just have a feeling it has >become more frequent lately. > >rob LS [1] https://cwrap.org/ [2] https://kojipkgs.fedoraproject.org//packages/freeipa/4.2.3/1.1.fc23/data/logs/x86_64/build.log From jcholast at redhat.com Thu Dec 3 09:55:49 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 3 Dec 2015 10:55:49 +0100 Subject: [Freeipa-devel] [PATCH] 0748 Handle encoding for ipautil.run In-Reply-To: <565EF7B9.6020507@redhat.com> References: <5638B8F9.6010703@redhat.com> <564D0FD9.4070505@redhat.com> <5652B5AC.2040508@redhat.com> <5652E174.50301@redhat.com> <56548E73.1090105@redhat.com> <565587AE.2070303@redhat.com> <5655CA08.8090701@redhat.com> <565C01FD.7050906@redhat.com> <565D83CC.8020802@redhat.com> <565EA4FA.3020307@redhat.com> <565EF7B9.6020507@redhat.com> Message-ID: <566011A5.4040200@redhat.com> On 2.12.2015 14:52, Petr Viktorin wrote: > On 12/02/2015 08:59 AM, Jan Cholasta wrote: >> On 1.12.2015 12:26, Petr Viktorin wrote: >>> On 11/30/2015 08:59 AM, Jan Cholasta wrote: >>>> On 25.11.2015 15:47, Petr Viktorin wrote: >>>>> On 11/25/2015 11:04 AM, Jan Cholasta wrote: >>>>>> On 24.11.2015 17:21, Petr Viktorin wrote: >>>>>>> On 11/23/2015 10:50 AM, Jan Cholasta wrote: >>>>>>>> On 23.11.2015 07:43, Jan Cholasta wrote: >>>>>>>>> On 19.11.2015 00:55, Petr Viktorin wrote: >>>>>>>>>> On 11/03/2015 02:39 PM, Petr Viktorin wrote: >>>>>>>>>>> Hello, >>>>>>>>>>> Python 3's strings are Unicode, so data coming to or leaving a >>>>>>>>>>> Python >>>>>>>>>>> program needs to be decoded/encoded if it's to be handled as a >>>>>>>>>>> string. >>>>>>>>>>> One of the boundaries where encoding is necessary is external >>>>>>>>>>> programs, >>>>>>>>>>> specifically, ipautil.run. >>>>>>>>>>> Unfortunately there's no one set of options that would work >>>>>>>>>>> for all >>>>>>>>>>> run() invocations, so I went through them all and specified the >>>>>>>>>>> stdin/stdout/stderr encoding where necessary. I've also >>>>>>>>>>> updated the >>>>>>>>>>> call >>>>>>>>>>> sites to make it clearer if the return values are used or not. >>>>>>>>>>> If an encoding is not set, run() will accept/return bytes. (This >>>>>>>>>>> is a >>>>>>>>>>> fail-safe setting, since it can't raise errors, and using bytes >>>>>>>>>>> where >>>>>>>>>>> strings are expected generally fails loudly in py3.) >>>>>>>>>>> >>>>>>>>>>> Note that the changes are not effective under Python 2. >>>>>>>>>> >>>>>>>>>> ping, >>>>>>>>>> Could someone look at this patch? >>>>>>>>> >>>>>>>>> Looking. >>>>>>>> >>>>>>>> 1) I think a better approach would be to use str for >>>>>>>> stdin/stdout/stderr >>>>>>>> by default in both Python 2 and 3, i.e. do nothing in Python 2 and >>>>>>>> encode/decode using locale.getpreferredencoding() in Python 3. >>>>>>>> This is >>>>>>>> consistent with sys.std{in,out,err} in both Python 2 and 3. Using >>>>>>>> bytes >>>>>>>> or different encoding should be opt-in. >>>>>>>> >>>>>>>> Note that different encoding should be used only if the LC_ALL or >>>>>>>> LANG >>>>>>>> variables are overriden in the command's environment. >>>>>>> >>>>>>> That would assume the output of *everything* run via ipautil.run >>>>>>> can be >>>>>>> decoded using the locale encoding. Any stray invalid byte would make >>>>>>> IPA >>>>>>> crash, even in cases where we don't care about the output. IPA calls >>>>>>> too >>>>>>> many weird tools to assume they all output text. >>>>>> >>>>>> Such a stray invalid byte may bubble through our code and cause havoc >>>>>> somewhere else, which is much harder to troubleshoot than a crash in >>>>>> ipautil.run(), where you can see that it was a misbehaving command >>>>>> that >>>>>> caused the crash and exactly what command it was. >>>>> >>>>> The invalid byte can't bubble through code, because if you don't >>>>> specify >>>>> an encoding you get bytes back. You'll get a crash when you try >>>>> using it >>>>> as a string. >>>> >>>> Invalid bytes *can* bubble through our code, into configuration files >>>> and other external places. The fact that it has not caused much grief so >>>> far suggests that it should be safe to use the locale encoding for most >>>> commands. >>> >>> Yes, "suggests" for "most". I'm still wary of using the locale encoding >>> for everything by default. >> >> OK, let's keep bytes as the default, but I still think the locale >> encoding should be the default when decoding output to text. >> >>> >>>> If we do subscribe to the statement that any command can output invalid >>>> bytes at any time, I don't see a point in handling encoding in >>>> ipautil.run() at all - just always return bytes and let the caller worry >>>> about encoding and handling related errors. >>> >>> Right, that's actually the basic idea. >>> >>> However, the encoding is two lines of boilerplate code ("if six.PY3" and >>> the actual encode). I've turned this into a convenience keyword argument >>> to minimize the repetition. >> >> If you always return bytes in both Python 2 and 3, you can always >> .decode() the output, so there won't be any "if six.PY3". Bearing that >> in mind, I don't see much difference between adding ", >> stdout_encoding='ascii'" to the run() call or "stdout = >> stdout.decode('ascii')" on the line below, except the latter is more >> explicit and lets you handle decoding errors yourself. >> >>>>> locale.getpreferredencoding() is not used consistently in all software, >>>>> especially if it's not written in Python. For instance, wget won't >>>>> magically re-encode the data it fetches for us. It's better to >>>>> explicitly specify the encoding every time. >>>> >>>> I would say it is used consistently in most software used by IPA. The >>>> wget example is obviously not relevant to this discussion, since it's >>>> returning external data. >>>> >>>> IMO setting the encoding to 'ascii' without also setting LC_ALL=C in the >>>> environment, as seen frequently in your patch, has even higher >>>> probability of breaking something than using the locale encoding. >>> >>> The patch should use 'ascii' with programs that don't output localized >>> messages: things like base64-encoded certs or the output of `ip`. >>> If you see a particular call where that's not the case, it's a bug in >>> the patch, please point it out. >> >> IMO it's better to be safe than sorry, so I would like to see either >> LC_ALL=C set with every use of 'ascii' encoding, or use the locale >> encoding instead of 'ascii'. >> >>> >>>>>> If we don't care about output somewhere, we should not capture it >>>>>> there >>>>>> at all. >>>>> >>>>> Then people need to remember to put "capture_output=True" everywhere >>>>> (except that also disables logging of the output, so we'd need an >>>>> additional option). >>>> >>>> capture_output=True is the default and capture_output=False should be >>>> set where output is not needed. >>> >>> "capture_output=True" being the default would mean that it's still >>> possible to leave "capture_output=False" out, but ignore the output. It >>> would make the wrong usage easier to type than the right one, which is >>> something to avoid. >> >> I'm not sure I understand what are you trying to say here. >> >> My point is that capture_output=True is currently the default (see for >> yourself) and it is indeed possible to leave capture_output=False but >> ignore the output. I believe this is wrong and that you should always >> have to explicitly request the output to be captured. > > OK, looks like I will need to refactor ipautil.run even for Python 2, then. If this is just about changing the default and fixing all run() invocations, I can provide the patch myself, if that would work better for you. > I seem to have have problems coming up with a solution that would match > your expectations, so let me write the semantics as I understand you'd > like hem: > > Would you be happy with the following signature? > > def run(args, stdin=None, raiseonerr=True, > nolog=(), env=None, > capture_output=False, skip_output=False, > cwd=None, runas=None, timeout=None, suplementary_groups=[], > capture_stderr=False, > encode_stdout=True, > encode_stderr=True, > ): > > Output and error would be always logged, except if skip_output is set. > But they would be only returned if capture_output/capture_stderr were set. I would personally also rename capture_output to capture_stdout. If we change the default, we have to update its every use, so we might as well rename it. (Again, I can provide the patch.) > If encode_* is False, bytes are returned, by default > locale.getpreferredencoding() is used. (If the caller wants any other > encoding, it can do that by itself.) I thought bytes should be the default after all? See my comment in the previous mail. > stdin it encoded using locale.getpreferredencoding() if it's unicode. If we do encode/decode in run(), I think there should be a way to override the default encoding. My idea was to either accept/return only bytes and let the caller handle encoding themselves, or to handle encoding in run(), but be able to override the defaults. Given we handle encoding in run(), I would imagine it like this: def run(args, stdin=None, raiseonerr=True, nolog=(), env=None, capture_stdout=False, skip_output=False, cwd=None, runas=None, timeout=None, suplementary_groups=[], capture_stderr=False, encode_stdout=False, encode_stderr=False, encoding=None): i.e. nothing is captured by default, when stdout or stderr are captured they are returned as bytes by default, when stdout or stderr are returned as text they are decoded using the locale encoding by default, or the encoding can be explicitly specified. Do you think this is feasible? > > Also, in this case I'd like the function would return a namedtuple, so > we don't have to set unused variables all the time. But I can leave that > as is if you disagree. Definitely, go for it. > > Would that be OK? > -- Jan Cholasta From jcholast at redhat.com Thu Dec 3 10:04:49 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 3 Dec 2015 11:04:49 +0100 Subject: [Freeipa-devel] [PATCH 25] Improve error logging for Dogtag subsystem installation In-Reply-To: <565EE7AD.7010502@redhat.com> References: <565DD610.5080608@redhat.com> <565DDC1E.5000008@redhat.com> <565E9FAF.5030805@redhat.com> <565EDB5F.4060001@redhat.com> <565EDC0A.5020408@redhat.com> <565EE2A9.9000906@redhat.com> <565EE7AD.7010502@redhat.com> Message-ID: <566013C1.3070703@redhat.com> On 2.12.2015 13:44, Petr Spacek wrote: > On 2.12.2015 13:23, Jan Cholasta wrote: >> On 2.12.2015 12:54, Petr Spacek wrote: >>> On 2.12.2015 12:51, Christian Heimes wrote: >>>> On 2015-12-02 08:37, Petr Spacek wrote: >>>>> On 1.12.2015 18:42, Christian Heimes wrote: >>>>>> From 33be1f56a64e53d261a1058c4606a7e48c0aac52 Mon Sep 17 00:00:00 2001 >>>>>> From: Christian Heimes >>>>>> Date: Tue, 1 Dec 2015 15:49:53 +0100 >>>>>> Subject: [PATCH 25] Improve error logging for Dogtag subsystem installation >>>>>> >>>>>> In the case of a failed installation or uninstallation of a Dogtag >>>>>> subsystem, the error output of pkispawn / pkidestroyed are now shown to >>>>>> the user. It makes it more obvious what went wrong and makes it easier >>>>>> to debug a problem. >>>>>> >>>>>> The error handler also attempts to get the full name of the installation >>>>>> / uninstallation log file from stdout. pkispawn and pkidestroy print the >>>>>> absolute name as 'Log file: /path/to/file.log'. The user no longer has >>>>>> to guess the right log file. >>>>>> >>>>>> Example: >>>>>> [1/8]: configuring KRA instance >>>>>> Failed to configure KRA instance: Command ''/usr/sbin/pkispawn' '-s' >>>>>> 'KRA' '-f' '/tmp/tmp1UpbwF'' returned non-zero exit status 1 >>>>>> pkispawn : ERROR ....... PKI subsystem 'KRA' for instance >>>>>> 'pki-tomcat' already exists! >>>>>> See the installation logs and the following files/directories for more >>>>>> information: >>>>>> /var/log/pki/pki-tomcat >>>>>> /var/log/pki/pki-kra-spawn.20151201151735.log >>>>>> [error] RuntimeError: KRA configuration failed. >>>>>> >>>>>> The patch also changes a couple of modules that were using >>>>>> the CalledProcessError exception object from subprocess instead of >>>>>> ipautil. >>>>> >>>>> I'm wondering if ipautil.run() can log stdout and stderr on log level ERROR >>>>> when return code is non-zero (and log on level DEBUG as usual when return >>>>> code >>>>> is zero). >>>>> >>>>> IMHO it would be nicer, universal, and does not require any changes in places >>>>> calling ipautil.run(). >>>> >>>> I think it's a bit confusing to print out stdout and stderr, because >>>> both streams are captured separately. The output is missing its >>>> chronological order. subprocess can capture stdout and stderr in the >>>> same stream, but then we can't distinguish between output and error >>>> output... >>> >>> I do not think it is a problem if these two are clearly marked as such: >>> standard output: %s (if non-empty) >>> stanrard error output: %s (if non-empty) >> >> We do not want to log with level ERROR by default when rc != 0, because some >> commands generate a *lot* of output. > > I do not agree, but whatever. Somebody needs to review the original > Christian's patch. We had a short discussion about this with Petr offline and we agreed that a reasonable compromise would be to log the last few lines of stderr with ERROR level when a command fails. This would mean adding custom __str__() to CalledProcessError, so that the stderr tail is logged when the CalledProcessError is not handled, and also logging it from ipautil.run() if raiseonerr == False. -- Jan Cholasta From abokovoy at redhat.com Thu Dec 3 10:21:37 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 3 Dec 2015 12:21:37 +0200 Subject: [Freeipa-devel] [PATCH 556-557] Add option to disable setkeytab extended operations In-Reply-To: <1449080213.9040.57.camel@redhat.com> References: <1448462853.29102.44.camel@redhat.com> <1448908709.3747.93.camel@redhat.com> <20151201081145.GJ9605@redhat.com> <1448977085.9040.1.camel@redhat.com> <565DA3DE.8000109@redhat.com> <1448978341.9040.5.camel@redhat.com> <565DB84C.2010401@redhat.com> <20151201152042.GQ9605@redhat.com> <565DC05B.1050804@redhat.com> <1449080213.9040.57.camel@redhat.com> Message-ID: <20151203102137.GE8374@redhat.com> On Wed, 02 Dec 2015, Simo Sorce wrote: >> >>>> We also have to fix the permission to change keytab, so that it uses >> >>>> the new >> >>>> style (https://fedorahosted.org/freeipa/ticket/5487). I would >> >>>> actually make >> >>>> this ticket and the ipa-sam ticket a blocker for this patch set. >> >>>> >> >>>> Otherwise you are actually introducing a switch that breaks FreeIPA >> >>>> as some of >> >>>> it's core server functions still use the old method. >> >>> >> >>> The point of this patchset is to introduce the switch early so that >> >>> current server will support the off switch when newer servers down the >> >>> road are ready to flip it. The defaults are still to allow these >> >>> operations for services/hosts. >> >> >> >> I still do not get the logic about an early switch. Now, if switch is >> >> turned >> >> on, FreeIPA server breaks on several places. I would really rather >> >> expect the >> >> switch to be introduced when the server itself supports it. Then, >> >> admin would >> >> enable it when the clients are sufficiently updated and can use the >> >> new method. >> >> >> >> Why would admin want to enable the switch early if it breaks FreeIPA >> >> some of >> >> the FreeIPA servers? Permission can be upgraded in newer version and get >> >> replicated, that's fine. But ipa-sam would be still broken on this old >> >> FreeIPA >> >> server. >> > Old server is not a problem here: ipa-sam only talks to the >> > localhost-based server as we always use ldapi protocol. So if server is >> > running old-behavior FreeIPA, ipa-sam on the same server will work >> > against it. >> > >> > What I don't want to have is a situation where setkeytab is disabled and >> > a new server obeys it but ipa-sam on this new server is not updated and >> > will expectedly fail. We are not that forced to do the change right now >> > in 4.3, given that the default will still be to keep setkeytab, thus we >> > can wait with this patch until ipa-sam is fixed and then push both >> > patches into the closest release, be it 4.3.x (x>=0) or whatever is the >> > next one. >> > >> >> +1, fixing ipa-sam would be then a release blocker for 4.3 which is not >> desired. >> >> Also what about adding support for "ipaProtectedoperation check" for >> user principals? >> >> I'm afraid that forbidding getting user principal might be regarded as a >> regression which might cause that admins won't set DisableSetKeytab. > >One thing at a time. >We have bugs open for all these things, but I see no reason not to add >an *optional* setting just because some things break when it is turned >on. The problem is that current getkeytab extended operation has not enough information to be a replacement for ipa-sam's use of setkeytab, as I've found with your current patches. So adding an optional setting in the hope that it will not be used in real life is a bit naive, but if people activate it, whole trust setup will break. I still prefer to have the patchset first be completed and then merged. There is no problem in keeping it aside while functionality is being implemented, we can trivially rebase. -- / Alexander Bokovoy From tbabej at redhat.com Thu Dec 3 10:35:20 2015 From: tbabej at redhat.com (Tomas Babej) Date: Thu, 3 Dec 2015 11:35:20 +0100 Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <20151202162519.GA19962@mail.corp.redhat.com> References: <565C3C6F.4040901@redhat.com> <20151130163256.GA7759@mail.corp.redhat.com> <565DD881.60401@redhat.com> <565EAAAE.8020306@redhat.com> <565F0332.9000804@redhat.com> <20151202162519.GA19962@mail.corp.redhat.com> Message-ID: <56601AE8.2020100@redhat.com> On 12/02/2015 05:25 PM, Lukas Slebodnik wrote: > On (02/12/15 15:41), Tomas Babej wrote: >> >> >> On 12/02/2015 09:24 AM, Tomas Babej wrote: >>> >>> >>> On 12/01/2015 06:27 PM, Tomas Babej wrote: >>>> >>>> >>>> On 11/30/2015 05:32 PM, Lukas Slebodnik wrote: >>>>> On (30/11/15 13:09), Tomas Babej wrote: >>>>>> Hi, >>>>>> >>>>>> IPA sudo tests worked under the assumption that the clients that >>>>>> are executing the sudo commands have their IPs assigned within >>>>>> 255.255.255.0 hostmask. >>>>>> >>>>>> Removes this (invalid) assumption and adds a dynamic detection of >>>>>> the hostmask of the IPA client. >>>>>> >>>>>> https://fedorahosted.org/freeipa/ticket/5501 >>>>> >>>>> >From e6f1846f0d7d17303e5b30b1643651ba739b2b6c Mon Sep 17 00:00:00 2001 >>>>>> From: Tomas Babej >>>>>> Date: Mon, 30 Nov 2015 12:53:39 +0100 >>>>>> Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on >>>>>> hostmask >>>>>> >>>>>> IPA sudo tests worked under the assumption that the clients that >>>>>> are executing the sudo commands have their IPs assigned within >>>>>> 255.255.255.0 hostmask. >>>>>> >>>>>> Removes this (invalid) assumption and adds a dynamic detection of >>>>>> the hostmask of the IPA client. >>>>>> >>>> >>>> Thanks, updated patch attached. >>>> >>>> Tomas >>>> >>> >>> Actually, a small improvement is necessary. >>> >>> Updated patch attached. >>> >>> Tomas >>> >>> >>> >> >> Thanks to Lukas, we found another problem with the test. >> >> Updated patch attached. >> > Thank you for 4th revision of patch > but there is still one issue. > > =================================== FAILURES =================================== > _________ TestSudo.test_sudo_rule_restricted_to_one_hostmask_negative __________ > > self = > > def test_sudo_rule_restricted_to_one_hostmask_negative(self): > result1 = self.list_sudo_commands("testuser1") >> assert result1.returncode != 0 > E assert 0 != 0 > E + where 0 = .returncode > > test_integration/test_sudo.py:323: AssertionError > ==================== 1 failed, 74 passed in 807.35 seconds ===================== > > LS > Here the test affected the negative test setup, which is fixed in the latest revision. Thanks, Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0388-5-tests-Add-hostmask-detection-for-sudo-rules-validati.patch Type: text/x-patch Size: 4053 bytes Desc: not available URL: From tbabej at redhat.com Thu Dec 3 11:06:11 2015 From: tbabej at redhat.com (Tomas Babej) Date: Thu, 3 Dec 2015 12:06:11 +0100 Subject: [Freeipa-devel] [PATCH 0390] man: Update the ipa-replica-install manpage with promotion Message-ID: <56602223.6020906@redhat.com> Hi, this patch updates the man page for the ipa-replica-install given the latest changes (including the Jan's OTP patch). Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0390-man-Update-the-ipa-replica-install-manpage-with-prom.patch Type: text/x-patch Size: 5483 bytes Desc: not available URL: From jcholast at redhat.com Thu Dec 3 11:41:30 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 3 Dec 2015 12:41:30 +0100 Subject: [Freeipa-devel] [PATCH] 939 topologysuffix: change iparepltopoconfroot API properties In-Reply-To: <565F14D9.9000207@redhat.com> References: <565F14D9.9000207@redhat.com> Message-ID: <56602A6A.2050509@redhat.com> On 2.12.2015 16:57, Petr Vobornik wrote: > Change CLI option, label and type to reflect that it is a only a DN > of the suffix. Works for me, ACK. Pushed to master: 581f5432bff7df909c1d7d7b8a55c5c81282afc0 -- Jan Cholasta From pvoborni at redhat.com Thu Dec 3 11:54:01 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 3 Dec 2015 12:54:01 +0100 Subject: [Freeipa-devel] [PATCH 0390] man: Update the ipa-replica-install manpage with promotion In-Reply-To: <56602223.6020906@redhat.com> References: <56602223.6020906@redhat.com> Message-ID: <56602D59.2050207@redhat.com> On 12/03/2015 12:06 PM, Tomas Babej wrote: > Hi, > > this patch updates the man page for the ipa-replica-install given the > latest changes (including the Jan's OTP patch). > > Tomas > Questions/suggestions: 1. "you cannot provide an replica file" It's true, but wouldn't it be better to say that "you don't need to". Or communicate that it is a good thing. Cannot sounds too negative to me. 2. Why options: --password, --admin-password are in 'BASIC OPTIONS' section and not also in new 'ENROLLMENT OPTIONS' section together with --principal, --keytab, etc..? Maybe 'ENROLLMENT OPTIONS' should be before "BASIC OPTIONS" and "BASIC OPTIONS" renamed. General ideas, not related to this patch: """ If the installation fails you may need to run ipa-server-install --uninstall and ipa-client-install before running ipa-replica-install again. The installation will fail if the host you are installing the replica on exists as a host in IPA or an existing replication agreement exists (for example, from a previously failed installation) """ I believe validators check this situation, right? So do we need to write it in the man page. -- Petr Vobornik From lslebodn at redhat.com Thu Dec 3 12:09:27 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Thu, 3 Dec 2015 13:09:27 +0100 Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <56601AE8.2020100@redhat.com> References: <565C3C6F.4040901@redhat.com> <20151130163256.GA7759@mail.corp.redhat.com> <565DD881.60401@redhat.com> <565EAAAE.8020306@redhat.com> <565F0332.9000804@redhat.com> <20151202162519.GA19962@mail.corp.redhat.com> <56601AE8.2020100@redhat.com> Message-ID: <20151203120927.GC5209@mail.corp.redhat.com> On (03/12/15 11:35), Tomas Babej wrote: > > >On 12/02/2015 05:25 PM, Lukas Slebodnik wrote: >> On (02/12/15 15:41), Tomas Babej wrote: >>> >>> >>> On 12/02/2015 09:24 AM, Tomas Babej wrote: >>>> >>>> >>>> On 12/01/2015 06:27 PM, Tomas Babej wrote: >>>>> >>>>> >>>>> On 11/30/2015 05:32 PM, Lukas Slebodnik wrote: >>>>>> On (30/11/15 13:09), Tomas Babej wrote: >>>>>>> Hi, >>>>>>> >>>>>>> IPA sudo tests worked under the assumption that the clients that >>>>>>> are executing the sudo commands have their IPs assigned within >>>>>>> 255.255.255.0 hostmask. >>>>>>> >>>>>>> Removes this (invalid) assumption and adds a dynamic detection of >>>>>>> the hostmask of the IPA client. >>>>>>> >>>>>>> https://fedorahosted.org/freeipa/ticket/5501 >>>>>> >>>>>> >From e6f1846f0d7d17303e5b30b1643651ba739b2b6c Mon Sep 17 00:00:00 2001 >>>>>>> From: Tomas Babej >>>>>>> Date: Mon, 30 Nov 2015 12:53:39 +0100 >>>>>>> Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on >>>>>>> hostmask >>>>>>> >>>>>>> IPA sudo tests worked under the assumption that the clients that >>>>>>> are executing the sudo commands have their IPs assigned within >>>>>>> 255.255.255.0 hostmask. >>>>>>> >>>>>>> Removes this (invalid) assumption and adds a dynamic detection of >>>>>>> the hostmask of the IPA client. >>>>>>> >>>>> >>>>> Thanks, updated patch attached. >>>>> >>>>> Tomas >>>>> >>>> >>>> Actually, a small improvement is necessary. >>>> >>>> Updated patch attached. >>>> >>>> Tomas >>>> >>>> >>>> >>> >>> Thanks to Lukas, we found another problem with the test. >>> >>> Updated patch attached. >>> >> Thank you for 4th revision of patch >> but there is still one issue. >> >> =================================== FAILURES =================================== >> _________ TestSudo.test_sudo_rule_restricted_to_one_hostmask_negative __________ >> >> self = >> >> def test_sudo_rule_restricted_to_one_hostmask_negative(self): >> result1 = self.list_sudo_commands("testuser1") >>> assert result1.returncode != 0 >> E assert 0 != 0 >> E + where 0 = .returncode >> >> test_integration/test_sudo.py:323: AssertionError >> ==================== 1 failed, 74 passed in 807.35 seconds ===================== >> >> LS >> > >Here the test affected the negative test setup, which is fixed in the >latest revision. > >Thanks, >Tomas >From 6242dfda205f6b2749627023384878448fd9e60c Mon Sep 17 00:00:00 2001 >From: Tomas Babej >Date: Wed, 2 Dec 2015 15:25:49 +0100 >Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on > hostmask > >IPA sudo tests worked under the assumption that the clients >that are executing the sudo commands have their IPs assigned >within 255.255.255.0 hostmask. > >Removes this (invalid) assumption and adds a >dynamic detection of the hostmask of the IPA client. > >https://fedorahosted.org/freeipa/ticket/5501 The sudo test passed with this patch. Thank you very much. Functional ACK. LS From cheimes at redhat.com Thu Dec 3 12:50:11 2015 From: cheimes at redhat.com (Christian Heimes) Date: Thu, 3 Dec 2015 13:50:11 +0100 Subject: [Freeipa-devel] [PATCH 25] Improve error logging for Dogtag subsystem installation In-Reply-To: <566013C1.3070703@redhat.com> References: <565DD610.5080608@redhat.com> <565DDC1E.5000008@redhat.com> <565E9FAF.5030805@redhat.com> <565EDB5F.4060001@redhat.com> <565EDC0A.5020408@redhat.com> <565EE2A9.9000906@redhat.com> <565EE7AD.7010502@redhat.com> <566013C1.3070703@redhat.com> Message-ID: <56603A83.30902@redhat.com> On 2015-12-03 11:04, Jan Cholasta wrote: > On 2.12.2015 13:44, Petr Spacek wrote: >> On 2.12.2015 13:23, Jan Cholasta wrote: >>> On 2.12.2015 12:54, Petr Spacek wrote: >>>> On 2.12.2015 12:51, Christian Heimes wrote: >>>>> On 2015-12-02 08:37, Petr Spacek wrote: >>>>>> On 1.12.2015 18:42, Christian Heimes wrote: >>>>>>> From 33be1f56a64e53d261a1058c4606a7e48c0aac52 Mon Sep 17 >>>>>>> 00:00:00 2001 >>>>>>> From: Christian Heimes >>>>>>> Date: Tue, 1 Dec 2015 15:49:53 +0100 >>>>>>> Subject: [PATCH 25] Improve error logging for Dogtag subsystem >>>>>>> installation >>>>>>> >>>>>>> In the case of a failed installation or uninstallation of a Dogtag >>>>>>> subsystem, the error output of pkispawn / pkidestroyed are now >>>>>>> shown to >>>>>>> the user. It makes it more obvious what went wrong and makes it >>>>>>> easier >>>>>>> to debug a problem. >>>>>>> >>>>>>> The error handler also attempts to get the full name of the >>>>>>> installation >>>>>>> / uninstallation log file from stdout. pkispawn and pkidestroy >>>>>>> print the >>>>>>> absolute name as 'Log file: /path/to/file.log'. The user no >>>>>>> longer has >>>>>>> to guess the right log file. >>>>>>> >>>>>>> Example: >>>>>>> [1/8]: configuring KRA instance >>>>>>> Failed to configure KRA instance: Command ''/usr/sbin/pkispawn' '-s' >>>>>>> 'KRA' '-f' '/tmp/tmp1UpbwF'' returned non-zero exit status 1 >>>>>>> pkispawn : ERROR ....... PKI subsystem 'KRA' for instance >>>>>>> 'pki-tomcat' already exists! >>>>>>> See the installation logs and the following files/directories for >>>>>>> more >>>>>>> information: >>>>>>> /var/log/pki/pki-tomcat >>>>>>> /var/log/pki/pki-kra-spawn.20151201151735.log >>>>>>> [error] RuntimeError: KRA configuration failed. >>>>>>> >>>>>>> The patch also changes a couple of modules that were using >>>>>>> the CalledProcessError exception object from subprocess instead of >>>>>>> ipautil. >>>>>> >>>>>> I'm wondering if ipautil.run() can log stdout and stderr on log >>>>>> level ERROR >>>>>> when return code is non-zero (and log on level DEBUG as usual when >>>>>> return >>>>>> code >>>>>> is zero). >>>>>> >>>>>> IMHO it would be nicer, universal, and does not require any >>>>>> changes in places >>>>>> calling ipautil.run(). >>>>> >>>>> I think it's a bit confusing to print out stdout and stderr, because >>>>> both streams are captured separately. The output is missing its >>>>> chronological order. subprocess can capture stdout and stderr in the >>>>> same stream, but then we can't distinguish between output and error >>>>> output... >>>> >>>> I do not think it is a problem if these two are clearly marked as such: >>>> standard output: %s (if non-empty) >>>> stanrard error output: %s (if non-empty) >>> >>> We do not want to log with level ERROR by default when rc != 0, >>> because some >>> commands generate a *lot* of output. >> >> I do not agree, but whatever. Somebody needs to review the original >> Christian's patch. > > We had a short discussion about this with Petr offline and we agreed > that a reasonable compromise would be to log the last few lines of > stderr with ERROR level when a command fails. > > This would mean adding custom __str__() to CalledProcessError, so that > the stderr tail is logged when the CalledProcessError is not handled, > and also logging it from ipautil.run() if raiseonerr == False. Yes, that sounds like a reasonable idea. In the default case (raiseonerr == True) ipautil.run() returns a custom CalledProcessError exception that prints the command and the last two or three non-empty lines from stderr. Callers can either log the exception directly or format the out as they see fit. With raiseonerr == False and exit code != 0 the same information is logged with log level ERROR. I can just create the exception object and log its string representation without raising the exception. Christian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From ofayans at redhat.com Thu Dec 3 12:50:31 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Thu, 3 Dec 2015 13:50:31 +0100 Subject: [Freeipa-devel] one-direction segments in ipaca suffix do not merge Message-ID: <56603A97.80204@redhat.com> Hi all, Should not these two one-directional segments in ipaca suffix be merged automatically? $ ipa topologysegment-find ipaca ------------------ 2 segments matched ------------------ Segment name: master.justfor.test-to-replica1.justfor.test Left node: master.justfor.test Right node: replica1.justfor.test Connectivity: left-right Segment name: replica1.justfor.test-to-master.justfor.test Left node: replica1.justfor.test Right node: master.justfor.test Connectivity: left-right ---------------------------- Number of entries returned 2 ---------------------------- -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From lkrispen at redhat.com Thu Dec 3 13:00:21 2015 From: lkrispen at redhat.com (Ludwig Krispenz) Date: Thu, 03 Dec 2015 14:00:21 +0100 Subject: [Freeipa-devel] one-direction segments in ipaca suffix do not merge In-Reply-To: <56603A97.80204@redhat.com> References: <56603A97.80204@redhat.com> Message-ID: <56603CE5.3030905@redhat.com> On 12/03/2015 01:50 PM, Oleg Fayans wrote: > Hi all, > > Should not these two one-directional segments in ipaca suffix be > merged automatically? yes they should, and normally do. What is your scenario when you get this, can you reproduce ? > > $ ipa topologysegment-find ipaca > ------------------ > 2 segments matched > ------------------ > Segment name: master.justfor.test-to-replica1.justfor.test > Left node: master.justfor.test > Right node: replica1.justfor.test > Connectivity: left-right > > Segment name: replica1.justfor.test-to-master.justfor.test > Left node: replica1.justfor.test > Right node: master.justfor.test > Connectivity: left-right > ---------------------------- > Number of entries returned 2 > ---------------------------- > > From pvoborni at redhat.com Thu Dec 3 13:31:58 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 3 Dec 2015 14:31:58 +0100 Subject: [Freeipa-devel] one-direction segments in ipaca suffix do not merge In-Reply-To: <56603CE5.3030905@redhat.com> References: <56603A97.80204@redhat.com> <56603CE5.3030905@redhat.com> Message-ID: <5660444E.5000604@redhat.com> On 12/03/2015 02:00 PM, Ludwig Krispenz wrote: > > On 12/03/2015 01:50 PM, Oleg Fayans wrote: >> Hi all, >> >> Should not these two one-directional segments in ipaca suffix be >> merged automatically? > yes they should, and normally do. What is your scenario when you get > this, can you reproduce ? I encountered it yesterday bu: * on A. `ipa-server-install --setup-dns` * on: B `ipa-replica-install --setup-ca` * on: C `ipa-replica-install --setup-ca` It created: A <-> B <-> C Agreements were not joined. Don't remember if in both suffixes. Btw, After I created A <-> C and removed B using `ipa-replica-manage del B`. I've encountered DS on A on 100%CPU for quite some time ~ 30s or more. I can provide logs/access for all 3 machines. >> >> $ ipa topologysegment-find ipaca >> ------------------ >> 2 segments matched >> ------------------ >> Segment name: master.justfor.test-to-replica1.justfor.test >> Left node: master.justfor.test >> Right node: replica1.justfor.test >> Connectivity: left-right >> >> Segment name: replica1.justfor.test-to-master.justfor.test >> Left node: replica1.justfor.test >> Right node: master.justfor.test >> Connectivity: left-right >> ---------------------------- >> Number of entries returned 2 >> ---------------------------- >> >> > -- Petr Vobornik From mbasti at redhat.com Thu Dec 3 14:11:32 2015 From: mbasti at redhat.com (Martin Basti) Date: Thu, 3 Dec 2015 15:11:32 +0100 Subject: [Freeipa-devel] [PATCHES 509-514] replica promotion: use host credentials when setting up replication In-Reply-To: <565D823E.5090600@redhat.com> References: <564DE017.4020408@redhat.com> <1447951434.5132.27.camel@redhat.com> <564F4105.1040204@redhat.com> <1448038723.5132.59.camel@redhat.com> <5652C63E.5090206@redhat.com> <1448289289.7892.13.camel@redhat.com> <565324C4.1090007@redhat.com> <1448290024.7892.15.camel@redhat.com> <565D823E.5090600@redhat.com> Message-ID: <56604D94.7000705@redhat.com> On 01.12.2015 12:19, Jan Cholasta wrote: > On 23.11.2015 15:47, Simo Sorce wrote: >> On Mon, 2015-11-23 at 15:37 +0100, Jan Cholasta wrote: >>> >>> Ad alternative is to add the host to ipaservers before the checks are >>> done and remove it again if any of them fail. >> >> Too error prone, I am ok with the current way in your patches >> until/unless I can think of a fail safe way. :-) > > Updated patches attached. Note that 520 should be applied between 509 > and 510. > > > Functional ACK -------------- next part -------------- An HTML attachment was scrubbed... URL: From lkrispen at redhat.com Thu Dec 3 14:18:27 2015 From: lkrispen at redhat.com (Ludwig Krispenz) Date: Thu, 03 Dec 2015 15:18:27 +0100 Subject: [Freeipa-devel] one-direction segments in ipaca suffix do not merge In-Reply-To: <5660444E.5000604@redhat.com> References: <56603A97.80204@redhat.com> <56603CE5.3030905@redhat.com> <5660444E.5000604@redhat.com> Message-ID: <56604F33.8070901@redhat.com> On 12/03/2015 02:31 PM, Petr Vobornik wrote: > On 12/03/2015 02:00 PM, Ludwig Krispenz wrote: >> >> On 12/03/2015 01:50 PM, Oleg Fayans wrote: >>> Hi all, >>> >>> Should not these two one-directional segments in ipaca suffix be >>> merged automatically? >> yes they should, and normally do. What is your scenario when you get >> this, can you reproduce ? > > I encountered it yesterday bu: > * on A. `ipa-server-install --setup-dns` > * on: B `ipa-replica-install --setup-ca` > * on: C `ipa-replica-install --setup-ca` > > It created: A <-> B <-> C > > Agreements were not joined. Don't remember if in both suffixes. > > Btw, > After I created A <-> C and removed B using `ipa-replica-manage del > B`. I've encountered DS on A on 100%CPU for quite some time ~ 30s or > more. please take a pstack next time it occurs > > I can provide logs/access for all 3 machines. yes, I will have a look > >>> >>> $ ipa topologysegment-find ipaca >>> ------------------ >>> 2 segments matched >>> ------------------ >>> Segment name: master.justfor.test-to-replica1.justfor.test >>> Left node: master.justfor.test >>> Right node: replica1.justfor.test >>> Connectivity: left-right >>> >>> Segment name: replica1.justfor.test-to-master.justfor.test >>> Left node: replica1.justfor.test >>> Right node: master.justfor.test >>> Connectivity: left-right >>> ---------------------------- >>> Number of entries returned 2 >>> ---------------------------- >>> >>> >> > > From ofayans at redhat.com Thu Dec 3 14:32:46 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Thu, 3 Dec 2015 15:32:46 +0100 Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <56601AE8.2020100@redhat.com> References: <565C3C6F.4040901@redhat.com> <20151130163256.GA7759@mail.corp.redhat.com> <565DD881.60401@redhat.com> <565EAAAE.8020306@redhat.com> <565F0332.9000804@redhat.com> <20151202162519.GA19962@mail.corp.redhat.com> <56601AE8.2020100@redhat.com> Message-ID: <5660528E.40904@redhat.com> Hi Tomas, I would prefer get_client_ip_with_hostmask function to go to ipatests/test_integration/tasks.py, and to be more generic: we probably will need to run it against an arbitrary host. The rest looks fine On 12/03/2015 11:35 AM, Tomas Babej wrote: > > > On 12/02/2015 05:25 PM, Lukas Slebodnik wrote: >> On (02/12/15 15:41), Tomas Babej wrote: >>> >>> >>> On 12/02/2015 09:24 AM, Tomas Babej wrote: >>>> >>>> >>>> On 12/01/2015 06:27 PM, Tomas Babej wrote: >>>>> >>>>> >>>>> On 11/30/2015 05:32 PM, Lukas Slebodnik wrote: >>>>>> On (30/11/15 13:09), Tomas Babej wrote: >>>>>>> Hi, >>>>>>> >>>>>>> IPA sudo tests worked under the assumption that the clients that >>>>>>> are executing the sudo commands have their IPs assigned within >>>>>>> 255.255.255.0 hostmask. >>>>>>> >>>>>>> Removes this (invalid) assumption and adds a dynamic detection of >>>>>>> the hostmask of the IPA client. >>>>>>> >>>>>>> https://fedorahosted.org/freeipa/ticket/5501 >>>>>> >>>>>> >From e6f1846f0d7d17303e5b30b1643651ba739b2b6c Mon Sep 17 00:00:00 2001 >>>>>>> From: Tomas Babej >>>>>>> Date: Mon, 30 Nov 2015 12:53:39 +0100 >>>>>>> Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on >>>>>>> hostmask >>>>>>> >>>>>>> IPA sudo tests worked under the assumption that the clients that >>>>>>> are executing the sudo commands have their IPs assigned within >>>>>>> 255.255.255.0 hostmask. >>>>>>> >>>>>>> Removes this (invalid) assumption and adds a dynamic detection of >>>>>>> the hostmask of the IPA client. >>>>>>> >>>>> >>>>> Thanks, updated patch attached. >>>>> >>>>> Tomas >>>>> >>>> >>>> Actually, a small improvement is necessary. >>>> >>>> Updated patch attached. >>>> >>>> Tomas >>>> >>>> >>>> >>> >>> Thanks to Lukas, we found another problem with the test. >>> >>> Updated patch attached. >>> >> Thank you for 4th revision of patch >> but there is still one issue. >> >> =================================== FAILURES =================================== >> _________ TestSudo.test_sudo_rule_restricted_to_one_hostmask_negative __________ >> >> self = >> >> def test_sudo_rule_restricted_to_one_hostmask_negative(self): >> result1 = self.list_sudo_commands("testuser1") >>> assert result1.returncode != 0 >> E assert 0 != 0 >> E + where 0 = .returncode >> >> test_integration/test_sudo.py:323: AssertionError >> ==================== 1 failed, 74 passed in 807.35 seconds ===================== >> >> LS >> > > Here the test affected the negative test setup, which is fixed in the > latest revision. > > Thanks, > Tomas > > > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From rcritten at redhat.com Thu Dec 3 14:34:36 2015 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 3 Dec 2015 09:34:36 -0500 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <565FFBB5.1030804@redhat.com> References: <565F34FD.5090204@redhat.com> <565FF89B.6090309@redhat.com> <565FFBB5.1030804@redhat.com> Message-ID: <566052FC.3000805@redhat.com> Martin Kosek wrote: > On 12/03/2015 09:08 AM, Petr Spacek wrote: >> On 2.12.2015 19:14, Rob Crittenden wrote: >>> Is it still mandatory that tests pass the unit tests before acceptance? >>> I've seen a number of cases over the past couple of months where a >>> change goes through then shortly afterward a patch to fix the tests. >>> IMHO this should be caught in advance. >>> >>> Things slip through and goodness knows I've acked more than a few >>> patches without running the full suite. I just have a feeling it has >>> become more frequent lately. >> >> When we are at it... An automated thingy which accepts URL to a Git repo, does >> all the test magic, and spits out test results without user interaction would >> be an awesome Christmas present! >> >> Bonus points if we can get Github integration so I can just push and have it >> tested automatically so I cannot forget to do that before sending the patch >> for review. > > +1. Having basic CI test suite run on top of a Pull Request would be awesome. > I'd be happy with just the ipatests being run manually with each review. And it's then reviewer that I'm focusing on here. A developer _should_ also run the tests but part of the reviewer's responsibility is to ensure the patch does what it says it does without breaking things. rob From pviktori at redhat.com Thu Dec 3 14:42:31 2015 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 3 Dec 2015 15:42:31 +0100 Subject: [Freeipa-devel] [PATCH] 0748 Handle encoding for ipautil.run In-Reply-To: <566011A5.4040200@redhat.com> References: <5638B8F9.6010703@redhat.com> <564D0FD9.4070505@redhat.com> <5652B5AC.2040508@redhat.com> <5652E174.50301@redhat.com> <56548E73.1090105@redhat.com> <565587AE.2070303@redhat.com> <5655CA08.8090701@redhat.com> <565C01FD.7050906@redhat.com> <565D83CC.8020802@redhat.com> <565EA4FA.3020307@redhat.com> <565EF7B9.6020507@redhat.com> <566011A5.4040200@redhat.com> Message-ID: <566054D7.9020008@redhat.com> On 12/03/2015 10:55 AM, Jan Cholasta wrote: [...] >>>>>>> If we don't care about output somewhere, we should not capture it >>>>>>> there >>>>>>> at all. >>>>>> >>>>>> Then people need to remember to put "capture_output=True" everywhere >>>>>> (except that also disables logging of the output, so we'd need an >>>>>> additional option). >>>>> >>>>> capture_output=True is the default and capture_output=False should be >>>>> set where output is not needed. >>>> >>>> "capture_output=True" being the default would mean that it's still >>>> possible to leave "capture_output=False" out, but ignore the output. It >>>> would make the wrong usage easier to type than the right one, which is >>>> something to avoid. >>> >>> I'm not sure I understand what are you trying to say here. >>> >>> My point is that capture_output=True is currently the default (see for >>> yourself) and it is indeed possible to leave capture_output=False but >>> ignore the output. I believe this is wrong and that you should always >>> have to explicitly request the output to be captured. >> >> OK, looks like I will need to refactor ipautil.run even for Python 2, >> then. > > If this is just about changing the default and fixing all run() > invocations, I can provide the patch myself, if that would work better > for you. > >> I seem to have have problems coming up with a solution that would match >> your expectations, so let me write the semantics as I understand you'd >> like hem: >> >> Would you be happy with the following signature? >> >> def run(args, stdin=None, raiseonerr=True, >> nolog=(), env=None, >> capture_output=False, skip_output=False, >> cwd=None, runas=None, timeout=None, suplementary_groups=[], >> capture_stderr=False, >> encode_stdout=True, >> encode_stderr=True, >> ): >> >> Output and error would be always logged, except if skip_output is set. >> But they would be only returned if capture_output/capture_stderr were >> set. > > I would personally also rename capture_output to capture_stdout. If we > change the default, we have to update its every use, so we might as well > rename it. (Again, I can provide the patch.) If we agree on the semantics, writing the patch isn't that big a deal. >> If encode_* is False, bytes are returned, by default >> locale.getpreferredencoding() is used. (If the caller wants any other >> encoding, it can do that by itself.) > > I thought bytes should be the default after all? See my comment in the > previous mail. If the default is no capturing, I'd rather go with text by default. With capturing by deafult, "run(args)" could raise encoding errors *if the output is then ignored*. If it's "run(args, capture_stdout=True)", I think it's fine to encode. >> stdin it encoded using locale.getpreferredencoding() if it's unicode. > > If we do encode/decode in run(), I think there should be a way to > override the default encoding. My idea was to either accept/return only > bytes and let the caller handle encoding themselves, or to handle > encoding in run(), but be able to override the defaults. > > Given we handle encoding in run(), I would imagine it like this: > > def run(args, stdin=None, raiseonerr=True, nolog=(), env=None, > capture_stdout=False, skip_output=False, cwd=None, > runas=None, timeout=None, suplementary_groups=[], > capture_stderr=False, encode_stdout=False, > encode_stderr=False, encoding=None): > > i.e. nothing is captured by default, when stdout or stderr are captured > they are returned as bytes by default, when stdout or stderr are > returned as text they are decoded using the locale encoding by default, > or the encoding can be explicitly specified. > > Do you think this is feasible? Feasible, yes. In the majority of cases where the output is needed, we want text encoded with locale.getpreferredencoding(), so I'd rather make that the default rather than bytes. Or we could make "encode_stdout" imply "capture_stdout", so you wouldn't have to always specify both. (It would be better named "encoded_stdout" in that case.) -- Petr Viktorin From pspacek at redhat.com Thu Dec 3 14:53:58 2015 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 3 Dec 2015 15:53:58 +0100 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <566052FC.3000805@redhat.com> References: <565F34FD.5090204@redhat.com> <565FF89B.6090309@redhat.com> <565FFBB5.1030804@redhat.com> <566052FC.3000805@redhat.com> Message-ID: <56605786.6030405@redhat.com> On 3.12.2015 15:34, Rob Crittenden wrote: > Martin Kosek wrote: >> On 12/03/2015 09:08 AM, Petr Spacek wrote: >>> On 2.12.2015 19:14, Rob Crittenden wrote: >>>> Is it still mandatory that tests pass the unit tests before acceptance? >>>> I've seen a number of cases over the past couple of months where a >>>> change goes through then shortly afterward a patch to fix the tests. >>>> IMHO this should be caught in advance. >>>> >>>> Things slip through and goodness knows I've acked more than a few >>>> patches without running the full suite. I just have a feeling it has >>>> become more frequent lately. >>> >>> When we are at it... An automated thingy which accepts URL to a Git repo, does >>> all the test magic, and spits out test results without user interaction would >>> be an awesome Christmas present! >>> >>> Bonus points if we can get Github integration so I can just push and have it >>> tested automatically so I cannot forget to do that before sending the patch >>> for review. >> >> +1. Having basic CI test suite run on top of a Pull Request would be awesome. >> > > I'd be happy with just the ipatests being run manually with each review. > > And it's then reviewer that I'm focusing on here. A developer _should_ > also run the tests but part of the reviewer's responsibility is to > ensure the patch does what it says it does without breaking things. Sure. I'm just saying that people are notoriously bad automation tool, so I would like to off-load this kind of checks to a tool which will not forget or be lazy :-) -- Petr^2 Spacek From rcritten at redhat.com Thu Dec 3 14:59:46 2015 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 3 Dec 2015 09:59:46 -0500 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <20151203090954.GB5209@mail.corp.redhat.com> References: <565F34FD.5090204@redhat.com> <20151203090954.GB5209@mail.corp.redhat.com> Message-ID: <566058E2.6040201@redhat.com> Lukas Slebodnik wrote: > On (02/12/15 13:14), Rob Crittenden wrote: >> Is it still mandatory that tests pass the unit tests before acceptance? > Unit test could be executed as part of "%check" phase in spec files. > I recently added C-base unit tests there. > > I was not bale to run "make tests" there because many tests failed. > If they require real IPA server for execution ( or lite server) > then they are not unit test but integration tests. > One solution would be to skip them or to usw cwrap[1] + lite server. > So it can be run also in mock/koji which has many restrictions. It would represent quite a lot of work but it may be a good idea to investigate. Ipsilon uses cwrap for its tests so some of the configuration can be gleaned from that. I would definitely be opposed to this as part of the freeipa.spec in the git tree. In Fedora it might help find problems when rawhide becomes Fedora.next so it would provide some value there. > > Also lint should be also part of "%check" phase and not part of > ordinary build. > BTW I could not see a lint[2] in fedora build at all. So I'm not sure > if it is executed with upstream spec file. It isn't there because the expectation is that lint already passes as part of the release process. I don't see the value on running lint on release tarballs. I also want to keep the focus on the reviewer's responsibility to ensure that patches do what they are supposed to and don't break things. rob > If someone wants to comply that it would take long time to build rpms > then you might take a look how long glibc is built. > http://koji.fedoraproject.org/koji/buildinfo?buildID=697271 > (half an hour on x86_64) > >> I've seen a number of cases over the past couple of months where a >> change goes through then shortly afterward a patch to fix the tests. >> IMHO this should be caught in advance. >> >> Things slip through and goodness knows I've acked more than a few >> patches without running the full suite. I just have a feeling it has >> become more frequent lately. >> >> rob > > LS > > [1] https://cwrap.org/ > [2] https://kojipkgs.fedoraproject.org//packages/freeipa/4.2.3/1.1.fc23/data/logs/x86_64/build.log > From lslebodn at redhat.com Thu Dec 3 15:00:58 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Thu, 3 Dec 2015 16:00:58 +0100 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <56605786.6030405@redhat.com> References: <565F34FD.5090204@redhat.com> <565FF89B.6090309@redhat.com> <565FFBB5.1030804@redhat.com> <566052FC.3000805@redhat.com> <56605786.6030405@redhat.com> Message-ID: <20151203150057.GA7994@mail.corp.redhat.com> On (03/12/15 15:53), Petr Spacek wrote: >On 3.12.2015 15:34, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On 12/03/2015 09:08 AM, Petr Spacek wrote: >>>> On 2.12.2015 19:14, Rob Crittenden wrote: >>>>> Is it still mandatory that tests pass the unit tests before acceptance? >>>>> I've seen a number of cases over the past couple of months where a >>>>> change goes through then shortly afterward a patch to fix the tests. >>>>> IMHO this should be caught in advance. >>>>> >>>>> Things slip through and goodness knows I've acked more than a few >>>>> patches without running the full suite. I just have a feeling it has >>>>> become more frequent lately. >>>> >>>> When we are at it... An automated thingy which accepts URL to a Git repo, does >>>> all the test magic, and spits out test results without user interaction would >>>> be an awesome Christmas present! >>>> >>>> Bonus points if we can get Github integration so I can just push and have it >>>> tested automatically so I cannot forget to do that before sending the patch >>>> for review. >>> >>> +1. Having basic CI test suite run on top of a Pull Request would be awesome. >>> >> >> I'd be happy with just the ipatests being run manually with each review. >> >> And it's then reviewer that I'm focusing on here. A developer _should_ >> also run the tests but part of the reviewer's responsibility is to >> ensure the patch does what it says it does without breaking things. > >Sure. I'm just saying that people are notoriously bad automation tool, so I >would like to off-load this kind of checks to a tool which will not forget or >be lazy :-) > Petr, patches are always welcomed. LS From mbasti at redhat.com Thu Dec 3 15:04:12 2015 From: mbasti at redhat.com (Martin Basti) Date: Thu, 3 Dec 2015 16:04:12 +0100 Subject: [Freeipa-devel] [PATCH 561] Catch up with upstream kerberos.ldif schema In-Reply-To: <1448408496.29102.31.camel@redhat.com> References: <1448408496.29102.31.camel@redhat.com> Message-ID: <566059EC.7080707@redhat.com> On 25.11.2015 00:41, Simo Sorce wrote: > Not much action here, but it will close a ticket (#2086) and get us on > par with upstream. > > Simo. > > > ACK Pushed to master: 5ed1b844dcb7d4a57a7067eef63b644df47bd740 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcritten at redhat.com Thu Dec 3 15:07:15 2015 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 3 Dec 2015 10:07:15 -0500 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <56605786.6030405@redhat.com> References: <565F34FD.5090204@redhat.com> <565FF89B.6090309@redhat.com> <565FFBB5.1030804@redhat.com> <566052FC.3000805@redhat.com> <56605786.6030405@redhat.com> Message-ID: <56605AA3.20800@redhat.com> Petr Spacek wrote: > On 3.12.2015 15:34, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On 12/03/2015 09:08 AM, Petr Spacek wrote: >>>> On 2.12.2015 19:14, Rob Crittenden wrote: >>>>> Is it still mandatory that tests pass the unit tests before acceptance? >>>>> I've seen a number of cases over the past couple of months where a >>>>> change goes through then shortly afterward a patch to fix the tests. >>>>> IMHO this should be caught in advance. >>>>> >>>>> Things slip through and goodness knows I've acked more than a few >>>>> patches without running the full suite. I just have a feeling it has >>>>> become more frequent lately. >>>> >>>> When we are at it... An automated thingy which accepts URL to a Git repo, does >>>> all the test magic, and spits out test results without user interaction would >>>> be an awesome Christmas present! >>>> >>>> Bonus points if we can get Github integration so I can just push and have it >>>> tested automatically so I cannot forget to do that before sending the patch >>>> for review. >>> >>> +1. Having basic CI test suite run on top of a Pull Request would be awesome. >>> >> >> I'd be happy with just the ipatests being run manually with each review. >> >> And it's then reviewer that I'm focusing on here. A developer _should_ >> also run the tests but part of the reviewer's responsibility is to >> ensure the patch does what it says it does without breaking things. > > Sure. I'm just saying that people are notoriously bad automation tool, so I > would like to off-load this kind of checks to a tool which will not forget or > be lazy :-) > Sure, but it's the _job_ of the reviewer to do these things. My previous statement is better read as: A developer _shall_ also run the tests. And even beyond that reviewers need to ensure that the patch works, does it properly in the context of IPA, doesn't break anything, works in SELinux enforcing mode, updates man pages, etc. In other words, when a patch breaks something, don't blame the developer, blame the reviewer. rob From pspacek at redhat.com Thu Dec 3 15:13:24 2015 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 3 Dec 2015 16:13:24 +0100 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <56605AA3.20800@redhat.com> References: <565F34FD.5090204@redhat.com> <565FF89B.6090309@redhat.com> <565FFBB5.1030804@redhat.com> <566052FC.3000805@redhat.com> <56605786.6030405@redhat.com> <56605AA3.20800@redhat.com> Message-ID: <56605C14.6030104@redhat.com> On 3.12.2015 16:07, Rob Crittenden wrote: > Petr Spacek wrote: >> On 3.12.2015 15:34, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> On 12/03/2015 09:08 AM, Petr Spacek wrote: >>>>> On 2.12.2015 19:14, Rob Crittenden wrote: >>>>>> Is it still mandatory that tests pass the unit tests before acceptance? >>>>>> I've seen a number of cases over the past couple of months where a >>>>>> change goes through then shortly afterward a patch to fix the tests. >>>>>> IMHO this should be caught in advance. >>>>>> >>>>>> Things slip through and goodness knows I've acked more than a few >>>>>> patches without running the full suite. I just have a feeling it has >>>>>> become more frequent lately. >>>>> >>>>> When we are at it... An automated thingy which accepts URL to a Git repo, does >>>>> all the test magic, and spits out test results without user interaction would >>>>> be an awesome Christmas present! >>>>> >>>>> Bonus points if we can get Github integration so I can just push and have it >>>>> tested automatically so I cannot forget to do that before sending the patch >>>>> for review. >>>> >>>> +1. Having basic CI test suite run on top of a Pull Request would be awesome. >>>> >>> >>> I'd be happy with just the ipatests being run manually with each review. >>> >>> And it's then reviewer that I'm focusing on here. A developer _should_ >>> also run the tests but part of the reviewer's responsibility is to >>> ensure the patch does what it says it does without breaking things. >> >> Sure. I'm just saying that people are notoriously bad automation tool, so I >> would like to off-load this kind of checks to a tool which will not forget or >> be lazy :-) >> > > Sure, but it's the _job_ of the reviewer to do these things. My previous > statement is better read as: A developer _shall_ also run the tests. > > And even beyond that reviewers need to ensure that the patch works, does > it properly in the context of IPA, doesn't break anything, works in > SELinux enforcing mode, updates man pages, etc. > > In other words, when a patch breaks something, don't blame the > developer, blame the reviewer. I would blame both ;-) Again, I'm not against proper reviews, just saying that it simply does not scale, so it needs automation. -- Petr^2 Spacek From mkosek at redhat.com Thu Dec 3 15:13:34 2015 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 3 Dec 2015 16:13:34 +0100 Subject: [Freeipa-devel] [PATCH] 495 Update Build instructions Message-ID: <56605C1E.7070300@redhat.com> Original dnf builddep command does not work, unless --spec option is added. -- Martin Kosek Associate Manager, Software Engineering - Identity Management Team Red Hat, Inc. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-495-update-build-instructions.patch Type: text/x-patch Size: 742 bytes Desc: not available URL: From pvoborni at redhat.com Thu Dec 3 15:23:38 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 3 Dec 2015 16:23:38 +0100 Subject: [Freeipa-devel] [PATCH] 495 Update Build instructions In-Reply-To: <56605C1E.7070300@redhat.com> References: <56605C1E.7070300@redhat.com> Message-ID: <56605E7A.8070100@redhat.com> On 12/03/2015 04:13 PM, Martin Kosek wrote: > Original dnf builddep command does not work, unless --spec option is > added. > ACK Pushed to master: 03c7d63c52615cefef260e169ec4dadb85d54842 -- Petr Vobornik From amarecek at redhat.com Thu Dec 3 15:26:08 2015 From: amarecek at redhat.com (=?utf-8?Q?Ale=C5=A1_Mare=C4=8Dek?=) Date: Thu, 3 Dec 2015 10:26:08 -0500 (EST) Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <5660528E.40904@redhat.com> References: <565C3C6F.4040901@redhat.com> <20151130163256.GA7759@mail.corp.redhat.com> <565DD881.60401@redhat.com> <565EAAAE.8020306@redhat.com> <565F0332.9000804@redhat.com> <20151202162519.GA19962@mail.corp.redhat.com> <56601AE8.2020100@redhat.com> <5660528E.40904@redhat.com> Message-ID: <149447113.23167965.1449156368210.JavaMail.zimbra@redhat.com> Hello, ACK for code NACK for the placing "get_client_ip_with_hostmask" function to test_sudo.py (this function should be in some more general file) ----- Original Message ----- > From: "Oleg Fayans" > To: freeipa-devel at redhat.com > Sent: Thursday, December 3, 2015 3:32:46 PM > Subject: Re: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating > > Hi Tomas, > > I would prefer get_client_ip_with_hostmask function to go to > ipatests/test_integration/tasks.py, and to be more generic: we probably > will need to run it against an arbitrary host. tasks.py is not the proper place as well > The rest looks fine > > On 12/03/2015 11:35 AM, Tomas Babej wrote: > > > > > > On 12/02/2015 05:25 PM, Lukas Slebodnik wrote: > >> On (02/12/15 15:41), Tomas Babej wrote: > >>> > >>> > >>> On 12/02/2015 09:24 AM, Tomas Babej wrote: > >>>> > >>>> > >>>> On 12/01/2015 06:27 PM, Tomas Babej wrote: > >>>>> > >>>>> > >>>>> On 11/30/2015 05:32 PM, Lukas Slebodnik wrote: > >>>>>> On (30/11/15 13:09), Tomas Babej wrote: > >>>>>>> Hi, > >>>>>>> > >>>>>>> IPA sudo tests worked under the assumption that the clients that > >>>>>>> are executing the sudo commands have their IPs assigned within > >>>>>>> 255.255.255.0 hostmask. > >>>>>>> > >>>>>>> Removes this (invalid) assumption and adds a dynamic detection of > >>>>>>> the hostmask of the IPA client. > >>>>>>> > >>>>>>> https://fedorahosted.org/freeipa/ticket/5501 > >>>>>> > >>>>>> >From e6f1846f0d7d17303e5b30b1643651ba739b2b6c Mon Sep 17 00:00:00 > >>>>>> >2001 > >>>>>>> From: Tomas Babej > >>>>>>> Date: Mon, 30 Nov 2015 12:53:39 +0100 > >>>>>>> Subject: [PATCH] tests: Add hostmask detection for sudo rules > >>>>>>> validating on > >>>>>>> hostmask > >>>>>>> > >>>>>>> IPA sudo tests worked under the assumption that the clients that > >>>>>>> are executing the sudo commands have their IPs assigned within > >>>>>>> 255.255.255.0 hostmask. > >>>>>>> > >>>>>>> Removes this (invalid) assumption and adds a dynamic detection of > >>>>>>> the hostmask of the IPA client. > >>>>>>> > >>>>> > >>>>> Thanks, updated patch attached. > >>>>> > >>>>> Tomas > >>>>> > >>>> > >>>> Actually, a small improvement is necessary. > >>>> > >>>> Updated patch attached. > >>>> > >>>> Tomas > >>>> > >>>> > >>>> > >>> > >>> Thanks to Lukas, we found another problem with the test. > >>> > >>> Updated patch attached. > >>> > >> Thank you for 4th revision of patch > >> but there is still one issue. > >> > >> =================================== FAILURES > >> =================================== > >> _________ TestSudo.test_sudo_rule_restricted_to_one_hostmask_negative > >> __________ > >> > >> self = >> 0x7ff695f56890> > >> > >> def test_sudo_rule_restricted_to_one_hostmask_negative(self): > >> result1 = self.list_sudo_commands("testuser1") > >>> assert result1.returncode != 0 > >> E assert 0 != 0 > >> E + where 0 = >> 0x7ff695f56bd0>.returncode > >> > >> test_integration/test_sudo.py:323: AssertionError > >> ==================== 1 failed, 74 passed in 807.35 seconds > >> ===================== > >> > >> LS > >> > > > > Here the test affected the negative test setup, which is fixed in the > > latest revision. > > > > Thanks, > > Tomas > > > > > > > > -- > Oleg Fayans > Quality Engineer > FreeIPA team > RedHat. > > -- > Manage your subscription for the Freeipa-devel mailing list: > https://www.redhat.com/mailman/listinfo/freeipa-devel > Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code > From tbabej at redhat.com Thu Dec 3 15:33:54 2015 From: tbabej at redhat.com (Tomas Babej) Date: Thu, 3 Dec 2015 16:33:54 +0100 Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <149447113.23167965.1449156368210.JavaMail.zimbra@redhat.com> References: <565C3C6F.4040901@redhat.com> <20151130163256.GA7759@mail.corp.redhat.com> <565DD881.60401@redhat.com> <565EAAAE.8020306@redhat.com> <565F0332.9000804@redhat.com> <20151202162519.GA19962@mail.corp.redhat.com> <56601AE8.2020100@redhat.com> <5660528E.40904@redhat.com> <149447113.23167965.1449156368210.JavaMail.zimbra@redhat.com> Message-ID: <566060E2.8030903@redhat.com> On 12/03/2015 04:26 PM, Ale? Mare?ek wrote: > Hello, > > ACK for code > NACK for the placing "get_client_ip_with_hostmask" function to test_sudo.py (this function should be in some more general file) > What place would you propose? The task.py is not a good place, as this is not really a task. Nevertheless, I'd rather have it moved when an use case outside test_sudo.py actually arises. Right now it would lead to unnecessary cluttering. Tomas From rcritten at redhat.com Thu Dec 3 15:38:19 2015 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 3 Dec 2015 10:38:19 -0500 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <56605C14.6030104@redhat.com> References: <565F34FD.5090204@redhat.com> <565FF89B.6090309@redhat.com> <565FFBB5.1030804@redhat.com> <566052FC.3000805@redhat.com> <56605786.6030405@redhat.com> <56605AA3.20800@redhat.com> <56605C14.6030104@redhat.com> Message-ID: <566061EB.90209@redhat.com> Petr Spacek wrote: > On 3.12.2015 16:07, Rob Crittenden wrote: >> Petr Spacek wrote: >>> On 3.12.2015 15:34, Rob Crittenden wrote: >>>> Martin Kosek wrote: >>>>> On 12/03/2015 09:08 AM, Petr Spacek wrote: >>>>>> On 2.12.2015 19:14, Rob Crittenden wrote: >>>>>>> Is it still mandatory that tests pass the unit tests before acceptance? >>>>>>> I've seen a number of cases over the past couple of months where a >>>>>>> change goes through then shortly afterward a patch to fix the tests. >>>>>>> IMHO this should be caught in advance. >>>>>>> >>>>>>> Things slip through and goodness knows I've acked more than a few >>>>>>> patches without running the full suite. I just have a feeling it has >>>>>>> become more frequent lately. >>>>>> >>>>>> When we are at it... An automated thingy which accepts URL to a Git repo, does >>>>>> all the test magic, and spits out test results without user interaction would >>>>>> be an awesome Christmas present! >>>>>> >>>>>> Bonus points if we can get Github integration so I can just push and have it >>>>>> tested automatically so I cannot forget to do that before sending the patch >>>>>> for review. >>>>> >>>>> +1. Having basic CI test suite run on top of a Pull Request would be awesome. >>>>> >>>> >>>> I'd be happy with just the ipatests being run manually with each review. >>>> >>>> And it's then reviewer that I'm focusing on here. A developer _should_ >>>> also run the tests but part of the reviewer's responsibility is to >>>> ensure the patch does what it says it does without breaking things. >>> >>> Sure. I'm just saying that people are notoriously bad automation tool, so I >>> would like to off-load this kind of checks to a tool which will not forget or >>> be lazy :-) >>> >> >> Sure, but it's the _job_ of the reviewer to do these things. My previous >> statement is better read as: A developer _shall_ also run the tests. >> >> And even beyond that reviewers need to ensure that the patch works, does >> it properly in the context of IPA, doesn't break anything, works in >> SELinux enforcing mode, updates man pages, etc. >> >> In other words, when a patch breaks something, don't blame the >> developer, blame the reviewer. > > I would blame both ;-) > > Again, I'm not against proper reviews, just saying that it simply does not > scale, so it needs automation. > I haven't worked on IPA for a while so won't comment on the scaling other than to say it's always been a busy project and reviews are always a trade-off. I don't want to devolve this into automation as a panacea. There are steps for review that aren't being followed either closely enough or at all. That's what I'm trying to address. There should not be follow-up patches to fix tests that are failing. There should not be follow-up patches to add missing manpage information. And that's on the reviewer. That's all I'm saying. rob From jhrozek at redhat.com Thu Dec 3 15:39:46 2015 From: jhrozek at redhat.com (Jakub Hrozek) Date: Thu, 3 Dec 2015 16:39:46 +0100 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <566058E2.6040201@redhat.com> References: <565F34FD.5090204@redhat.com> <20151203090954.GB5209@mail.corp.redhat.com> <566058E2.6040201@redhat.com> Message-ID: <20151203153946.GO2992@hendrix> On Thu, Dec 03, 2015 at 09:59:46AM -0500, Rob Crittenden wrote: > Lukas Slebodnik wrote: > > On (02/12/15 13:14), Rob Crittenden wrote: > >> Is it still mandatory that tests pass the unit tests before acceptance? > > Unit test could be executed as part of "%check" phase in spec files. > > I recently added C-base unit tests there. > > > > I was not bale to run "make tests" there because many tests failed. > > If they require real IPA server for execution ( or lite server) > > then they are not unit test but integration tests. > > One solution would be to skip them or to usw cwrap[1] + lite server. > > So it can be run also in mock/koji which has many restrictions. > > It would represent quite a lot of work but it may be a good idea to > investigate. Ipsilon uses cwrap for its tests so some of the > configuration can be gleaned from that. SSSD uses cwrap tests as well, Nikolai wrote tests that spin up an openldap server and there are even tests that test against a Samba DC on review..so IPA developers can also look into the SSSD tree. From pvoborni at redhat.com Thu Dec 3 15:58:09 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 3 Dec 2015 16:58:09 +0100 Subject: [Freeipa-devel] [PATCH] 940 Update ipa-(cs)replica-manage man pages Message-ID: <56606691.2090402@redhat.com> SSIA -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0940-Update-ipa-cs-replica-manage-man-pages.patch Type: text/x-patch Size: 4018 bytes Desc: not available URL: From pvoborni at redhat.com Thu Dec 3 15:58:38 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 3 Dec 2015 16:58:38 +0100 Subject: [Freeipa-devel] [PATCH] 941 Extend topology help Message-ID: <566066AE.2090702@redhat.com> `ipa help topology` is improved. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0941-Extend-topology-help.patch Type: text/x-patch Size: 2818 bytes Desc: not available URL: From mbabinsk at redhat.com Thu Dec 3 16:05:17 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Thu, 3 Dec 2015 17:05:17 +0100 Subject: [Freeipa-devel] [PATCH 0108] replica install: improvements in the handling of CA-related IPA config entries Message-ID: <5660683D.5000803@redhat.com> This patch fixes https://fedorahosted.org/freeipa/ticket/5506 and also reorganizes the way CA installer updates IPA default.conf. Maybe a simpler patch would suffice, but I had a need to improve things a bit. This one is for master branch only. IIRC the situation described in #5506 does not occur in domain level 0, however the root cause (incorrect forwarding of certmonger requests by CA-less replicas) manifests when enrolling a client against CA-less replica and requesting host certificate. Should I open a separate ticket for that? -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0108-replica-install-improvements-in-the-handling-of-CA-r.patch Type: text/x-patch Size: 5267 bytes Desc: not available URL: From pviktori at redhat.com Thu Dec 3 16:32:16 2015 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 3 Dec 2015 17:32:16 +0100 Subject: [Freeipa-devel] [PATCH] 0751 spec: Split out python-ipap11helper and, python-default_encoding_utf8 Message-ID: <56606E90.3000909@redhat.com> Hello, This specfile patch makes python-ipalib noarch, by splitting out the arch-dependent stuff. It depends on Honza Cholasta's patches 516 and 517. Part of https://fedorahosted.org/freeipa/ticket/3197 -- Petr Viktorin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0751-spec-Split-out-python-ipap11helper-and-python-defaul.patch Type: text/x-patch Size: 3493 bytes Desc: not available URL: From fskola at redhat.com Thu Dec 3 16:38:43 2015 From: fskola at redhat.com (Filip =?UTF-8?B?xaBrb2xh?=) Date: Thu, 3 Dec 2015 17:38:43 +0100 Subject: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin In-Reply-To: <56448E17.2010409@redhat.com> References: <20151106113201.5e6a7042@dhcp-24-123.brq.redhat.com> <564092F6.9000800@redhat.com> <20151109163512.3d3beb5f@redhat.com> <5641BE6D.4060608@redhat.com> <20151110121350.06295a68@dhcp-24-103.brq.redhat.com> <56448E17.2010409@redhat.com> Message-ID: <20151203173843.37179230@vor2.netbox.priv> Hi, sending corrected version. F. On Thu, 12 Nov 2015 14:03:19 +0100 Milan Kub?k wrote: > On 11/10/2015 12:13 PM, Filip ?kola wrote: > > Hi, > > > > fixed. > > > > F. > > > > On Tue, 10 Nov 2015 10:52:45 +0100 > > Milan Kub?k wrote: > > > >> On 11/09/2015 04:35 PM, Filip ?kola wrote: > >>> Another patch was applied in the meantime. > >>> > >>> Attaching an updated version. > >>> > >>> F. > >>> > >>> On Mon, 9 Nov 2015 13:35:02 +0100 > >>> Milan Kub?k wrote: > >>> > >>>> On 11/06/2015 11:32 AM, Filip ?kola wrote: > >>>> Hi, > >>>> the patch doesn't apply. > >>>> > >> Please fix this. > >> > >> ipatests/test_xmlrpc/test_user_plugin.py:1419: > >> [E0602(undefined-variable), > >> TestDeniedBindWithExpiredPrincipal.teardown_class] Undefined > >> variable 'user1') > >> > >> Also, use the version numbers for your changed patches. > >> > > > > > Thanks for the patch. Several issues: > > 1. Use dict.items instead of dict.iteritems, for python3 compatibility > > 2. What is the purpose of TestPrepare class? The 'purge' methods do > not call any ipa commands. > Tracker.make_fixture should be used to make the Tracked resources > clean themselves up when they're out of scope. > > 3. Why reference the resources by hardcoded name if they have a > fixture representation? > > 4. Rewrite {create,delete}_test_group to a fixture. You may want to > use different scope (or not). > > 5. In `def atest_rename_to_invalid_login(self, user):` - use > pytest.skipif decorator and provide a reason if you must, > do not obfuscate method name in order not to run it. > > -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-fskola-0001-3-Refactor-test_user_plugin.patch Type: text/x-patch Size: 98251 bytes Desc: not available URL: From simo at redhat.com Thu Dec 3 17:14:15 2015 From: simo at redhat.com (Simo Sorce) Date: Thu, 03 Dec 2015 12:14:15 -0500 Subject: [Freeipa-devel] [PATCH 562-563] Fix ipa-sam to use the getkeytab control instead of the setkeytab control Message-ID: <1449162855.3445.3.camel@redhat.com> The first patch is preparatory and is needed in general now that we want top allow alias and use krbCanonicalName as the canonical name when multiple values are avilable in krbPrincipalName. The second patch changes slightly how the interdomain trust account is created so that the getkeytab control can generate the proper key (with the right salt) for interop reasons with AD. The change should be upgrade safe because keys are generate at account creation so older accounts lacking the alias won't be a problem. Fixes ##5495 Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-562-1-Improve-keytab-code-to-select-the-right-principal.patch Type: text/x-patch Size: 6208 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-563-1-Convert-ipa-sam-to-use-the-new-getkeytab-control.patch Type: text/x-patch Size: 9800 bytes Desc: not available URL: From simo at redhat.com Thu Dec 3 17:14:48 2015 From: simo at redhat.com (Simo Sorce) Date: Thu, 03 Dec 2015 12:14:48 -0500 Subject: [Freeipa-devel] [PATCH 556-557] Add option to disable setkeytab extended operations In-Reply-To: <20151203102137.GE8374@redhat.com> References: <1448462853.29102.44.camel@redhat.com> <1448908709.3747.93.camel@redhat.com> <20151201081145.GJ9605@redhat.com> <1448977085.9040.1.camel@redhat.com> <565DA3DE.8000109@redhat.com> <1448978341.9040.5.camel@redhat.com> <565DB84C.2010401@redhat.com> <20151201152042.GQ9605@redhat.com> <565DC05B.1050804@redhat.com> <1449080213.9040.57.camel@redhat.com> <20151203102137.GE8374@redhat.com> Message-ID: <1449162888.3445.4.camel@redhat.com> On Thu, 2015-12-03 at 12:21 +0200, Alexander Bokovoy wrote: > On Wed, 02 Dec 2015, Simo Sorce wrote: > >> >>>> We also have to fix the permission to change keytab, so that it uses > >> >>>> the new > >> >>>> style (https://fedorahosted.org/freeipa/ticket/5487). I would > >> >>>> actually make > >> >>>> this ticket and the ipa-sam ticket a blocker for this patch set. > >> >>>> > >> >>>> Otherwise you are actually introducing a switch that breaks FreeIPA > >> >>>> as some of > >> >>>> it's core server functions still use the old method. > >> >>> > >> >>> The point of this patchset is to introduce the switch early so that > >> >>> current server will support the off switch when newer servers down the > >> >>> road are ready to flip it. The defaults are still to allow these > >> >>> operations for services/hosts. > >> >> > >> >> I still do not get the logic about an early switch. Now, if switch is > >> >> turned > >> >> on, FreeIPA server breaks on several places. I would really rather > >> >> expect the > >> >> switch to be introduced when the server itself supports it. Then, > >> >> admin would > >> >> enable it when the clients are sufficiently updated and can use the > >> >> new method. > >> >> > >> >> Why would admin want to enable the switch early if it breaks FreeIPA > >> >> some of > >> >> the FreeIPA servers? Permission can be upgraded in newer version and get > >> >> replicated, that's fine. But ipa-sam would be still broken on this old > >> >> FreeIPA > >> >> server. > >> > Old server is not a problem here: ipa-sam only talks to the > >> > localhost-based server as we always use ldapi protocol. So if server is > >> > running old-behavior FreeIPA, ipa-sam on the same server will work > >> > against it. > >> > > >> > What I don't want to have is a situation where setkeytab is disabled and > >> > a new server obeys it but ipa-sam on this new server is not updated and > >> > will expectedly fail. We are not that forced to do the change right now > >> > in 4.3, given that the default will still be to keep setkeytab, thus we > >> > can wait with this patch until ipa-sam is fixed and then push both > >> > patches into the closest release, be it 4.3.x (x>=0) or whatever is the > >> > next one. > >> > > >> > >> +1, fixing ipa-sam would be then a release blocker for 4.3 which is not > >> desired. > >> > >> Also what about adding support for "ipaProtectedoperation check" for > >> user principals? > >> > >> I'm afraid that forbidding getting user principal might be regarded as a > >> regression which might cause that admins won't set DisableSetKeytab. > > > >One thing at a time. > >We have bugs open for all these things, but I see no reason not to add > >an *optional* setting just because some things break when it is turned > >on. > The problem is that current getkeytab extended operation has not enough > information to be a replacement for ipa-sam's use of setkeytab, as I've > found with your current patches. So adding an optional setting in the > hope that it will not be used in real life is a bit naive, but if people > activate it, whole trust setup will break. I still prefer to have the > patchset first be completed and then merged. There is no problem in > keeping it aside while functionality is being implemented, we can > trivially rebase. I just sent patches 562 and 563 which will address the ipa-sam issue (tested by Alexander already). Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Thu Dec 3 17:15:39 2015 From: simo at redhat.com (Simo Sorce) Date: Thu, 03 Dec 2015 12:15:39 -0500 Subject: [Freeipa-devel] [PATCH 556-557] Add option to disable setkeytab extended operations In-Reply-To: <565FFDFE.5010604@redhat.com> References: <1448399848.29102.13.camel@redhat.com> <56555AC7.2000908@redhat.com> <5655BF5B.6020405@redhat.com> <1448462853.29102.44.camel@redhat.com> <1448908709.3747.93.camel@redhat.com> <20151201081145.GJ9605@redhat.com> <1448977085.9040.1.camel@redhat.com> <565DA3DE.8000109@redhat.com> <1448978341.9040.5.camel@redhat.com> <565DB84C.2010401@redhat.com> <20151201152042.GQ9605@redhat.com> <565DC05B.1050804@redhat.com> <1449080213.9040.57.camel@redhat.com> <565FFDFE.5010604@redhat.com> Message-ID: <1449162939.3445.5.camel@redhat.com> On Thu, 2015-12-03 at 09:31 +0100, Petr Vobornik wrote: > On 12/02/2015 07:16 PM, Simo Sorce wrote: > > On Tue, 2015-12-01 at 16:44 +0100, Petr Vobornik wrote: > >> On 12/01/2015 04:20 PM, Alexander Bokovoy wrote: > >>> On Tue, 01 Dec 2015, Martin Kosek wrote: > >>>> On 12/01/2015 02:59 PM, Simo Sorce wrote: > >>>>> On Tue, 2015-12-01 at 14:42 +0100, Martin Kosek wrote: > >>>>>> On 12/01/2015 02:38 PM, Simo Sorce wrote: > >>>>>>> On Tue, 2015-12-01 at 10:11 +0200, Alexander Bokovoy wrote: > >>>>>>>> On Mon, 30 Nov 2015, Simo Sorce wrote: > >>>>>>>>> On Wed, 2015-11-25 at 09:47 -0500, Simo Sorce wrote: > >>>>>>>>>> On Wed, 2015-11-25 at 09:02 -0500, Rob Crittenden wrote: > >>>>>>>>>>> Jan Cholasta wrote: > >>>>>>>>>>>> On 24.11.2015 22:17, Simo Sorce wrote: > >>>>>>>>>>>>> On Tue, 2015-11-24 at 14:57 -0500, Simo Sorce wrote: > >>>>>>>>>>>>>> On Tue, 2015-11-24 at 14:42 -0500, Simo Sorce wrote: > >>>>>>>>>>>>>>> Since some time we use the getkeytab operation to fetch > >>>>>>>>>>>>>>> keytabs on > >>>>>>>>>>>>>>> newer > >>>>>>>>>>>>>>> clients. According to bug #232 setkeytab can be used to > >>>>>>>>>>>>>>> circumvent > >>>>>>>>>>>>>>> password quality controls so it needs to be slowly retired. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> The attached patches implement #5485 in 2 parts. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> The first introduces the option DisableSetKeytab which > >>>>>>>>>>>>>>> globally > >>>>>>>>>>>>>>> disables > >>>>>>>>>>>>>>> the setkeytab extended operation. This is set to false by > >>>>>>>>>>>>>>> default for > >>>>>>>>>>>>>>> backwards compatibility. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> The second introduces an option called > >>>>>>>>>>>>>>> DisableUserSetKeytab, which is > >>>>>>>>>>>>>>> active by default in new installs (but not in upgraded > >>>>>>>>>>>>>>> ones), and only > >>>>>>>>>>>>>>> disables the use of setkeytab for ipa suers, but not for > >>>>>>>>>>>>>>> hosts/services. > >>>>>>>>>>>>>>> This is because user's are the ones that may abuse the > >>>>>>>>>>>>>>> interface to > >>>>>>>>>>>>>>> escape password policies and users also normally do not > >>>>>>>>>>>>>>> acquire > >>>>>>>>>>>>>>> keytabs, > >>>>>>>>>>>>>>> so it is a safe bet to disable just them by default in new > >>>>>>>>>>>>>>> installs. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> (Testing in progress) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Tested and working as expected. > >>>>>>>>>>>>> > >>>>>>>>>>>>> I realized that adding options to ipaConfig require to add > >>>>>>>>>>>>> them in the > >>>>>>>>>>>>> UI as well, attached patches add options in API.txt and > >>>>>>>>>>>>> config.py > >>>>>>>>>>>>> Make now complain I should change API Major or Minor, but it > >>>>>>>>>>>>> is not > >>>>>>>>>>>>> clear to me why given this are additional values and no real > >>>>>>>>>>>>> change or > >>>>>>>>>>>>> new function is introduced. What's the recommendation ? > >>>>>>>>>>>> > >>>>>>>>>>>> When does make complain? It is supposed to complain only when > >>>>>>>>>>>> API.txt > >>>>>>>>>>>> does not match code. > >>>>>>>>>>>> > >>>>>>>>>>>> Anyway, we usually bump minor version even for backward > >>>>>>>>>>>> compatible > >>>>>>>>>>>> changes, see e.g. commit 9549a59. > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> The point of API.txt (and the heavy client) was to save a > >>>>>>>>>>> round-trip. > >>>>>>>>>>> Being able to pass in an invalid option would void that rule hence > >>>>>>>>>>> having to update the API when new values are added. > >>>>>>>>>>> > >>>>>>>>>>> rob > >>>>>>>>>> > >>>>>>>>>> Ok added version change to the second patch (so we bump it only > >>>>>>>>>> once > >>>>>>>>>> given these are basically related changes. > >>>>>>>>> > >>>>>>>>> Bump, is this ok ? > >>>>>>>> This patch is fine but please fix setkeytab use in ipa-sam before > >>>>>>>> committing this patch. > >>>>>>> > >>>>>>> This patch does not disable setkeytab yet, so it can go in right away > >>>>>>> (it blocks other patches already). We have a bug to change ipa-sam, > >>>>>>> should we mark it blocker for 4.4 ? > >>>>>> > >>>>>> We also have to fix the permission to change keytab, so that it uses > >>>>>> the new > >>>>>> style (https://fedorahosted.org/freeipa/ticket/5487). I would > >>>>>> actually make > >>>>>> this ticket and the ipa-sam ticket a blocker for this patch set. > >>>>>> > >>>>>> Otherwise you are actually introducing a switch that breaks FreeIPA > >>>>>> as some of > >>>>>> it's core server functions still use the old method. > >>>>> > >>>>> The point of this patchset is to introduce the switch early so that > >>>>> current server will support the off switch when newer servers down the > >>>>> road are ready to flip it. The defaults are still to allow these > >>>>> operations for services/hosts. > >>>> > >>>> I still do not get the logic about an early switch. Now, if switch is > >>>> turned > >>>> on, FreeIPA server breaks on several places. I would really rather > >>>> expect the > >>>> switch to be introduced when the server itself supports it. Then, > >>>> admin would > >>>> enable it when the clients are sufficiently updated and can use the > >>>> new method. > >>>> > >>>> Why would admin want to enable the switch early if it breaks FreeIPA > >>>> some of > >>>> the FreeIPA servers? Permission can be upgraded in newer version and get > >>>> replicated, that's fine. But ipa-sam would be still broken on this old > >>>> FreeIPA > >>>> server. > >>> Old server is not a problem here: ipa-sam only talks to the > >>> localhost-based server as we always use ldapi protocol. So if server is > >>> running old-behavior FreeIPA, ipa-sam on the same server will work > >>> against it. > >>> > >>> What I don't want to have is a situation where setkeytab is disabled and > >>> a new server obeys it but ipa-sam on this new server is not updated and > >>> will expectedly fail. We are not that forced to do the change right now > >>> in 4.3, given that the default will still be to keep setkeytab, thus we > >>> can wait with this patch until ipa-sam is fixed and then push both > >>> patches into the closest release, be it 4.3.x (x>=0) or whatever is the > >>> next one. > >>> > >> > >> +1, fixing ipa-sam would be then a release blocker for 4.3 which is not > >> desired. > >> > >> Also what about adding support for "ipaProtectedoperation check" for > >> user principals? > >> > >> I'm afraid that forbidding getting user principal might be regarded as a > >> regression which might cause that admins won't set DisableSetKeytab. > > > > One thing at a time. > > We have bugs open for all these things, but I see no reason not to add > > an *optional* setting just because some things break when it is turned > > on. > > > > Simo. > > > > I would agree if the optional setting(DisableUserSetKeytab) was not > enabled by default. Do we have a ticket to fix whatever is broken by this ? If not can you please create one with the details ? In either case can you assign the ticket to me ? Simo. -- Simo Sorce * Red Hat, Inc * New York From abokovoy at redhat.com Thu Dec 3 17:33:08 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 3 Dec 2015 19:33:08 +0200 Subject: [Freeipa-devel] [PATCH 562-563] Fix ipa-sam to use the getkeytab control instead of the setkeytab control In-Reply-To: <1449162855.3445.3.camel@redhat.com> References: <1449162855.3445.3.camel@redhat.com> Message-ID: <20151203173308.GI8374@redhat.com> On Thu, 03 Dec 2015, Simo Sorce wrote: >The first patch is preparatory and is needed in general now that we want >top allow alias and use krbCanonicalName as the canonical name when >multiple values are avilable in krbPrincipalName. > >The second patch changes slightly how the interdomain trust account is >created so that the getkeytab control can generate the proper key (with >the right salt) for interop reasons with AD. The change should be >upgrade safe because keys are generate at account creation so older >accounts lacking the alias won't be a problem. > >Fixes ##5495 Thanks. ACK to both. They work for me against Windows Server 2012R2. Now we need to fix Samba AD salt generation so that it is compatible with both Windows and FreeIPA for AES/DES keys and not only RC4... ;) -- / Alexander Bokovoy From simo at redhat.com Thu Dec 3 18:52:35 2015 From: simo at redhat.com (Simo Sorce) Date: Thu, 03 Dec 2015 13:52:35 -0500 Subject: [Freeipa-devel] [PATCH 562-563] Fix ipa-sam to use the getkeytab control instead of the setkeytab control In-Reply-To: <20151203173308.GI8374@redhat.com> References: <1449162855.3445.3.camel@redhat.com> <20151203173308.GI8374@redhat.com> Message-ID: <1449168755.3445.19.camel@redhat.com> On Thu, 2015-12-03 at 19:33 +0200, Alexander Bokovoy wrote: > On Thu, 03 Dec 2015, Simo Sorce wrote: > >The first patch is preparatory and is needed in general now that we want > >top allow alias and use krbCanonicalName as the canonical name when > >multiple values are avilable in krbPrincipalName. > > > >The second patch changes slightly how the interdomain trust account is > >created so that the getkeytab control can generate the proper key (with > >the right salt) for interop reasons with AD. The change should be > >upgrade safe because keys are generate at account creation so older > >accounts lacking the alias won't be a problem. > > > >Fixes ##5495 > Thanks. ACK to both. They work for me against Windows Server 2012R2. > > Now we need to fix Samba AD salt generation so that it is compatible > with both Windows and FreeIPA for AES/DES keys and not only RC4... ;) And so we did: https://git.samba.org/?p=idra/samba.git;a=commitdiff;h=8e87601a998b43f58589ff88341946ca4d9ab5ee;hp=412cefc7c8222ccc77e15099a162f9fb7bb01c57 and: https://twitter.com/abbrasuo/status/672480716928716800 :-) Simo. -- Simo Sorce * Red Hat, Inc * New York From abokovoy at redhat.com Thu Dec 3 19:14:53 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 3 Dec 2015 21:14:53 +0200 Subject: [Freeipa-devel] [PATCH 562-563] Fix ipa-sam to use the getkeytab control instead of the setkeytab control In-Reply-To: <1449168755.3445.19.camel@redhat.com> References: <1449162855.3445.3.camel@redhat.com> <20151203173308.GI8374@redhat.com> <1449168755.3445.19.camel@redhat.com> Message-ID: <20151203191453.GK8374@redhat.com> On Thu, 03 Dec 2015, Simo Sorce wrote: >On Thu, 2015-12-03 at 19:33 +0200, Alexander Bokovoy wrote: >> On Thu, 03 Dec 2015, Simo Sorce wrote: >> >The first patch is preparatory and is needed in general now that we want >> >top allow alias and use krbCanonicalName as the canonical name when >> >multiple values are avilable in krbPrincipalName. >> > >> >The second patch changes slightly how the interdomain trust account is >> >created so that the getkeytab control can generate the proper key (with >> >the right salt) for interop reasons with AD. The change should be >> >upgrade safe because keys are generate at account creation so older >> >accounts lacking the alias won't be a problem. >> > >> >Fixes ##5495 >> Thanks. ACK to both. They work for me against Windows Server 2012R2. >> >> Now we need to fix Samba AD salt generation so that it is compatible >> with both Windows and FreeIPA for AES/DES keys and not only RC4... ;) > >And so we did: >https://git.samba.org/?p=idra/samba.git;a=commitdiff;h=8e87601a998b43f58589ff88341946ca4d9ab5ee;hp=412cefc7c8222ccc77e15099a162f9fb7bb01c57 >and: >https://twitter.com/abbrasuo/status/672480716928716800 > >:-) Yep, thanks Simo! I've posted few screenshots of the current status of Samba AD with MIT Kerberos running on Fedora 23 and establishing cross-forest trust to FreeIPA on my Google+ page: https://plus.google.com/+AlexanderBokovoy/posts/NgozL7Rgw64 The patches to Samba are in Andreas' git tree, plus few changes Simo did for proper generation of the salt for interdomain trust object keys. Currently Samba generates the salt principal wrongly for TDO keys and it works in Heimdal only because Heimdal users RC4 keys for cross-realm trust which does not use the salt. Once Simo fixed the salt in password_hash ldb module, we were able to complete trust to FreeIPA in such way that MIT KDC was able to respond on AS request for the interdomain TDO principal and SSSD on FreeIPA side was able to use the resulting Kerberos session to authenticate with SASL GSSAPI to Samba AD's LDAP to look up users and groups. The POSIX attributes are managed by FreeIPA (UID/GIDs are autogenerated in this deployment) but they can also be picked up from Samba AD. We plan to work on remaining fixes to eventually get the full Samba AD support in Fedora 24, but this represents a huge milestone in our four year quest to make it a reality. Thanks to everyone! -- / Alexander Bokovoy From fskola at redhat.com Thu Dec 3 19:15:56 2015 From: fskola at redhat.com (Filip =?UTF-8?B?xaBrb2xh?=) Date: Thu, 3 Dec 2015 20:15:56 +0100 Subject: [Freeipa-devel] [PATCH 0002] Refactor test_group_plugin In-Reply-To: <565C76D6.3010406@redhat.com> References: <20151120135636.71171d5c@vor2.netbox.priv> <20151123145934.36901470@dhcp-25-21.brq.redhat.com> <20151123164249.4a576706@dhcp-25-21.brq.redhat.com> <565C76D6.3010406@redhat.com> Message-ID: <20151203201556.4ff69dba@vor2.netbox.priv> On Mon, 30 Nov 2015 17:18:30 +0100 Milan Kub?k wrote: > On 11/23/2015 04:42 PM, Filip ?kola wrote: > > Sending updated patch. > > > > F. > > > > On Mon, 23 Nov 2015 14:59:34 +0100 > > Filip ?kola wrote: > > > >> Found couple of issues (broke some dependencies). > >> > >> NACK > >> > >> F. > >> > >> On Fri, 20 Nov 2015 13:56:36 +0100 > >> Filip ?kola wrote: > >> > >>> Another one. > >>> > >>> F. > >> > > > > > > Hi, the tests look good. Few remarks, though. > > 1. Please, use the shortes copyright notice in new modules. > > # > # Copyright (C) 2015 FreeIPA Contributors see COPYING for > license # > > 2. The tests `test_group_remove_group_from_protected_group` and > `test_group_full_set_of_objectclass_not_available_post_detach` > were not ported. Please, include them in the patch. > > Also, for less hassle, please rebase your patches on top of > freeipa-mkubik-0025-3-Separated-Tracker-implementations-into-standalone-pa.patch > Which changes the location of tracker implementations and prevents > circular imports. > > Thanks. > Hi, these cases are there, in corresponding classes. They are marked with the original comments. (However I can move them to separate class if desirable.) The copyright notice is changed. Also included a few changes in the test with user without private group. Filip -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-fskola-0002-3-Refactor-test_group_plugin.patch Type: text/x-patch Size: 73752 bytes Desc: not available URL: From jcholast at redhat.com Fri Dec 4 07:51:26 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 4 Dec 2015 08:51:26 +0100 Subject: [Freeipa-devel] [PATCH] 0748 Handle encoding for ipautil.run In-Reply-To: <566054D7.9020008@redhat.com> References: <5638B8F9.6010703@redhat.com> <564D0FD9.4070505@redhat.com> <5652B5AC.2040508@redhat.com> <5652E174.50301@redhat.com> <56548E73.1090105@redhat.com> <565587AE.2070303@redhat.com> <5655CA08.8090701@redhat.com> <565C01FD.7050906@redhat.com> <565D83CC.8020802@redhat.com> <565EA4FA.3020307@redhat.com> <565EF7B9.6020507@redhat.com> <566011A5.4040200@redhat.com> <566054D7.9020008@redhat.com> Message-ID: <566145FE.3090005@redhat.com> On 3.12.2015 15:42, Petr Viktorin wrote: > On 12/03/2015 10:55 AM, Jan Cholasta wrote: > [...] >>>>>>>> If we don't care about output somewhere, we should not capture it >>>>>>>> there >>>>>>>> at all. >>>>>>> >>>>>>> Then people need to remember to put "capture_output=True" everywhere >>>>>>> (except that also disables logging of the output, so we'd need an >>>>>>> additional option). >>>>>> >>>>>> capture_output=True is the default and capture_output=False should be >>>>>> set where output is not needed. >>>>> >>>>> "capture_output=True" being the default would mean that it's still >>>>> possible to leave "capture_output=False" out, but ignore the output. It >>>>> would make the wrong usage easier to type than the right one, which is >>>>> something to avoid. >>>> >>>> I'm not sure I understand what are you trying to say here. >>>> >>>> My point is that capture_output=True is currently the default (see for >>>> yourself) and it is indeed possible to leave capture_output=False but >>>> ignore the output. I believe this is wrong and that you should always >>>> have to explicitly request the output to be captured. >>> >>> OK, looks like I will need to refactor ipautil.run even for Python 2, >>> then. >> >> If this is just about changing the default and fixing all run() >> invocations, I can provide the patch myself, if that would work better >> for you. >> >>> I seem to have have problems coming up with a solution that would match >>> your expectations, so let me write the semantics as I understand you'd >>> like hem: >>> >>> Would you be happy with the following signature? >>> >>> def run(args, stdin=None, raiseonerr=True, >>> nolog=(), env=None, >>> capture_output=False, skip_output=False, >>> cwd=None, runas=None, timeout=None, suplementary_groups=[], >>> capture_stderr=False, >>> encode_stdout=True, >>> encode_stderr=True, >>> ): >>> >>> Output and error would be always logged, except if skip_output is set. >>> But they would be only returned if capture_output/capture_stderr were >>> set. >> >> I would personally also rename capture_output to capture_stdout. If we >> change the default, we have to update its every use, so we might as well >> rename it. (Again, I can provide the patch.) > > If we agree on the semantics, writing the patch isn't that big a deal. Right. > >>> If encode_* is False, bytes are returned, by default >>> locale.getpreferredencoding() is used. (If the caller wants any other >>> encoding, it can do that by itself.) >> >> I thought bytes should be the default after all? See my comment in the >> previous mail. > > If the default is no capturing, I'd rather go with text by default. > With capturing by deafult, "run(args)" could raise encoding errors *if > the output is then ignored*. If it's "run(args, capture_stdout=True)", I > think it's fine to encode. OK, makes sense. > >>> stdin it encoded using locale.getpreferredencoding() if it's unicode. >> >> If we do encode/decode in run(), I think there should be a way to >> override the default encoding. My idea was to either accept/return only >> bytes and let the caller handle encoding themselves, or to handle >> encoding in run(), but be able to override the defaults. >> >> Given we handle encoding in run(), I would imagine it like this: >> >> def run(args, stdin=None, raiseonerr=True, nolog=(), env=None, >> capture_stdout=False, skip_output=False, cwd=None, >> runas=None, timeout=None, suplementary_groups=[], >> capture_stderr=False, encode_stdout=False, >> encode_stderr=False, encoding=None): >> >> i.e. nothing is captured by default, when stdout or stderr are captured >> they are returned as bytes by default, when stdout or stderr are >> returned as text they are decoded using the locale encoding by default, >> or the encoding can be explicitly specified. >> >> Do you think this is feasible? > > Feasible, yes. > In the majority of cases where the output is needed, we want text > encoded with locale.getpreferredencoding(), so I'd rather make that the > default rather than bytes. > > Or we could make "encode_stdout" imply "capture_stdout", so you wouldn't > have to always specify both. (It would be better named "encoded_stdout" > in that case.) I think it should be OK to decode by default. (IMO it would be best to reverse the logic and name it "raw_stdout", with False default. Do we need "raw_stderr" at all? I don't think we do.) So the signature will be this: def run(args, stdin=None, raiseonerr=True, nolog=(), env=None, capture_stdout=False, skip_output=False, cwd=None, runas=None, timeout=None, suplementary_groups=[], capture_stderr=False, raw_stdout=False, encoding=None): Then, when you want to capture just error messages on stderr: rc, _out, err = run(args, capture_stderr=True) When you want to capture text on both stdout and stderr: rc, out, err = run(args, capture_stdout=True, capture_stderr=True) When you want to capture bytes on stdout and text on stderr (think "certutil -L -r"): rc, out, err = run(args, capture_stdout=True, capture_stderr=True, raw_stdout=True) When you want to capture text on both stdout and stderr, where the text on stdout uses different encoding (think your wget example from before): rc, out, err = run(args, capture_stdout=True, capture_stderr=True, raw_stdout=True) out = out.decode('utf-8') And when you want to force the "C" locale: rc, _out, err = run(args, env={'LC_ALL': 'C'}, capture_stderr=True, encoding='ascii') Right? -- Jan Cholasta From fskola at redhat.com Fri Dec 4 09:04:02 2015 From: fskola at redhat.com (Filip =?UTF-8?B?xaBrb2xh?=) Date: Fri, 4 Dec 2015 10:04:02 +0100 Subject: [Freeipa-devel] [PATCH 0003] Refactor test_replace Message-ID: <20151204100402.0474eeaf@vor2.netbox.priv> Hi, sending rather short one this time. F. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-fskola-0003-Refactor-test_replace.patch Type: text/x-patch Size: 6664 bytes Desc: not available URL: From mkubik at redhat.com Fri Dec 4 09:08:40 2015 From: mkubik at redhat.com (=?UTF-8?Q?Milan_Kub=c3=adk?=) Date: Fri, 4 Dec 2015 10:08:40 +0100 Subject: [Freeipa-devel] [PATCH 0003] Refactor test_replace In-Reply-To: <20151204100402.0474eeaf@vor2.netbox.priv> References: <20151204100402.0474eeaf@vor2.netbox.priv> Message-ID: <56615818.6060402@redhat.com> On 12/04/2015 10:04 AM, Filip ?kola wrote: > Hi, > > sending rather short one this time. > > F. NACK, UserTracker is implemented in ipatests.test_xmlrpc.tracker.user_plugin. -- Milan Kubik From pviktori at redhat.com Fri Dec 4 09:53:54 2015 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 4 Dec 2015 10:53:54 +0100 Subject: [Freeipa-devel] [PATCH] 0748 Handle encoding for ipautil.run In-Reply-To: <566145FE.3090005@redhat.com> References: <5638B8F9.6010703@redhat.com> <564D0FD9.4070505@redhat.com> <5652B5AC.2040508@redhat.com> <5652E174.50301@redhat.com> <56548E73.1090105@redhat.com> <565587AE.2070303@redhat.com> <5655CA08.8090701@redhat.com> <565C01FD.7050906@redhat.com> <565D83CC.8020802@redhat.com> <565EA4FA.3020307@redhat.com> <565EF7B9.6020507@redhat.com> <566011A5.4040200@redhat.com> <566054D7.9020008@redhat.com> <566145FE.3090005@redhat.com> Message-ID: <566162B2.9010809@redhat.com> On 12/04/2015 08:51 AM, Jan Cholasta wrote: > On 3.12.2015 15:42, Petr Viktorin wrote: >> On 12/03/2015 10:55 AM, Jan Cholasta wrote: >> [...] >>>>>>>>> If we don't care about output somewhere, we should not capture it >>>>>>>>> there >>>>>>>>> at all. >>>>>>>> >>>>>>>> Then people need to remember to put "capture_output=True" >>>>>>>> everywhere >>>>>>>> (except that also disables logging of the output, so we'd need an >>>>>>>> additional option). >>>>>>> >>>>>>> capture_output=True is the default and capture_output=False >>>>>>> should be >>>>>>> set where output is not needed. >>>>>> >>>>>> "capture_output=True" being the default would mean that it's still >>>>>> possible to leave "capture_output=False" out, but ignore the >>>>>> output. It >>>>>> would make the wrong usage easier to type than the right one, >>>>>> which is >>>>>> something to avoid. >>>>> >>>>> I'm not sure I understand what are you trying to say here. >>>>> >>>>> My point is that capture_output=True is currently the default (see for >>>>> yourself) and it is indeed possible to leave capture_output=False but >>>>> ignore the output. I believe this is wrong and that you should always >>>>> have to explicitly request the output to be captured. >>>> >>>> OK, looks like I will need to refactor ipautil.run even for Python 2, >>>> then. >>> >>> If this is just about changing the default and fixing all run() >>> invocations, I can provide the patch myself, if that would work better >>> for you. >>> >>>> I seem to have have problems coming up with a solution that would match >>>> your expectations, so let me write the semantics as I understand you'd >>>> like hem: >>>> >>>> Would you be happy with the following signature? >>>> >>>> def run(args, stdin=None, raiseonerr=True, >>>> nolog=(), env=None, >>>> capture_output=False, skip_output=False, >>>> cwd=None, runas=None, timeout=None, suplementary_groups=[], >>>> capture_stderr=False, >>>> encode_stdout=True, >>>> encode_stderr=True, >>>> ): >>>> >>>> Output and error would be always logged, except if skip_output is set. >>>> But they would be only returned if capture_output/capture_stderr were >>>> set. >>> >>> I would personally also rename capture_output to capture_stdout. If we >>> change the default, we have to update its every use, so we might as well >>> rename it. (Again, I can provide the patch.) >> >> If we agree on the semantics, writing the patch isn't that big a deal. > > Right. > >> >>>> If encode_* is False, bytes are returned, by default >>>> locale.getpreferredencoding() is used. (If the caller wants any other >>>> encoding, it can do that by itself.) >>> >>> I thought bytes should be the default after all? See my comment in the >>> previous mail. >> >> If the default is no capturing, I'd rather go with text by default. >> With capturing by deafult, "run(args)" could raise encoding errors *if >> the output is then ignored*. If it's "run(args, capture_stdout=True)", I >> think it's fine to encode. > > OK, makes sense. > >> >>>> stdin it encoded using locale.getpreferredencoding() if it's unicode. >>> >>> If we do encode/decode in run(), I think there should be a way to >>> override the default encoding. My idea was to either accept/return only >>> bytes and let the caller handle encoding themselves, or to handle >>> encoding in run(), but be able to override the defaults. >>> >>> Given we handle encoding in run(), I would imagine it like this: >>> >>> def run(args, stdin=None, raiseonerr=True, nolog=(), env=None, >>> capture_stdout=False, skip_output=False, cwd=None, >>> runas=None, timeout=None, suplementary_groups=[], >>> capture_stderr=False, encode_stdout=False, >>> encode_stderr=False, encoding=None): >>> >>> i.e. nothing is captured by default, when stdout or stderr are captured >>> they are returned as bytes by default, when stdout or stderr are >>> returned as text they are decoded using the locale encoding by default, >>> or the encoding can be explicitly specified. >>> >>> Do you think this is feasible? >> >> Feasible, yes. >> In the majority of cases where the output is needed, we want text >> encoded with locale.getpreferredencoding(), so I'd rather make that the >> default rather than bytes. >> >> Or we could make "encode_stdout" imply "capture_stdout", so you wouldn't >> have to always specify both. (It would be better named "encoded_stdout" >> in that case.) > > I think it should be OK to decode by default. (IMO it would be best to > reverse the logic and name it "raw_stdout", with False default. Do we > need "raw_stderr" at all? I don't think we do.) Actually, now that I think about it: if the result was a namedtuple subclass, we can always set extra raw_stdout/raw_stderr attributes on it. (Unless skip_output=True, of course.) result = run(['generate_binary_data']) use(result.raw_stdout) # and for backcompat, rc, _out, _err = result > So the signature will be this: > > def run(args, stdin=None, raiseonerr=True, nolog=(), env=None, > capture_stdout=False, skip_output=False, cwd=None, > runas=None, timeout=None, suplementary_groups=[], > capture_stderr=False, raw_stdout=False, encoding=None): > > Then, when you want to capture just error messages on stderr: > > rc, _out, err = run(args, > capture_stderr=True) With namedtuple: result = run(args, capture_stderr=True, raiseonerr=False) if result.rc != 0: complain(result.err) > When you want to capture text on both stdout and stderr: > > rc, out, err = run(args, > capture_stdout=True, > capture_stderr=True) > > When you want to capture bytes on stdout and text on stderr (think > "certutil -L -r"): > > rc, out, err = run(args, > capture_stdout=True, > capture_stderr=True, > raw_stdout=True) result = run(args, capture_stderr=True) data = result.raw_stdout > When you want to capture text on both stdout and stderr, where the text > on stdout uses different encoding (think your wget example from before): > > rc, out, err = run(args, > capture_stdout=True, > capture_stderr=True, > raw_stdout=True) > out = out.decode('utf-8') result = run(args, capture_stderr=True) out = result.raw_stdout.decode('utf-8') > And when you want to force the "C" locale: > > rc, _out, err = run(args, > env={'LC_ALL': 'C'}, > capture_stderr=True, > encoding='ascii') It's more complicated if the existing env matters: env = dict(os.environ) env['LC_ALL'] = 'C' result = run(args, env=env, capture_stderr=True, encoding='ascii') -- Petr Viktorin From pvoborni at redhat.com Fri Dec 4 10:15:58 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 4 Dec 2015 11:15:58 +0100 Subject: [Freeipa-devel] [PATCHES 509-514] replica promotion: use host credentials when setting up replication In-Reply-To: <56604D94.7000705@redhat.com> References: <564DE017.4020408@redhat.com> <1447951434.5132.27.camel@redhat.com> <564F4105.1040204@redhat.com> <1448038723.5132.59.camel@redhat.com> <5652C63E.5090206@redhat.com> <1448289289.7892.13.camel@redhat.com> <565324C4.1090007@redhat.com> <1448290024.7892.15.camel@redhat.com> <565D823E.5090600@redhat.com> <56604D94.7000705@redhat.com> Message-ID: <566167DE.9090204@redhat.com> On 12/03/2015 03:11 PM, Martin Basti wrote: > > > On 01.12.2015 12:19, Jan Cholasta wrote: >> On 23.11.2015 15:47, Simo Sorce wrote: >>> On Mon, 2015-11-23 at 15:37 +0100, Jan Cholasta wrote: >>>> >>>> Ad alternative is to add the host to ipaservers before the checks are >>>> done and remove it again if any of them fail. >>> >>> Too error prone, I am ok with the current way in your patches >>> until/unless I can think of a fail safe way. :-) >> >> Updated patches attached. Note that 520 should be applied between 509 >> and 510. >> >> >> > Functional ACK > Simo, do you want to review the ACIs or other things it the patches? Or can the patches be pushed? -- Petr Vobornik From fskola at redhat.com Fri Dec 4 10:29:55 2015 From: fskola at redhat.com (Filip =?UTF-8?B?xaBrb2xh?=) Date: Fri, 4 Dec 2015 11:29:55 +0100 Subject: [Freeipa-devel] [PATCH 0003] Refactor test_replace In-Reply-To: <56615818.6060402@redhat.com> References: <20151204100402.0474eeaf@vor2.netbox.priv> <56615818.6060402@redhat.com> Message-ID: <20151204112955.51b5c72f@vor2.netbox.priv> On Fri, 4 Dec 2015 10:08:40 +0100 Milan Kub?k wrote: > On 12/04/2015 10:04 AM, Filip ?kola wrote: > > Hi, > > > > sending rather short one this time. > > > > F. > NACK, UserTracker is implemented in > ipatests.test_xmlrpc.tracker.user_plugin. > Ah, sorry for this. F. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-fskola-0003-2-Refactor-test_replace.patch Type: text/x-patch Size: 6667 bytes Desc: not available URL: From pvoborni at redhat.com Fri Dec 4 11:08:36 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 4 Dec 2015 12:08:36 +0100 Subject: [Freeipa-devel] [PATCH 0390] man: Update the ipa-replica-install manpage with promotion In-Reply-To: <56602D59.2050207@redhat.com> References: <56602223.6020906@redhat.com> <56602D59.2050207@redhat.com> Message-ID: <56617434.6000402@redhat.com> On 12/03/2015 12:54 PM, Petr Vobornik wrote: > On 12/03/2015 12:06 PM, Tomas Babej wrote: >> Hi, >> >> this patch updates the man page for the ipa-replica-install given the >> latest changes (including the Jan's OTP patch). >> >> Tomas >> > > > Questions/suggestions: > > 1. "you cannot provide an replica file" > > It's true, but wouldn't it be better to say that "you don't need to". Or > communicate that it is a good thing. Cannot sounds too negative to me. > > 2. Why options: --password, --admin-password are in 'BASIC OPTIONS' > section and not also in new 'ENROLLMENT OPTIONS' section together with > --principal, --keytab, etc..? Maybe 'ENROLLMENT OPTIONS' should be > before "BASIC OPTIONS" and "BASIC OPTIONS" renamed. > > > General ideas, not related to this patch: > """ > If the installation fails you may need to run ipa-server-install > --uninstall and ipa-client-install before running ipa-replica-install > again. > > The installation will fail if the host you are installing the > replica on exists as a host in IPA or an existing replication agreement > exists (for example, from a previously failed installation) > """ > > I believe validators check this situation, right? So do we need to write > it in the man page. Attaching updated patch which reflects off-line cooperation with Tomas. New option sections were added: DOMAIN LEVEL 1 OPTIONS DOMAIN LEVEL 1 CLIENT ENROLLMENT OPTIONS DOMAIN LEVEL 0 OPTIONS -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0390-1-man-Update-the-ipa-replica-install-manpage-with-prom.patch Type: text/x-patch Size: 5857 bytes Desc: not available URL: From pvoborni at redhat.com Fri Dec 4 11:09:40 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 4 Dec 2015 12:09:40 +0100 Subject: [Freeipa-devel] [PATCH] 940 Update ipa-(cs)replica-manage man pages In-Reply-To: <56606691.2090402@redhat.com> References: <56606691.2090402@redhat.com> Message-ID: <56617474.9020200@redhat.com> On 12/03/2015 04:58 PM, Petr Vobornik wrote: > SSIA > Updated patch attached. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0940-1-Update-ipa-cs-replica-manage-man-pages.patch Type: text/x-patch Size: 4018 bytes Desc: not available URL: From jcholast at redhat.com Fri Dec 4 11:15:28 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 4 Dec 2015 12:15:28 +0100 Subject: [Freeipa-devel] [PATCH] 0748 Handle encoding for ipautil.run In-Reply-To: <566162B2.9010809@redhat.com> References: <5638B8F9.6010703@redhat.com> <564D0FD9.4070505@redhat.com> <5652B5AC.2040508@redhat.com> <5652E174.50301@redhat.com> <56548E73.1090105@redhat.com> <565587AE.2070303@redhat.com> <5655CA08.8090701@redhat.com> <565C01FD.7050906@redhat.com> <565D83CC.8020802@redhat.com> <565EA4FA.3020307@redhat.com> <565EF7B9.6020507@redhat.com> <566011A5.4040200@redhat.com> <566054D7.9020008@redhat.com> <566145FE.3090005@redhat.com> <566162B2.9010809@redhat.com> Message-ID: <566175D0.1000105@redhat.com> On 4.12.2015 10:53, Petr Viktorin wrote: > On 12/04/2015 08:51 AM, Jan Cholasta wrote: >> On 3.12.2015 15:42, Petr Viktorin wrote: >>> On 12/03/2015 10:55 AM, Jan Cholasta wrote: >>> [...] >>>>>>>>>> If we don't care about output somewhere, we should not capture it >>>>>>>>>> there >>>>>>>>>> at all. >>>>>>>>> >>>>>>>>> Then people need to remember to put "capture_output=True" >>>>>>>>> everywhere >>>>>>>>> (except that also disables logging of the output, so we'd need an >>>>>>>>> additional option). >>>>>>>> >>>>>>>> capture_output=True is the default and capture_output=False >>>>>>>> should be >>>>>>>> set where output is not needed. >>>>>>> >>>>>>> "capture_output=True" being the default would mean that it's still >>>>>>> possible to leave "capture_output=False" out, but ignore the >>>>>>> output. It >>>>>>> would make the wrong usage easier to type than the right one, >>>>>>> which is >>>>>>> something to avoid. >>>>>> >>>>>> I'm not sure I understand what are you trying to say here. >>>>>> >>>>>> My point is that capture_output=True is currently the default (see for >>>>>> yourself) and it is indeed possible to leave capture_output=False but >>>>>> ignore the output. I believe this is wrong and that you should always >>>>>> have to explicitly request the output to be captured. >>>>> >>>>> OK, looks like I will need to refactor ipautil.run even for Python 2, >>>>> then. >>>> >>>> If this is just about changing the default and fixing all run() >>>> invocations, I can provide the patch myself, if that would work better >>>> for you. >>>> >>>>> I seem to have have problems coming up with a solution that would match >>>>> your expectations, so let me write the semantics as I understand you'd >>>>> like hem: >>>>> >>>>> Would you be happy with the following signature? >>>>> >>>>> def run(args, stdin=None, raiseonerr=True, >>>>> nolog=(), env=None, >>>>> capture_output=False, skip_output=False, >>>>> cwd=None, runas=None, timeout=None, suplementary_groups=[], >>>>> capture_stderr=False, >>>>> encode_stdout=True, >>>>> encode_stderr=True, >>>>> ): >>>>> >>>>> Output and error would be always logged, except if skip_output is set. >>>>> But they would be only returned if capture_output/capture_stderr were >>>>> set. >>>> >>>> I would personally also rename capture_output to capture_stdout. If we >>>> change the default, we have to update its every use, so we might as well >>>> rename it. (Again, I can provide the patch.) >>> >>> If we agree on the semantics, writing the patch isn't that big a deal. >> >> Right. >> >>> >>>>> If encode_* is False, bytes are returned, by default >>>>> locale.getpreferredencoding() is used. (If the caller wants any other >>>>> encoding, it can do that by itself.) >>>> >>>> I thought bytes should be the default after all? See my comment in the >>>> previous mail. >>> >>> If the default is no capturing, I'd rather go with text by default. >>> With capturing by deafult, "run(args)" could raise encoding errors *if >>> the output is then ignored*. If it's "run(args, capture_stdout=True)", I >>> think it's fine to encode. >> >> OK, makes sense. >> >>> >>>>> stdin it encoded using locale.getpreferredencoding() if it's unicode. >>>> >>>> If we do encode/decode in run(), I think there should be a way to >>>> override the default encoding. My idea was to either accept/return only >>>> bytes and let the caller handle encoding themselves, or to handle >>>> encoding in run(), but be able to override the defaults. >>>> >>>> Given we handle encoding in run(), I would imagine it like this: >>>> >>>> def run(args, stdin=None, raiseonerr=True, nolog=(), env=None, >>>> capture_stdout=False, skip_output=False, cwd=None, >>>> runas=None, timeout=None, suplementary_groups=[], >>>> capture_stderr=False, encode_stdout=False, >>>> encode_stderr=False, encoding=None): >>>> >>>> i.e. nothing is captured by default, when stdout or stderr are captured >>>> they are returned as bytes by default, when stdout or stderr are >>>> returned as text they are decoded using the locale encoding by default, >>>> or the encoding can be explicitly specified. >>>> >>>> Do you think this is feasible? >>> >>> Feasible, yes. >>> In the majority of cases where the output is needed, we want text >>> encoded with locale.getpreferredencoding(), so I'd rather make that the >>> default rather than bytes. >>> >>> Or we could make "encode_stdout" imply "capture_stdout", so you wouldn't >>> have to always specify both. (It would be better named "encoded_stdout" >>> in that case.) >> >> I think it should be OK to decode by default. (IMO it would be best to >> reverse the logic and name it "raw_stdout", with False default. Do we >> need "raw_stderr" at all? I don't think we do.) > > Actually, now that I think about it: if the result was a namedtuple > subclass, we can always set extra raw_stdout/raw_stderr attributes on > it. (Unless skip_output=True, of course.) > > result = run(['generate_binary_data']) > use(result.raw_stdout) > > # and for backcompat, > rc, _out, _err = result Good idea. Perhaps we should use the same names as CalledProcessError for the attributes, i.e. "returncode", "output", and "error_output", "raw_output", "raw_error_output" for the new attributes? -- Jan Cholasta From lslebodn at redhat.com Fri Dec 4 11:19:22 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Fri, 4 Dec 2015 12:19:22 +0100 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <566058E2.6040201@redhat.com> References: <565F34FD.5090204@redhat.com> <20151203090954.GB5209@mail.corp.redhat.com> <566058E2.6040201@redhat.com> Message-ID: <20151204111921.GE28768@mail.corp.redhat.com> On (03/12/15 09:59), Rob Crittenden wrote: >Lukas Slebodnik wrote: >> On (02/12/15 13:14), Rob Crittenden wrote: >>> Is it still mandatory that tests pass the unit tests before acceptance? >> Unit test could be executed as part of "%check" phase in spec files. >> I recently added C-base unit tests there. >> >> I was not bale to run "make tests" there because many tests failed. >> If they require real IPA server for execution ( or lite server) >> then they are not unit test but integration tests. >> One solution would be to skip them or to usw cwrap[1] + lite server. >> So it can be run also in mock/koji which has many restrictions. > >It would represent quite a lot of work but it may be a good idea to >investigate. Ipsilon uses cwrap for its tests so some of the >configuration can be gleaned from that. > >I would definitely be opposed to this as part of the freeipa.spec in the ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ What do you mean by this part? Did you mean "running make tests" in spec file? If yes, could you elaborate why it's not good idea? many projects run tests in "%check" sssd, certmonger, glibc ... Currently only C-based test are executed. And I added it only recently. >git tree. In Fedora it might help find problems when rawhide becomes >Fedora.next so it would provide some value there. > >> >> Also lint should be also part of "%check" phase and not part of >> ordinary build. >> BTW I could not see a lint[2] in fedora build at all. So I'm not sure >> if it is executed with upstream spec file. > >It isn't there because the expectation is that lint already passes as >part of the release process. I don't see the value on running lint on >release tarballs. > That's just an expectation. It needn't be true. Your initial mail was about stricter review process. And automating things is best way how to enforce it. So reviewer would just build rpms and if "%check" phase will not pass then he will not continue with review. If it will be part of "%check" then you can use mock and easily ensure that test passes on stable fedora and fedora rawhide (and maybe centOS) >I also want to keep the focus on the reviewer's responsibility to ensure >that patches do what they are supposed to and don't break things. > yes, but it does not mean that we cannot simplify/automate reviewer's tasks. IMHO, as much as possible should be done in spec file; "%check". LS From pviktori at redhat.com Fri Dec 4 11:43:04 2015 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 4 Dec 2015 12:43:04 +0100 Subject: [Freeipa-devel] [PATCH] 0748 Handle encoding for ipautil.run In-Reply-To: <566175D0.1000105@redhat.com> References: <5638B8F9.6010703@redhat.com> <564D0FD9.4070505@redhat.com> <5652B5AC.2040508@redhat.com> <5652E174.50301@redhat.com> <56548E73.1090105@redhat.com> <565587AE.2070303@redhat.com> <5655CA08.8090701@redhat.com> <565C01FD.7050906@redhat.com> <565D83CC.8020802@redhat.com> <565EA4FA.3020307@redhat.com> <565EF7B9.6020507@redhat.com> <566011A5.4040200@redhat.com> <566054D7.9020008@redhat.com> <566145FE.3090005@redhat.com> <566162B2.9010809@redhat.com> <566175D0.1000105@redhat.com> Message-ID: <56617C48.4030005@redhat.com> On 12/04/2015 12:15 PM, Jan Cholasta wrote: > On 4.12.2015 10:53, Petr Viktorin wrote: >> On 12/04/2015 08:51 AM, Jan Cholasta wrote: >>> On 3.12.2015 15:42, Petr Viktorin wrote: >>>> On 12/03/2015 10:55 AM, Jan Cholasta wrote: [...] >>>>> If we do encode/decode in run(), I think there should be a way to >>>>> override the default encoding. My idea was to either accept/return >>>>> only >>>>> bytes and let the caller handle encoding themselves, or to handle >>>>> encoding in run(), but be able to override the defaults. >>>>> >>>>> Given we handle encoding in run(), I would imagine it like this: >>>>> >>>>> def run(args, stdin=None, raiseonerr=True, nolog=(), env=None, >>>>> capture_stdout=False, skip_output=False, cwd=None, >>>>> runas=None, timeout=None, suplementary_groups=[], >>>>> capture_stderr=False, encode_stdout=False, >>>>> encode_stderr=False, encoding=None): >>>>> >>>>> i.e. nothing is captured by default, when stdout or stderr are >>>>> captured >>>>> they are returned as bytes by default, when stdout or stderr are >>>>> returned as text they are decoded using the locale encoding by >>>>> default, >>>>> or the encoding can be explicitly specified. >>>>> >>>>> Do you think this is feasible? >>>> >>>> Feasible, yes. >>>> In the majority of cases where the output is needed, we want text >>>> encoded with locale.getpreferredencoding(), so I'd rather make that the >>>> default rather than bytes. >>>> >>>> Or we could make "encode_stdout" imply "capture_stdout", so you >>>> wouldn't >>>> have to always specify both. (It would be better named "encoded_stdout" >>>> in that case.) >>> >>> I think it should be OK to decode by default. (IMO it would be best to >>> reverse the logic and name it "raw_stdout", with False default. Do we >>> need "raw_stderr" at all? I don't think we do.) >> >> Actually, now that I think about it: if the result was a namedtuple >> subclass, we can always set extra raw_stdout/raw_stderr attributes on >> it. (Unless skip_output=True, of course.) >> >> result = run(['generate_binary_data']) >> use(result.raw_stdout) >> >> # and for backcompat, >> rc, _out, _err = result > > Good idea. > > Perhaps we should use the same names as CalledProcessError for the > attributes, i.e. "returncode", "output", and "error_output", > "raw_output", "raw_error_output" for the new attributes? OK. It's also good to use different names than Popen's stdout/stderr, which are file-like objects rather than strings. But then, capture_stdout/capture_stderr should be capture_output/capture_error_output. -- Petr Viktorin From jcholast at redhat.com Fri Dec 4 11:49:15 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 4 Dec 2015 12:49:15 +0100 Subject: [Freeipa-devel] [PATCH] 0748 Handle encoding for ipautil.run In-Reply-To: <56617C48.4030005@redhat.com> References: <5638B8F9.6010703@redhat.com> <564D0FD9.4070505@redhat.com> <5652B5AC.2040508@redhat.com> <5652E174.50301@redhat.com> <56548E73.1090105@redhat.com> <565587AE.2070303@redhat.com> <5655CA08.8090701@redhat.com> <565C01FD.7050906@redhat.com> <565D83CC.8020802@redhat.com> <565EA4FA.3020307@redhat.com> <565EF7B9.6020507@redhat.com> <566011A5.4040200@redhat.com> <566054D7.9020008@redhat.com> <566145FE.3090005@redhat.com> <566162B2.9010809@redhat.com> <566175D0.1000105@redhat.com> <56617C48.4030005@redhat.com> Message-ID: <56617DBB.9000109@redhat.com> On 4.12.2015 12:43, Petr Viktorin wrote: > On 12/04/2015 12:15 PM, Jan Cholasta wrote: >> On 4.12.2015 10:53, Petr Viktorin wrote: >>> On 12/04/2015 08:51 AM, Jan Cholasta wrote: >>>> On 3.12.2015 15:42, Petr Viktorin wrote: >>>>> On 12/03/2015 10:55 AM, Jan Cholasta wrote: > [...] >>>>>> If we do encode/decode in run(), I think there should be a way to >>>>>> override the default encoding. My idea was to either accept/return >>>>>> only >>>>>> bytes and let the caller handle encoding themselves, or to handle >>>>>> encoding in run(), but be able to override the defaults. >>>>>> >>>>>> Given we handle encoding in run(), I would imagine it like this: >>>>>> >>>>>> def run(args, stdin=None, raiseonerr=True, nolog=(), env=None, >>>>>> capture_stdout=False, skip_output=False, cwd=None, >>>>>> runas=None, timeout=None, suplementary_groups=[], >>>>>> capture_stderr=False, encode_stdout=False, >>>>>> encode_stderr=False, encoding=None): >>>>>> >>>>>> i.e. nothing is captured by default, when stdout or stderr are >>>>>> captured >>>>>> they are returned as bytes by default, when stdout or stderr are >>>>>> returned as text they are decoded using the locale encoding by >>>>>> default, >>>>>> or the encoding can be explicitly specified. >>>>>> >>>>>> Do you think this is feasible? >>>>> >>>>> Feasible, yes. >>>>> In the majority of cases where the output is needed, we want text >>>>> encoded with locale.getpreferredencoding(), so I'd rather make that the >>>>> default rather than bytes. >>>>> >>>>> Or we could make "encode_stdout" imply "capture_stdout", so you >>>>> wouldn't >>>>> have to always specify both. (It would be better named "encoded_stdout" >>>>> in that case.) >>>> >>>> I think it should be OK to decode by default. (IMO it would be best to >>>> reverse the logic and name it "raw_stdout", with False default. Do we >>>> need "raw_stderr" at all? I don't think we do.) >>> >>> Actually, now that I think about it: if the result was a namedtuple >>> subclass, we can always set extra raw_stdout/raw_stderr attributes on >>> it. (Unless skip_output=True, of course.) >>> >>> result = run(['generate_binary_data']) >>> use(result.raw_stdout) >>> >>> # and for backcompat, >>> rc, _out, _err = result >> >> Good idea. >> >> Perhaps we should use the same names as CalledProcessError for the >> attributes, i.e. "returncode", "output", and "error_output", >> "raw_output", "raw_error_output" for the new attributes? (Actually, since "returncode" is not "return_code", it should probably be "returncode", "output", "erroroutput", "raw_output", "raw_erroroutput".) > > OK. It's also good to use different names than Popen's stdout/stderr, > which are file-like objects rather than strings. But then, > capture_stdout/capture_stderr should be capture_output/capture_error_output. Works for me. -- Jan Cholasta From mbasti at redhat.com Fri Dec 4 12:00:50 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 4 Dec 2015 13:00:50 +0100 Subject: [Freeipa-devel] [PATCH] 938 rename topology suffixes to "domain" and "ca" In-Reply-To: <565DE06A.9010602@redhat.com> References: <565C35ED.6090701@redhat.com> <565D54A1.1050509@redhat.com> <565DE06A.9010602@redhat.com> Message-ID: <56618072.8050305@redhat.com> On 01.12.2015 19:01, Petr Vobornik wrote: > On 12/01/2015 09:04 AM, Jan Cholasta wrote: >> On 30.11.2015 12:41, Petr Vobornik wrote: >>> see >>> https://www.redhat.com/archives/freeipa-devel/2015-November/msg00485.html >>> >> >> LGTM, but I would s/_SUFFIX/_SUFFIX_NAME/. >> > > Updated patch attached. > > > ACK Pushed to master: 517aa84569ae144a8b781ef7ad67e356d9021757 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbabinsk at redhat.com Fri Dec 4 13:23:20 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Fri, 4 Dec 2015 14:23:20 +0100 Subject: [Freeipa-devel] [PATCH 558] Allow disabling requireing preauth by default for Service Principal Names In-Reply-To: <1449004137.9040.18.camel@redhat.com> References: <1448400058.29102.17.camel@redhat.com> <56558064.7010600@redhat.com> <1448908922.3747.94.camel@redhat.com> <565DB5E1.8040500@redhat.com> <1449004137.9040.18.camel@redhat.com> Message-ID: <566193C8.9050609@redhat.com> On 12/01/2015 10:08 PM, Simo Sorce wrote: > On Tue, 2015-12-01 at 15:59 +0100, Martin Babinsky wrote: >> On 11/30/2015 07:42 PM, Simo Sorce wrote: >>> On Wed, 2015-11-25 at 10:33 +0100, Martin Babinsky wrote: >>>> On 11/24/2015 10:20 PM, Simo Sorce wrote: >>>>> This addresses #3860, giving admins the option to not require preauth >>>>> for Hosts and services. >>>>> >>>>> I did not add this option by default, although it does reduce the load >>>>> on the KDC as well as speed up TGT acquisition for service principal >>>>> accounts that acquire TGTs. >>>>> >>>>> Tested and working as expected (SPNs are not returned PREAUTH_NEEDED >>>>> error while normal users are). >>>>> >>>>> HTH, >>>>> Simo. >>>>> >>>>> >>>>> >>>> Hi Simo, >>>> >>>> I was not able to apply the patch on current master branch: >>>> >>>> """ >>>> git am >>>> ../review/ssorce/3860/freeipa-simo-558-1-Allow-admins-to-disable-preauth-for-SPNs.patch >>>> -3 >>>> >>>> Applying: Allow admins to disable preauth for SPNs. >>>> error: invalid object 100644 a6b4d4349a9ac6de453d9ad3c679ec32add4e43b >>>> for 'ipalib/plugins/config.py' >>>> fatal: git-write-tree: error building trees >>>> Repository lacks necessary blobs to fall back on 3-way merge. >>>> Cannot fall back to three-way merge. >>>> Patch failed at 0001 Allow admins to disable preauth for SPNs. >>>> """ >>>> >>>> It seems that I nedd to apply some of your other patches first (which one?) >>> >>> Sorry did not see this question earlier, it requires 556 and 557, I just >>> bumped that thread. >>> >>> Simo. >>> >> It seems that I need something else, patch 556-2 applies cleanly, but >> patch 557-3 fails with http://fpaste.org/296230/89819431/ on both master >> and 4-2 branch. >> > > Rebased 556,557 in their thread, and here is the rebase for 558 on top > of them. > > Simo. > ACK. I'm afraid that this patch and 556, 557 will require another round of rebase before pushing, though. -- Martin^3 Babinsky From jcholast at redhat.com Fri Dec 4 13:29:25 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 4 Dec 2015 14:29:25 +0100 Subject: [Freeipa-devel] [PATCH] 0751 spec: Split out python-ipap11helper and, python-default_encoding_utf8 In-Reply-To: <56606E90.3000909@redhat.com> References: <56606E90.3000909@redhat.com> Message-ID: <56619535.1010700@redhat.com> Hi, On 3.12.2015 17:32, Petr Viktorin wrote: > Hello, > This specfile patch makes python-ipalib noarch, by splitting out the > arch-dependent stuff. I'm not sure if this should be done at all. Both py_default_encoding and (especially) _ipap11helper are internal submodules of ipapython and I don't think they should be packaged separately. If this is just about packaging arch-specific code separately from noarch code, I'm not sure either - all other packages with both Python and C modules I have seen (e.g. python-ldap, PyYAML) simply do not care and put everything in a single arch-specific package. IMHO we should follow. > > It depends on Honza Cholasta's patches 516 and 517. > > Part of https://fedorahosted.org/freeipa/ticket/3197 > Honza -- Jan Cholasta From rcritten at redhat.com Fri Dec 4 13:42:25 2015 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 4 Dec 2015 08:42:25 -0500 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <20151204111921.GE28768@mail.corp.redhat.com> References: <565F34FD.5090204@redhat.com> <20151203090954.GB5209@mail.corp.redhat.com> <566058E2.6040201@redhat.com> <20151204111921.GE28768@mail.corp.redhat.com> Message-ID: <56619841.3010809@redhat.com> Lukas Slebodnik wrote: > On (03/12/15 09:59), Rob Crittenden wrote: >> Lukas Slebodnik wrote: >>> On (02/12/15 13:14), Rob Crittenden wrote: >>>> Is it still mandatory that tests pass the unit tests before acceptance? >>> Unit test could be executed as part of "%check" phase in spec files. >>> I recently added C-base unit tests there. >>> >>> I was not bale to run "make tests" there because many tests failed. >>> If they require real IPA server for execution ( or lite server) >>> then they are not unit test but integration tests. >>> One solution would be to skip them or to usw cwrap[1] + lite server. >>> So it can be run also in mock/koji which has many restrictions. >> >> It would represent quite a lot of work but it may be a good idea to >> investigate. Ipsilon uses cwrap for its tests so some of the >> configuration can be gleaned from that. >> >> I would definitely be opposed to this as part of the freeipa.spec in the > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > What do you mean by this part? > > Did you mean "running make tests" in spec file? > If yes, could you elaborate why it's not good idea? > many projects run tests in "%check" > sssd, certmonger, glibc ... > > Currently only C-based test are executed. And I added it only recently. Because it would be overkill during development. The expectation is that developers and reviewers run the tests before submission/acceptance. If they fail to do that then it will be obvious. >> git tree. In Fedora it might help find problems when rawhide becomes >> Fedora.next so it would provide some value there. >> >>> >>> Also lint should be also part of "%check" phase and not part of >>> ordinary build. >>> BTW I could not see a lint[2] in fedora build at all. So I'm not sure >>> if it is executed with upstream spec file. >> >> It isn't there because the expectation is that lint already passes as >> part of the release process. I don't see the value on running lint on >> release tarballs. >> > That's just an expectation. It needn't be true. Your initial mail was about > stricter review process. And automating things is best way how to > enforce it. So reviewer would just build rpms and if "%check" phase > will not pass then he will not continue with review. > If it will be part of "%check" then you can use mock and easily ensure > that test passes on stable fedora and fedora rawhide (and maybe centOS) By the time downstream gets a tarball it is too late to fix lint errors. If upstream is doing a release with lint errors then there is something deeply wrong with the release process. If someone wants to add it to the downstream spec files I'm not going to complain, I just find it extremely unlikely that it will provide any value, ever. I guess the question to ask is: since implementing lint in the dev process has a release ever gone out where things failed that would have been caught? I'm pretty sure the answer is no. >> I also want to keep the focus on the reviewer's responsibility to ensure >> that patches do what they are supposed to and don't break things. >> > yes, but it does not mean that we cannot simplify/automate reviewer's tasks. > IMHO, as much as possible should be done in spec file; "%check". It would be grossly inefficient to run the tests with every make rpms. Are you at all familiar with the IPA build and install process? It is usually build here and test over there over and over again. Running the tests with each iterative build would be horrible. It is sometimes bad enough waiting for the lint to finish. If someone wants to setup a Jenkins service to automate testing then fine. It just needs to be done in some way, and it clearly isn't on a regular basis and it is easily solved without inventing a whole new CI. rob From jcholast at redhat.com Fri Dec 4 14:12:44 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 4 Dec 2015 15:12:44 +0100 Subject: [Freeipa-devel] [PATCHES 509-514] replica promotion: use host credentials when setting up replication In-Reply-To: <566167DE.9090204@redhat.com> References: <564DE017.4020408@redhat.com> <1447951434.5132.27.camel@redhat.com> <564F4105.1040204@redhat.com> <1448038723.5132.59.camel@redhat.com> <5652C63E.5090206@redhat.com> <1448289289.7892.13.camel@redhat.com> <565324C4.1090007@redhat.com> <1448290024.7892.15.camel@redhat.com> <565D823E.5090600@redhat.com> <56604D94.7000705@redhat.com> <566167DE.9090204@redhat.com> Message-ID: <56619F5C.5000900@redhat.com> On 4.12.2015 11:15, Petr Vobornik wrote: > On 12/03/2015 03:11 PM, Martin Basti wrote: >> >> >> On 01.12.2015 12:19, Jan Cholasta wrote: >>> On 23.11.2015 15:47, Simo Sorce wrote: >>>> On Mon, 2015-11-23 at 15:37 +0100, Jan Cholasta wrote: >>>>> >>>>> Ad alternative is to add the host to ipaservers before the checks are >>>>> done and remove it again if any of them fail. >>>> >>>> Too error prone, I am ok with the current way in your patches >>>> until/unless I can think of a fail safe way. :-) >>> >>> Updated patches attached. Note that 520 should be applied between 509 >>> and 510. >>> >>> >>> >> Functional ACK >> > > Simo, do you want to review the ACIs or other things it the patches? Or > can the patches be pushed? There were no changes in the ACIs since last time. Rebased patches attached. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-509.2-aci-add-IPA-servers-host-group-ipaservers.patch Type: text/x-patch Size: 9835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-520.1-aci-replace-per-server-ACIs-with-ipaserver-based-ACI.patch Type: text/x-patch Size: 11571 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-510.2-aci-allow-members-of-ipaservers-to-set-up-replicatio.patch Type: text/x-patch Size: 5563 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-511.2-ipautil-use-file-in-a-temporary-dir-as-ccache-in-pri.patch Type: text/x-patch Size: 1392 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-512.2-replica-promotion-use-host-credentials-when-setting-.patch Type: text/x-patch Size: 4922 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-514.2-replica-promotion-automatically-add-the-local-host-t.patch Type: text/x-patch Size: 3332 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-521.1-custodia-do-not-modify-memberPrincipal-on-key-update.patch Type: text/x-patch Size: 970 bytes Desc: not available URL: From jcholast at redhat.com Fri Dec 4 14:16:46 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 4 Dec 2015 15:16:46 +0100 Subject: [Freeipa-devel] [PATCHES 509-514] replica promotion: use host credentials when setting up replication In-Reply-To: <56619F5C.5000900@redhat.com> References: <564DE017.4020408@redhat.com> <1447951434.5132.27.camel@redhat.com> <564F4105.1040204@redhat.com> <1448038723.5132.59.camel@redhat.com> <5652C63E.5090206@redhat.com> <1448289289.7892.13.camel@redhat.com> <565324C4.1090007@redhat.com> <1448290024.7892.15.camel@redhat.com> <565D823E.5090600@redhat.com> <56604D94.7000705@redhat.com> <566167DE.9090204@redhat.com> <56619F5C.5000900@redhat.com> Message-ID: <5661A04E.9080409@redhat.com> On 4.12.2015 15:12, Jan Cholasta wrote: > On 4.12.2015 11:15, Petr Vobornik wrote: >> On 12/03/2015 03:11 PM, Martin Basti wrote: >>> >>> >>> On 01.12.2015 12:19, Jan Cholasta wrote: >>>> On 23.11.2015 15:47, Simo Sorce wrote: >>>>> On Mon, 2015-11-23 at 15:37 +0100, Jan Cholasta wrote: >>>>>> >>>>>> Ad alternative is to add the host to ipaservers before the checks are >>>>>> done and remove it again if any of them fail. >>>>> >>>>> Too error prone, I am ok with the current way in your patches >>>>> until/unless I can think of a fail safe way. :-) >>>> >>>> Updated patches attached. Note that 520 should be applied between 509 >>>> and 510. >>>> >>>> >>>> >>> Functional ACK >>> >> >> Simo, do you want to review the ACIs or other things it the patches? Or >> can the patches be pushed? > > There were no changes in the ACIs since last time. Actually, memberPrincipal was removed from the "IPA server hosts can manage own Custodia secrets" ACI, as per Simo's request. > > Rebased patches attached. -- Jan Cholasta From jcholast at redhat.com Fri Dec 4 14:39:15 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 4 Dec 2015 15:39:15 +0100 Subject: [Freeipa-devel] [PATCHES 509-514] replica promotion: use host credentials when setting up replication In-Reply-To: <5661A04E.9080409@redhat.com> References: <564DE017.4020408@redhat.com> <1447951434.5132.27.camel@redhat.com> <564F4105.1040204@redhat.com> <1448038723.5132.59.camel@redhat.com> <5652C63E.5090206@redhat.com> <1448289289.7892.13.camel@redhat.com> <565324C4.1090007@redhat.com> <1448290024.7892.15.camel@redhat.com> <565D823E.5090600@redhat.com> <56604D94.7000705@redhat.com> <566167DE.9090204@redhat.com> <56619F5C.5000900@redhat.com> <5661A04E.9080409@redhat.com> Message-ID: <5661A593.5030003@redhat.com> On 4.12.2015 15:16, Jan Cholasta wrote: > On 4.12.2015 15:12, Jan Cholasta wrote: >> On 4.12.2015 11:15, Petr Vobornik wrote: >>> On 12/03/2015 03:11 PM, Martin Basti wrote: >>>> >>>> >>>> On 01.12.2015 12:19, Jan Cholasta wrote: >>>>> On 23.11.2015 15:47, Simo Sorce wrote: >>>>>> On Mon, 2015-11-23 at 15:37 +0100, Jan Cholasta wrote: >>>>>>> >>>>>>> Ad alternative is to add the host to ipaservers before the checks >>>>>>> are >>>>>>> done and remove it again if any of them fail. >>>>>> >>>>>> Too error prone, I am ok with the current way in your patches >>>>>> until/unless I can think of a fail safe way. :-) >>>>> >>>>> Updated patches attached. Note that 520 should be applied between 509 >>>>> and 510. >>>>> >>>>> >>>>> >>>> Functional ACK >>>> >>> >>> Simo, do you want to review the ACIs or other things it the patches? Or >>> can the patches be pushed? >> >> There were no changes in the ACIs since last time. > > Actually, memberPrincipal was removed from the "IPA server hosts can > manage own Custodia secrets" ACI, as per Simo's request. > >> >> Rebased patches attached. Note that 520 should still be applied between 509 and 510. -- Jan Cholasta From pvoborni at redhat.com Fri Dec 4 15:15:30 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 4 Dec 2015 16:15:30 +0100 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <565F34FD.5090204@redhat.com> References: <565F34FD.5090204@redhat.com> Message-ID: <5661AE12.6060007@redhat.com> On 12/02/2015 07:14 PM, Rob Crittenden wrote: > Is it still mandatory that tests pass the unit tests before acceptance? > I've seen a number of cases over the past couple of months where a > change goes through then shortly afterward a patch to fix the tests. > IMHO this should be caught in advance. > > Things slip through and goodness knows I've acked more than a few > patches without running the full suite. I just have a feeling it has > become more frequent lately. > > rob > At 4.2 retrospective a review check list was discussed. I have a draft [1]. Comments welcome! I'm sorry, that it's only pdf at the moment. Maybe sanity checks should be less verbose, but I wanted to have it spelled out. My goal is to have both wiki page and a printable check list which can lie on a table. [1] https://pvoborni.fedorapeople.org/FreeIPAdeveloperschecklist.pdf -- Petr Vobornik From fskola at redhat.com Fri Dec 4 15:24:16 2015 From: fskola at redhat.com (Filip =?UTF-8?B?xaBrb2xh?=) Date: Fri, 4 Dec 2015 16:24:16 +0100 Subject: [Freeipa-devel] [PATCH 0004] Refactor test_attr Message-ID: <20151204162416.28170b8e@vor2.netbox.priv> Hi, sending a new version of test_attr. F. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-fskola-0004-Refactor-test_attr.patch Type: text/x-patch Size: 26883 bytes Desc: not available URL: From simo at redhat.com Fri Dec 4 15:58:34 2015 From: simo at redhat.com (Simo Sorce) Date: Fri, 04 Dec 2015 10:58:34 -0500 Subject: [Freeipa-devel] [PATCHES 509-514] replica promotion: use host credentials when setting up replication In-Reply-To: <5661A593.5030003@redhat.com> References: <564DE017.4020408@redhat.com> <1447951434.5132.27.camel@redhat.com> <564F4105.1040204@redhat.com> <1448038723.5132.59.camel@redhat.com> <5652C63E.5090206@redhat.com> <1448289289.7892.13.camel@redhat.com> <565324C4.1090007@redhat.com> <1448290024.7892.15.camel@redhat.com> <565D823E.5090600@redhat.com> <56604D94.7000705@redhat.com> <566167DE.9090204@redhat.com> <56619F5C.5000900@redhat.com> <5661A04E.9080409@redhat.com> <5661A593.5030003@redhat.com> Message-ID: <1449244714.3445.67.camel@redhat.com> On Fri, 2015-12-04 at 15:39 +0100, Jan Cholasta wrote: > On 4.12.2015 15:16, Jan Cholasta wrote: > > On 4.12.2015 15:12, Jan Cholasta wrote: > >> On 4.12.2015 11:15, Petr Vobornik wrote: > >>> On 12/03/2015 03:11 PM, Martin Basti wrote: > >>>> > >>>> > >>>> On 01.12.2015 12:19, Jan Cholasta wrote: > >>>>> On 23.11.2015 15:47, Simo Sorce wrote: > >>>>>> On Mon, 2015-11-23 at 15:37 +0100, Jan Cholasta wrote: > >>>>>>> > >>>>>>> Ad alternative is to add the host to ipaservers before the checks > >>>>>>> are > >>>>>>> done and remove it again if any of them fail. > >>>>>> > >>>>>> Too error prone, I am ok with the current way in your patches > >>>>>> until/unless I can think of a fail safe way. :-) > >>>>> > >>>>> Updated patches attached. Note that 520 should be applied between 509 > >>>>> and 510. > >>>>> > >>>>> > >>>>> > >>>> Functional ACK > >>>> > >>> > >>> Simo, do you want to review the ACIs or other things it the patches? Or > >>> can the patches be pushed? > >> > >> There were no changes in the ACIs since last time. > > > > Actually, memberPrincipal was removed from the "IPA server hosts can > > manage own Custodia secrets" ACI, as per Simo's request. > > > >> > >> Rebased patches attached. > > Note that 520 should still be applied between 509 and 510. > LGTM -- Simo Sorce * Red Hat, Inc * New York From pvoborni at redhat.com Fri Dec 4 16:12:57 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 4 Dec 2015 17:12:57 +0100 Subject: [Freeipa-devel] [PATCH 0096] check whether replica exists before executing the domain level 1 deletion code In-Reply-To: <565D57C7.40105@redhat.com> References: <564C6E3E.5040606@redhat.com> <565D57C7.40105@redhat.com> Message-ID: <5661BB89.9010504@redhat.com> On 12/01/2015 09:18 AM, Martin Basti wrote: > > > On 18.11.2015 13:25, Martin Babinsky wrote: >> Additional fix for https://fedorahosted.org/freeipa/ticket/5424 >> >> In current implementation the topology suffices are checked first and >> after that the error about non-existent host is raised. This does not >> make much sense to me, we should check for host existence before any >> work is done. >> >> >> > ACK > Any reason to use host-show and not server-show? host-show will be successful on any host, server-show only if the host is an IPA server. -- Petr Vobornik From mbabinsk at redhat.com Fri Dec 4 16:23:57 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Fri, 4 Dec 2015 17:23:57 +0100 Subject: [Freeipa-devel] [PATCH 0096] check whether replica exists before executing the domain level 1 deletion code In-Reply-To: <5661BB89.9010504@redhat.com> References: <564C6E3E.5040606@redhat.com> <565D57C7.40105@redhat.com> <5661BB89.9010504@redhat.com> Message-ID: <5661BE1D.3080707@redhat.com> On 12/04/2015 05:12 PM, Petr Vobornik wrote: > On 12/01/2015 09:18 AM, Martin Basti wrote: >> >> >> On 18.11.2015 13:25, Martin Babinsky wrote: >>> Additional fix for https://fedorahosted.org/freeipa/ticket/5424 >>> >>> In current implementation the topology suffices are checked first and >>> after that the error about non-existent host is raised. This does not >>> make much sense to me, we should check for host existence before any >>> work is done. >>> >>> >>> >> ACK >> > > Any reason to use host-show and not server-show? > > host-show will be successful on any host, server-show only if the host > is an IPA server. Good catch, I have fixed it in this patch. I have also modified the error message slightly. -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0096.1-check-whether-replica-exists-before-executing-the-do.patch Type: text/x-patch Size: 2141 bytes Desc: not available URL: From tbabej at redhat.com Fri Dec 4 17:41:08 2015 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 4 Dec 2015 18:41:08 +0100 Subject: [Freeipa-devel] [PATCH 0390] man: Update the ipa-replica-install manpage with promotion In-Reply-To: <56617434.6000402@redhat.com> References: <56602223.6020906@redhat.com> <56602D59.2050207@redhat.com> <56617434.6000402@redhat.com> Message-ID: <5661D034.30104@redhat.com> On 12/04/2015 12:08 PM, Petr Vobornik wrote: > On 12/03/2015 12:54 PM, Petr Vobornik wrote: >> On 12/03/2015 12:06 PM, Tomas Babej wrote: >>> Hi, >>> >>> this patch updates the man page for the ipa-replica-install given the >>> latest changes (including the Jan's OTP patch). >>> >>> Tomas >>> >> >> >> Questions/suggestions: >> >> 1. "you cannot provide an replica file" >> >> It's true, but wouldn't it be better to say that "you don't need to". Or >> communicate that it is a good thing. Cannot sounds too negative to me. >> >> 2. Why options: --password, --admin-password are in 'BASIC OPTIONS' >> section and not also in new 'ENROLLMENT OPTIONS' section together with >> --principal, --keytab, etc..? Maybe 'ENROLLMENT OPTIONS' should be >> before "BASIC OPTIONS" and "BASIC OPTIONS" renamed. >> >> >> General ideas, not related to this patch: >> """ >> If the installation fails you may need to run ipa-server-install >> --uninstall and ipa-client-install before running ipa-replica-install >> again. >> >> The installation will fail if the host you are installing the >> replica on exists as a host in IPA or an existing replication agreement >> exists (for example, from a previously failed installation) >> """ >> >> I believe validators check this situation, right? So do we need to write >> it in the man page. > > > Attaching updated patch which reflects off-line cooperation with Tomas. > > New option sections were added: > DOMAIN LEVEL 1 OPTIONS > DOMAIN LEVEL 1 CLIENT ENROLLMENT OPTIONS > DOMAIN LEVEL 0 OPTIONS ACK, Pushed to master: bb7934e3bc91155f21b45b5d5186bfda76a21f15. From tbabej at redhat.com Fri Dec 4 17:49:37 2015 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 4 Dec 2015 18:49:37 +0100 Subject: [Freeipa-devel] [PATCH] 940 Update ipa-(cs)replica-manage man pages In-Reply-To: <56617474.9020200@redhat.com> References: <56606691.2090402@redhat.com> <56617474.9020200@redhat.com> Message-ID: <5661D231.5030804@redhat.com> On 12/04/2015 12:09 PM, Petr Vobornik wrote: > On 12/03/2015 04:58 PM, Petr Vobornik wrote: >> SSIA >> > > Updated patch attached. > > ACK, Pushed to master: 95d659b634b2ea13d18d26cacbd73e19972145f2 From tbabej at redhat.com Fri Dec 4 17:58:20 2015 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 4 Dec 2015 18:58:20 +0100 Subject: [Freeipa-devel] [PATCH] 941 Extend topology help In-Reply-To: <566066AE.2090702@redhat.com> References: <566066AE.2090702@redhat.com> Message-ID: <5661D43C.2030506@redhat.com> On 12/03/2015 04:58 PM, Petr Vobornik wrote: > `ipa help topology` is improved. > > Looks good. I changed one part of the documentation for more clarity, see the attached patch. Otherwise ACK from me. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0941.1-Extend-topology-help.patch Type: text/x-patch Size: 2829 bytes Desc: not available URL: From tbabej at redhat.com Fri Dec 4 18:04:29 2015 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 4 Dec 2015 19:04:29 +0100 Subject: [Freeipa-devel] [PATCH] 941 Extend topology help In-Reply-To: <5661D43C.2030506@redhat.com> References: <566066AE.2090702@redhat.com> <5661D43C.2030506@redhat.com> Message-ID: <5661D5AD.2060109@redhat.com> On 12/04/2015 06:58 PM, Tomas Babej wrote: > > > On 12/03/2015 04:58 PM, Petr Vobornik wrote: >> `ipa help topology` is improved. >> >> > > Looks good. I changed one part of the documentation for more clarity, > see the attached patch. > > Otherwise ACK from me. > > Tomas > > > Fixed a typo. s/the/that Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0941.2-Extend-topology-help.patch Type: text/x-patch Size: 2830 bytes Desc: not available URL: From tbabej at redhat.com Fri Dec 4 18:17:04 2015 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 4 Dec 2015 19:17:04 +0100 Subject: [Freeipa-devel] [PATCH 0391] replicainstall: Add check for domain if server is specified Message-ID: <5661D8A0.7030303@redhat.com> Hi, Avoids failing in the later stages during the ipa-client-install command. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0391-replicainstall-Add-check-for-domain-if-server-is-spe.patch Type: text/x-patch Size: 1352 bytes Desc: not available URL: From mbasti at redhat.com Fri Dec 4 18:38:24 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 4 Dec 2015 19:38:24 +0100 Subject: [Freeipa-devel] [PATCH 0108] replica install: improvements in the handling of CA-related IPA config entries In-Reply-To: <5660683D.5000803@redhat.com> References: <5660683D.5000803@redhat.com> Message-ID: <5661DDA0.8010900@redhat.com> On 03.12.2015 17:05, Martin Babinsky wrote: > This patch fixes https://fedorahosted.org/freeipa/ticket/5506 and also > reorganizes the way CA installer updates IPA default.conf. > > Maybe a simpler patch would suffice, but I had a need to improve > things a bit. > > This one is for master branch only. IIRC the situation described in > #5506 does not occur in domain level 0, however the root cause > (incorrect forwarding of certmonger requests by CA-less replicas) > manifests when enrolling a client against CA-less replica and > requesting host certificate. > > Should I open a separate ticket for that? > > > ACK Pushed to master: a497288b3eafe00ab9c819dd4a51d0b421824b36 -------------- next part -------------- An HTML attachment was scrubbed... URL: From pvoborni at redhat.com Fri Dec 4 18:53:13 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 4 Dec 2015 19:53:13 +0100 Subject: [Freeipa-devel] [PATCH] 941 Extend topology help In-Reply-To: <5661D5AD.2060109@redhat.com> References: <566066AE.2090702@redhat.com> <5661D43C.2030506@redhat.com> <5661D5AD.2060109@redhat.com> Message-ID: <5661E119.2030709@redhat.com> On 12/04/2015 07:04 PM, Tomas Babej wrote: > > > On 12/04/2015 06:58 PM, Tomas Babej wrote: >> >> >> On 12/03/2015 04:58 PM, Petr Vobornik wrote: >>> `ipa help topology` is improved. >>> >>> >> >> Looks good. I changed one part of the documentation for more clarity, >> see the attached patch. >> >> Otherwise ACK from me. >> >> Tomas >> >> >> > > Fixed a typo. > > s/the/that > > Tomas > > ACK on the change. Pushed to master: 81c06327b955f71cbbd89e5cfa47594157c3fcae -- Petr Vobornik From mbabinsk at redhat.com Fri Dec 4 19:14:10 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Fri, 4 Dec 2015 20:14:10 +0100 Subject: [Freeipa-devel] [PATCH 0109-0110] fix auto-forwarders in standalone DNS installer and update man pages Message-ID: <5661E602.6080104@redhat.com> Additional fixes for https://fedorahosted.org/freeipa/ticket/5438 -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0110-add-auto-forwarders-description-to-server-replica-DN.patch Type: text/x-patch Size: 2726 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0109-add-auto-forwarders-option-to-standalone-DNS-install.patch Type: text/x-patch Size: 1379 bytes Desc: not available URL: From mkosek at redhat.com Fri Dec 4 19:15:45 2015 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 4 Dec 2015 20:15:45 +0100 Subject: [Freeipa-devel] [PATCH 0391] replicainstall: Add check for domain if server is specified In-Reply-To: <5661D8A0.7030303@redhat.com> References: <5661D8A0.7030303@redhat.com> Message-ID: <5661E661.5090601@redhat.com> On 12/04/2015 07:17 PM, Tomas Babej wrote: > Hi, > > Avoids failing in the later stages during the ipa-client-install > command. > > Tomas Is this change needed? Wouldn't it be better to update ipa-client-install or ipa-replica-install to not require the --domain option? I would hope that --domain can be figured out during installation and not passed to ipa-replica-install manually by the admin. I just think that calling # ipa-replica-install --server=master.example.com is better than # ipa-replica-install --server=master.example.com --domain example.com if possible. Martin From mbasti at redhat.com Fri Dec 4 19:17:07 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 4 Dec 2015 20:17:07 +0100 Subject: [Freeipa-devel] [PATCH 0066] Migrate wget references to curl In-Reply-To: References: Message-ID: <5661E6B3.3070103@redhat.com> On 01.12.2015 15:00, Gabe Alford wrote: > Hello, > > Fix for https://fedorahosted.org/freeipa/ticket/5458 > > Thanks, > > Gabe > > Hello, I haven't looked closer, but your patch is causing this: Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes 30 seconds [1/27]: creating certificate server user [2/27]: configuring certificate server instance [3/27]: stopping certificate server instance to update CS.cfg [4/27]: backing up CS.cfg [5/27]: disabling nonces [6/27]: set up CRL publishing [7/27]: enable PKIX certificate path discovery and validation [8/27]: starting certificate server instance ipa.ipaserver.install.cainstance.CAInstance: CRITICAL Failed to restart the Dogtag instance.See the installation log for details. [9/27]: creating RA agent certificate database [10/27]: importing CA chain to RA certificate database [11/27]: fixing RA database permissions [12/27]: setting up signing cert profile [13/27]: setting audit signing renewal to 2 years [14/27]: restarting certificate server ipa.ipaserver.install.cainstance.CAInstance: CRITICAL Failed to restart the Dogtag instance.See the installation log for details. [15/27]: requesting RA certificate from CA [16/27]: issuing RA agent certificate [17/27]: adding RA agent as a trusted user [18/27]: authorizing RA to modify profiles [19/27]: configure certmonger for renewals [20/27]: configure certificate renewals [21/27]: configure RA certificate renewal [22/27]: configure Server-Cert certificate renewal [23/27]: Configure HTTP to proxy connections [24/27]: restarting certificate server ipa.ipaserver.install.cainstance.CAInstance: CRITICAL Failed to restart the Dogtag instance.See the installation log for details. [25/27]: migrating certificate profiles to LDAP [26/27]: importing IPA certificate profiles [27/27]: adding default CA ACL CA is operational and ready, but IPA installer is not able to detect it correctly 2015-12-04T19:08:54Z DEBUG stderr=curl: option --connect-timeout 30: is unknown curl: try 'curl --help' or 'curl --manual' for more information Martin^2 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbasti at redhat.com Fri Dec 4 19:20:54 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 4 Dec 2015 20:20:54 +0100 Subject: [Freeipa-devel] [PATCH 0391] replicainstall: Add check for domain if server is specified In-Reply-To: <5661E661.5090601@redhat.com> References: <5661D8A0.7030303@redhat.com> <5661E661.5090601@redhat.com> Message-ID: <5661E796.6070807@redhat.com> On 04.12.2015 20:15, Martin Kosek wrote: > On 12/04/2015 07:17 PM, Tomas Babej wrote: >> Hi, >> >> Avoids failing in the later stages during the ipa-client-install >> command. >> >> Tomas > > Is this change needed? Wouldn't it be better to update > ipa-client-install or ipa-replica-install to not require the --domain > option? I would hope that --domain can be figured out during > installation and not passed to ipa-replica-install manually by the admin. > > I just think that calling > # ipa-replica-install --server=master.example.com > is better than > # ipa-replica-install --server=master.example.com --domain example.com > if possible. > > Martin > How about hostname 'my.ipa.example.com' and and domain 'example.com' We already have plenty of code that is trying to use a crystal ball to detect what is right, we do not need additional one. Martin^2 From rcritten at redhat.com Fri Dec 4 19:22:56 2015 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 4 Dec 2015 14:22:56 -0500 Subject: [Freeipa-devel] [PATCH 0391] replicainstall: Add check for domain if server is specified In-Reply-To: <5661E661.5090601@redhat.com> References: <5661D8A0.7030303@redhat.com> <5661E661.5090601@redhat.com> Message-ID: <5661E810.3060503@redhat.com> Martin Kosek wrote: > On 12/04/2015 07:17 PM, Tomas Babej wrote: >> Hi, >> >> Avoids failing in the later stages during the ipa-client-install >> command. >> >> Tomas > > Is this change needed? Wouldn't it be better to update > ipa-client-install or ipa-replica-install to not require the --domain > option? I would hope that --domain can be figured out during > installation and not passed to ipa-replica-install manually by the admin. > > I just think that calling > # ipa-replica-install --server=master.example.com > is better than > # ipa-replica-install --server=master.example.com --domain example.com > if possible. IIRC this is for service discovery when using a specific server and not LDAP. This is the domain used to search for the kerberos realm, for example. That isn't to say this isn't discoverable but it would require another function in discovery to query what the IPA domain is from the given master but it gets tricky if anonymous search is disabled, for example. rob From mbabinsk at redhat.com Fri Dec 4 19:32:10 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Fri, 4 Dec 2015 20:32:10 +0100 Subject: [Freeipa-devel] [PATCH 0111] prevent crashes of server uninstall check caused by failed, 5 LDAP connections Message-ID: <5661EA3A.3070504@redhat.com> https://fedorahosted.org/freeipa/ticket/5409 -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0111-prevent-crashes-of-server-uninstall-check-caused-by-.patch Type: text/x-patch Size: 1541 bytes Desc: not available URL: From rcritten at redhat.com Fri Dec 4 19:49:21 2015 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 4 Dec 2015 14:49:21 -0500 Subject: [Freeipa-devel] [PATCH 0111] prevent crashes of server uninstall check caused by failed, 5 LDAP connections In-Reply-To: <5661EA3A.3070504@redhat.com> References: <5661EA3A.3070504@redhat.com> Message-ID: <5661EE41.1090900@redhat.com> Martin Babinsky wrote: > https://fedorahosted.org/freeipa/ticket/5409 Should it also warn about the potential loss of the DNSSEC master? rob From mbasti at redhat.com Fri Dec 4 20:47:10 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 4 Dec 2015 21:47:10 +0100 Subject: [Freeipa-devel] [PATCH 0111] prevent crashes of server uninstall check caused by failed, 5 LDAP connections In-Reply-To: <5661EE41.1090900@redhat.com> References: <5661EA3A.3070504@redhat.com> <5661EE41.1090900@redhat.com> Message-ID: <5661FBCE.3020904@redhat.com> On 04.12.2015 20:49, Rob Crittenden wrote: > Martin Babinsky wrote: >> https://fedorahosted.org/freeipa/ticket/5409 > Should it also warn about the potential loss of the DNSSEC master? > > rob > IMO we can detect from statefiles if there were DNSSEC installed. Then it also should warn about potential loss of CA and KRA in case that it is the last replica containing it. Otherwise the patch LGTM. From redhatrises at gmail.com Fri Dec 4 21:55:26 2015 From: redhatrises at gmail.com (Gabe Alford) Date: Fri, 4 Dec 2015 14:55:26 -0700 Subject: [Freeipa-devel] [PATCH 0066] Migrate wget references to curl In-Reply-To: <5661E6B3.3070103@redhat.com> References: <5661E6B3.3070103@redhat.com> Message-ID: My bad. Copy and paste error. Updated patch attached. Thanks, Gabe On Fri, Dec 4, 2015 at 12:17 PM, Martin Basti wrote: > > > On 01.12.2015 15:00, Gabe Alford wrote: > > Hello, > > Fix for https://fedorahosted.org/freeipa/ticket/5458 > > Thanks, > > Gabe > > > Hello, > > I haven't looked closer, but your patch is causing this: > > Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes 30 > seconds > [1/27]: creating certificate server user > [2/27]: configuring certificate server instance > [3/27]: stopping certificate server instance to update CS.cfg > [4/27]: backing up CS.cfg > [5/27]: disabling nonces > [6/27]: set up CRL publishing > [7/27]: enable PKIX certificate path discovery and validation > [8/27]: starting certificate server instance > ipa.ipaserver.install.cainstance.CAInstance: CRITICAL Failed to restart > the Dogtag instance.See the installation log for details. > [9/27]: creating RA agent certificate database > [10/27]: importing CA chain to RA certificate database > [11/27]: fixing RA database permissions > [12/27]: setting up signing cert profile > [13/27]: setting audit signing renewal to 2 years > [14/27]: restarting certificate server > ipa.ipaserver.install.cainstance.CAInstance: CRITICAL Failed to restart > the Dogtag instance.See the installation log for details. > [15/27]: requesting RA certificate from CA > [16/27]: issuing RA agent certificate > [17/27]: adding RA agent as a trusted user > [18/27]: authorizing RA to modify profiles > [19/27]: configure certmonger for renewals > [20/27]: configure certificate renewals > [21/27]: configure RA certificate renewal > [22/27]: configure Server-Cert certificate renewal > [23/27]: Configure HTTP to proxy connections > [24/27]: restarting certificate server > > ipa.ipaserver.install.cainstance.CAInstance: CRITICAL Failed to restart > the Dogtag instance.See the installation log for details. > [25/27]: migrating certificate profiles to LDAP > [26/27]: importing IPA certificate profiles > [27/27]: adding default CA ACL > > > CA is operational and ready, but IPA installer is not able to detect it > correctly > > 2015-12-04T19:08:54Z DEBUG stderr=curl: option --connect-timeout 30: is > unknown > curl: try 'curl --help' or 'curl --manual' for more information > > Martin^2 > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rga-0066-2-Migrate-wget-references-and-usage-to-curl.patch Type: text/x-patch Size: 9494 bytes Desc: not available URL: From mbasti at redhat.com Fri Dec 4 22:04:21 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 4 Dec 2015 23:04:21 +0100 Subject: [Freeipa-devel] [PATCH 0109-0110] fix auto-forwarders in standalone DNS installer and update man pages In-Reply-To: <5661E602.6080104@redhat.com> References: <5661E602.6080104@redhat.com> Message-ID: <56620DE5.6050300@redhat.com> On 04.12.2015 20:14, Martin Babinsky wrote: > Additional fixes for https://fedorahosted.org/freeipa/ticket/5438 > > > ACK Pushed to master: 0997f6b9aadfe996a02004b4f3d03411ff5d141c -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbasti at redhat.com Fri Dec 4 22:10:47 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 4 Dec 2015 23:10:47 +0100 Subject: [Freeipa-devel] [PATCH 0096] check whether replica exists before executing the domain level 1 deletion code In-Reply-To: <5661BE1D.3080707@redhat.com> References: <564C6E3E.5040606@redhat.com> <565D57C7.40105@redhat.com> <5661BB89.9010504@redhat.com> <5661BE1D.3080707@redhat.com> Message-ID: <56620F67.4010805@redhat.com> On 04.12.2015 17:23, Martin Babinsky wrote: > On 12/04/2015 05:12 PM, Petr Vobornik wrote: >> On 12/01/2015 09:18 AM, Martin Basti wrote: >>> >>> >>> On 18.11.2015 13:25, Martin Babinsky wrote: >>>> Additional fix for https://fedorahosted.org/freeipa/ticket/5424 >>>> >>>> In current implementation the topology suffices are checked first and >>>> after that the error about non-existent host is raised. This does not >>>> make much sense to me, we should check for host existence before any >>>> work is done. >>>> >>>> >>>> >>> ACK >>> >> >> Any reason to use host-show and not server-show? >> >> host-show will be successful on any host, server-show only if the host >> is an IPA server. > Good catch, I have fixed it in this patch. I have also modified the > error message slightly. > ACK Pushed to master: ee853a3d35701d1d799f902f823b8a8cedb90013 From mbasti at redhat.com Sun Dec 6 20:32:56 2015 From: mbasti at redhat.com (Martin Basti) Date: Sun, 6 Dec 2015 21:32:56 +0100 Subject: [Freeipa-devel] [PATCHES 509-514] replica promotion: use host credentials when setting up replication In-Reply-To: <1449244714.3445.67.camel@redhat.com> References: <564DE017.4020408@redhat.com> <1447951434.5132.27.camel@redhat.com> <564F4105.1040204@redhat.com> <1448038723.5132.59.camel@redhat.com> <5652C63E.5090206@redhat.com> <1448289289.7892.13.camel@redhat.com> <565324C4.1090007@redhat.com> <1448290024.7892.15.camel@redhat.com> <565D823E.5090600@redhat.com> <56604D94.7000705@redhat.com> <566167DE.9090204@redhat.com> <56619F5C.5000900@redhat.com> <5661A04E.9080409@redhat.com> <5661A593.5030003@redhat.com> <1449244714.3445.67.camel@redhat.com> Message-ID: <56649B78.1080805@redhat.com> On 04.12.2015 16:58, Simo Sorce wrote: > On Fri, 2015-12-04 at 15:39 +0100, Jan Cholasta wrote: >> On 4.12.2015 15:16, Jan Cholasta wrote: >>> On 4.12.2015 15:12, Jan Cholasta wrote: >>>> On 4.12.2015 11:15, Petr Vobornik wrote: >>>>> On 12/03/2015 03:11 PM, Martin Basti wrote: >>>>>> >>>>>> On 01.12.2015 12:19, Jan Cholasta wrote: >>>>>>> On 23.11.2015 15:47, Simo Sorce wrote: >>>>>>>> On Mon, 2015-11-23 at 15:37 +0100, Jan Cholasta wrote: >>>>>>>>> Ad alternative is to add the host to ipaservers before the checks >>>>>>>>> are >>>>>>>>> done and remove it again if any of them fail. >>>>>>>> Too error prone, I am ok with the current way in your patches >>>>>>>> until/unless I can think of a fail safe way. :-) >>>>>>> Updated patches attached. Note that 520 should be applied between 509 >>>>>>> and 510. >>>>>>> >>>>>>> >>>>>>> >>>>>> Functional ACK >>>>>> >>>>> Simo, do you want to review the ACIs or other things it the patches? Or >>>>> can the patches be pushed? >>>> There were no changes in the ACIs since last time. >>> Actually, memberPrincipal was removed from the "IPA server hosts can >>> manage own Custodia secrets" ACI, as per Simo's request. >>> >>>> Rebased patches attached. >> Note that 520 should still be applied between 509 and 510. >> > LGTM > ACK From mbasti at redhat.com Sun Dec 6 20:45:57 2015 From: mbasti at redhat.com (Martin Basti) Date: Sun, 6 Dec 2015 21:45:57 +0100 Subject: [Freeipa-devel] [PATCH 0369] Force creation of service during replica install Message-ID: <56649E85.1000303@redhat.com> Replica install should not fail due a missing A record, if there are proper entries in hosts. Patch attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0369-Force-creation-of-services-during-replica-install.patch Type: text/x-patch Size: 1193 bytes Desc: not available URL: From mbasti at redhat.com Sun Dec 6 20:52:38 2015 From: mbasti at redhat.com (Martin Basti) Date: Sun, 6 Dec 2015 21:52:38 +0100 Subject: [Freeipa-devel] [PATCH 0371] CI: fix function that prepares /etc/hosts Message-ID: <5664A016.5060900@redhat.com> Without this fix, function removes more entries from /etc/host than is required, and it causes installation failure in tests without DNS Patch attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0371-CI-fix-function-that-prepare-the-hosts-file-before-C.patch Type: text/x-patch Size: 1596 bytes Desc: not available URL: From mbasti at redhat.com Sun Dec 6 21:01:17 2015 From: mbasti at redhat.com (Martin Basti) Date: Sun, 6 Dec 2015 22:01:17 +0100 Subject: [Freeipa-devel] [PATCH 0370] CI: test various topologies with 3 replicas Message-ID: <5664A21D.7040802@redhat.com> Patch attached, to work properly it requires all patches I sent today + workaround patch attached (Martin3 will provide proper fix). The last two test are failing due to bug in test framework (ipa-replica-manage should not be used with domain level1) Martin^2 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0000-CI-uninstall-WORKAROUND-DO-NOT-PUSH.patch Type: text/x-patch Size: 960 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0370-CI-test-various-topologies-with-multiple-replicas.patch Type: text/x-patch Size: 3340 bytes Desc: not available URL: From mbasti at redhat.com Sun Dec 6 21:22:32 2015 From: mbasti at redhat.com (Martin Basti) Date: Sun, 6 Dec 2015 22:22:32 +0100 Subject: [Freeipa-devel] [PATCH 0372] CI: installation tests Message-ID: <5664A718.6080101@redhat.com> My favorite today \o/ --> 67 <-- \o/ test cases, no more manual testing of installers \o/. Test suite contains: 6 combination how to install components on replica X 4 combinations of server installation + 3 extra server tests To save time tests install 1 master and 3 replicas per test class (except extra server tests): Class name specifies what is installed on master. Remember, option "-k" is your friend $ ipa-run-tests -k '' otherwise you will need a lot of time until tests finish. To list all tests: $ ipa-run-tests test_integration/test_installation.py --collect-only Patch attached, it needs to have attached all patches I sent today and also attached workaround patch (Martin3 will provide proper fix) I haven't had time/mood/resources to test this patch with domain level 0, so testing this patch with domain level 0 is appreciated. Oleg, I added some methods to tasks.py which you may want to reuse. Martin^2 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0372-CI-installation-tests.patch Type: text/x-patch Size: 7810 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0000-CI-uninstall-WORKAROUND-DO-NOT-PUSH.patch Type: text/x-patch Size: 960 bytes Desc: not available URL: From ftweedal at redhat.com Mon Dec 7 05:26:11 2015 From: ftweedal at redhat.com (Fraser Tweedale) Date: Mon, 7 Dec 2015 15:26:11 +1000 Subject: [Freeipa-devel] [PATCH] 0046 Create server certs with DNS altname Message-ID: <20151207052611.GG23644@dhcp-40-8.bne.redhat.com> The attached patch fixes https://fedorahosted.org/freeipa/ticket/4970. Note that the problem is addressed by adding the appropriate request extension to the CSR; the fix does not involve changing the default profile behaviour, which is complicated (see ticket for details). Thanks, Fraser -------------- next part -------------- From e984b2cbfd419a2a71aa40ba4b42dd29857a66d9 Mon Sep 17 00:00:00 2001 From: Fraser Tweedale Date: Mon, 7 Dec 2015 16:14:28 +1100 Subject: [PATCH] Create server certs with DNS altname Currently server (HTTP / LDAP) certs are created without a Subject Alternative Name extension during server install or replica prepare, a potentially problematic violation of RFC 2818. Add the hostname as a SAN dNSName when these certs are created. (Certmonger already adds an appropriate request extension when renewing the certificate, so nothing needs to be done for renewal). Fixes: https://fedorahosted.org/freeipa/ticket/4970 --- ipaserver/install/certs.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/ipaserver/install/certs.py b/ipaserver/install/certs.py index c918791f0be7a17e20123fe6f94c4ac0bbf09d7b..bd1792d32246bc3034c5403f1d868e0966ec0014 100644 --- a/ipaserver/install/certs.py +++ b/ipaserver/install/certs.py @@ -335,7 +335,7 @@ class CertDB(object): cdb = self if subject is None: subject=DN(('CN', hostname), self.subject_base) - self.request_cert(subject) + self.request_cert(subject, san_dnsnames=[hostname]) cdb.issue_server_cert(self.certreq_fname, self.certder_fname) self.import_cert(self.certder_fname, nickname) fd = open(self.certder_fname, "r") @@ -359,7 +359,9 @@ class CertDB(object): os.unlink(self.certreq_fname) os.unlink(self.certder_fname) - def request_cert(self, subject, certtype="rsa", keysize="2048"): + def request_cert( + self, subject, certtype="rsa", keysize="2048", + san_dnsnames=None): assert isinstance(subject, DN) self.create_noise_file() self.setup_cert_request() @@ -370,6 +372,8 @@ class CertDB(object): "-z", self.noise_fname, "-f", self.passwd_fname, "-a"] + if san_dnsnames is not None and len(san_dnsnames) > 0: + args += ['-8', ','.join(san_dnsnames)] (stdout, stderr, returncode) = self.run_certutil(args) os.remove(self.noise_fname) return (stdout, stderr) -- 2.4.3 From abokovoy at redhat.com Mon Dec 7 06:52:37 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 7 Dec 2015 08:52:37 +0200 Subject: [Freeipa-devel] [PATCH] 0046 Create server certs with DNS altname In-Reply-To: <20151207052611.GG23644@dhcp-40-8.bne.redhat.com> References: <20151207052611.GG23644@dhcp-40-8.bne.redhat.com> Message-ID: <20151207065237.GT8374@redhat.com> On Mon, 07 Dec 2015, Fraser Tweedale wrote: >The attached patch fixes >https://fedorahosted.org/freeipa/ticket/4970. > >Note that the problem is addressed by adding the appropriate request >extension to the CSR; the fix does not involve changing the default >profile behaviour, which is complicated (see ticket for details). LGTM. I haven't tested the patch yet but it does what is explained. -- / Alexander Bokovoy From jcholast at redhat.com Mon Dec 7 07:14:55 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 7 Dec 2015 08:14:55 +0100 Subject: [Freeipa-devel] [PATCHES 509-514] replica promotion: use host credentials when setting up replication In-Reply-To: <56649B78.1080805@redhat.com> References: <564DE017.4020408@redhat.com> <1447951434.5132.27.camel@redhat.com> <564F4105.1040204@redhat.com> <1448038723.5132.59.camel@redhat.com> <5652C63E.5090206@redhat.com> <1448289289.7892.13.camel@redhat.com> <565324C4.1090007@redhat.com> <1448290024.7892.15.camel@redhat.com> <565D823E.5090600@redhat.com> <56604D94.7000705@redhat.com> <566167DE.9090204@redhat.com> <56619F5C.5000900@redhat.com> <5661A04E.9080409@redhat.com> <5661A593.5030003@redhat.com> <1449244714.3445.67.camel@redhat.com> <56649B78.1080805@redhat.com> Message-ID: <566531EF.7000603@redhat.com> On 6.12.2015 21:32, Martin Basti wrote: > > > On 04.12.2015 16:58, Simo Sorce wrote: >> On Fri, 2015-12-04 at 15:39 +0100, Jan Cholasta wrote: >>> On 4.12.2015 15:16, Jan Cholasta wrote: >>>> On 4.12.2015 15:12, Jan Cholasta wrote: >>>>> On 4.12.2015 11:15, Petr Vobornik wrote: >>>>>> On 12/03/2015 03:11 PM, Martin Basti wrote: >>>>>>> >>>>>>> On 01.12.2015 12:19, Jan Cholasta wrote: >>>>>>>> On 23.11.2015 15:47, Simo Sorce wrote: >>>>>>>>> On Mon, 2015-11-23 at 15:37 +0100, Jan Cholasta wrote: >>>>>>>>>> Ad alternative is to add the host to ipaservers before the checks >>>>>>>>>> are >>>>>>>>>> done and remove it again if any of them fail. >>>>>>>>> Too error prone, I am ok with the current way in your patches >>>>>>>>> until/unless I can think of a fail safe way. :-) >>>>>>>> Updated patches attached. Note that 520 should be applied >>>>>>>> between 509 >>>>>>>> and 510. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> Functional ACK >>>>>>> >>>>>> Simo, do you want to review the ACIs or other things it the >>>>>> patches? Or >>>>>> can the patches be pushed? >>>>> There were no changes in the ACIs since last time. >>>> Actually, memberPrincipal was removed from the "IPA server hosts can >>>> manage own Custodia secrets" ACI, as per Simo's request. >>>> >>>>> Rebased patches attached. >>> Note that 520 should still be applied between 509 and 510. >>> >> LGTM >> > ACK Thanks. Pushed to master: 01ddf51df76f3298499973355c5461727e46ab5b -- Jan Cholasta From jcholast at redhat.com Mon Dec 7 07:21:42 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 7 Dec 2015 08:21:42 +0100 Subject: [Freeipa-devel] [PATCH 522] replica promotion: allow OTP bulk client enrollment In-Reply-To: <565F0D00.6050805@redhat.com> References: <565F0D00.6050805@redhat.com> Message-ID: <56653386.5020703@redhat.com> On 2.12.2015 16:23, Jan Cholasta wrote: > Hi, > > the attached patch fixes . > > Note that you still have to provide admin password in > ipa-replica-install, either using --admin-password or interactively, > because: > > a) Admin password is required for replica promotion. This will be fixed > with . > > Patches are on the list: > . Pushed. > > > b) Admin password is required for connection check. This will be fixed > with . Martin Basti pointed out that admin password should not be asked interactively during OTP replica promotion. Fixed. Updated and rebased patch attached. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-522.1-replica-promotion-allow-OTP-bulk-client-enrollment.patch Type: text/x-patch Size: 5842 bytes Desc: not available URL: From ofayans at redhat.com Mon Dec 7 09:36:43 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Mon, 7 Dec 2015 10:36:43 +0100 Subject: [Freeipa-devel] You cannot specify '--admin-password' option(s) with replica file Message-ID: <5665532B.6070004@redhat.com> This is an error message that I received at the attempt to install replica with the following command: ipa-replica-install --setup-ca -p -w /var/lib/ipa/replica-info-replica2.justfor.test.gpg However, if I remove the '-w ', then I get the password prompt for admin password interactively. The domain level is 0. The packages are built last Friday from upstream code. -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From tbabej at redhat.com Mon Dec 7 09:58:12 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 7 Dec 2015 10:58:12 +0100 Subject: [Freeipa-devel] [PATCH 0392] tests: Fix incorrect uninstall method invocation Message-ID: <56655834.7000006@redhat.com> Hi, this fixes: https://fedorahosted.org/freeipa/ticket/5516 Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0392-tests-Fix-incorrect-uninstall-method-invocation.patch Type: text/x-patch Size: 982 bytes Desc: not available URL: From tbabej at redhat.com Mon Dec 7 09:59:56 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 7 Dec 2015 10:59:56 +0100 Subject: [Freeipa-devel] [PATCH 0392] tests: Fix incorrect uninstall method invocation In-Reply-To: <56655834.7000006@redhat.com> References: <56655834.7000006@redhat.com> Message-ID: <5665589C.7080808@redhat.com> On 12/07/2015 10:58 AM, Tomas Babej wrote: > Hi, > > this fixes: https://fedorahosted.org/freeipa/ticket/5516 > > Tomas > Pushed under oneliner rule: master: 5cb003f0b4b85dce47499f594c410b34b5c961e2 ipa-4-2: e5189ef6e23e4691f6c74541da5bc1a0b0f2e73f From mbabinsk at redhat.com Mon Dec 7 10:05:24 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Mon, 7 Dec 2015 11:05:24 +0100 Subject: [Freeipa-devel] [PATCH 0112] CI tests: ignore disconnected domain level 1 topology on IPA master teardown Message-ID: <566559E4.9020506@redhat.com> This patch should fix teardown methods in replication-related CI tests ran at non-zero domain level. -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0112-CI-tests-ignore-disconnected-domain-level-1-topology.patch Type: text/x-patch Size: 3000 bytes Desc: not available URL: From ofayans at redhat.com Mon Dec 7 10:12:50 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Mon, 7 Dec 2015 11:12:50 +0100 Subject: [Freeipa-devel] [PATCH 0392] tests: Fix incorrect uninstall method invocation In-Reply-To: <5665589C.7080808@redhat.com> References: <56655834.7000006@redhat.com> <5665589C.7080808@redhat.com> Message-ID: <56655BA2.1000001@redhat.com> Hi Toma?, Could you please review my patch regarding caless tests. I's been on the list since ages. It contains this particular change together with a lot more of them. On 12/07/2015 10:59 AM, Tomas Babej wrote: > On 12/07/2015 10:58 AM, Tomas Babej wrote: >> Hi, >> >> this fixes: https://fedorahosted.org/freeipa/ticket/5516 >> >> Tomas >> > > Pushed under oneliner rule: > master: 5cb003f0b4b85dce47499f594c410b34b5c961e2 > ipa-4-2: e5189ef6e23e4691f6c74541da5bc1a0b0f2e73f > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From ofayans at redhat.com Mon Dec 7 11:07:37 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Mon, 7 Dec 2015 12:07:37 +0100 Subject: [Freeipa-devel] [PATCH 0112] CI tests: ignore disconnected domain level 1 topology on IPA master teardown In-Reply-To: <566559E4.9020506@redhat.com> References: <566559E4.9020506@redhat.com> Message-ID: <56656879.2050007@redhat.com> Hi Martin, CONFIGURED_DOMAIN_LEVEL is declared, but not used. The rest looks fine to me On 12/07/2015 11:05 AM, Martin Babinsky wrote: > This patch should fix teardown methods in replication-related CI tests > ran at non-zero domain level. > > > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From pvoborni at redhat.com Mon Dec 7 11:33:41 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 7 Dec 2015 12:33:41 +0100 Subject: [Freeipa-devel] [PATCH] bz 1288863 Fix minor typos Message-ID: <56656E95.9010201@redhat.com> attached patch was attached to https://bugzilla.redhat.com/show_bug.cgi?id=1288863 ACK Pushed to master: 2180d5db8a8e99007c39466c19759a4b1bf098fa -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-minor-typos.patch Type: text/x-patch Size: 1669 bytes Desc: not available URL: From fskola at redhat.com Mon Dec 7 12:00:41 2015 From: fskola at redhat.com (Filip =?UTF-8?B?xaBrb2xh?=) Date: Mon, 7 Dec 2015 13:00:41 +0100 Subject: [Freeipa-devel] [PATCH 0004] Refactor test_attr In-Reply-To: <20151204162416.28170b8e@vor2.netbox.priv> References: <20151204162416.28170b8e@vor2.netbox.priv> Message-ID: <20151207130041.409f5022@dhcp-24-120.brq.redhat.com> Self-NACK, resubmitting with the last commit which includes UserTracker from the right location... F. On Fri, 4 Dec 2015 16:24:16 +0100 Filip ?kola wrote: > Hi, > > sending a new version of test_attr. > > F. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-fskola-0004-2-Refactor-test_attr.patch Type: text/x-patch Size: 26886 bytes Desc: not available URL: From ofayans at redhat.com Mon Dec 7 12:24:35 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Mon, 7 Dec 2015 13:24:35 +0100 Subject: [Freeipa-devel] [PATCH 0372] CI: installation tests In-Reply-To: <5664A718.6080101@redhat.com> References: <5664A718.6080101@redhat.com> Message-ID: <56657A83.6020702@redhat.com> Hi Martin, I would prefer both install_kra and install_ca methods to have raiseonerr parameter set to True by default. We need a way to test negatives and analyze results. Mine looks like this: def install_kra(host, domain_level=None, first_instance=False, raiseonerr=True): if not domain_level: domain_level = host.config.domain_level command = ["ipa-kra-install", "-U", "-p", host.config.dirman_password] if domain_level == DOMAIN_LEVEL_0 and not first_instance: replica_file = get_replica_filename(host) command.append(replica_file) return host.run_command(command, raiseonerr=raiseonerr) The rest looks good to me, but I did not run the tests yet. On 12/06/2015 10:22 PM, Martin Basti wrote: > My favorite today \o/ --> 67 <-- \o/ test cases, no more manual testing > of installers \o/. > > Test suite contains: 6 combination how to install components on replica > X 4 combinations of server installation + 3 extra server tests > > To save time tests install 1 master and 3 replicas per test class > (except extra server tests): > Class name specifies what is installed on master. > > Remember, option "-k" is your friend > $ ipa-run-tests -k '' > otherwise you will need a lot of time until tests finish. > > To list all tests: > $ ipa-run-tests test_integration/test_installation.py --collect-only > > > Patch attached, it needs to have attached all patches I sent today and > also attached workaround patch (Martin3 will provide proper fix) > > I haven't had time/mood/resources to test this patch with domain level > 0, so testing this patch with domain level 0 is appreciated. > > > Oleg, I added some methods to tasks.py which you may want to reuse. > > Martin^2 > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From fskola at redhat.com Mon Dec 7 12:25:27 2015 From: fskola at redhat.com (Filip =?UTF-8?B?xaBrb2xh?=) Date: Mon, 7 Dec 2015 13:25:27 +0100 Subject: [Freeipa-devel] [PATCH 0004] Refactor test_attr In-Reply-To: <20151207130041.409f5022@dhcp-24-120.brq.redhat.com> References: <20151204162416.28170b8e@vor2.netbox.priv> <20151207130041.409f5022@dhcp-24-120.brq.redhat.com> Message-ID: <20151207132527.7963c5ae@dhcp-24-120.brq.redhat.com> Now the tier marker have lost somewhere on the way... which is corrected in this patch. /me apologizes for the noise F. On Mon, 7 Dec 2015 13:00:41 +0100 Filip ?kola wrote: > Self-NACK, resubmitting with the last commit which includes > UserTracker from the right location... > > F. > > On Fri, 4 Dec 2015 16:24:16 +0100 > Filip ?kola wrote: > > > Hi, > > > > sending a new version of test_attr. > > > > F. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-fskola-0004-3-Refactor-test_attr.patch Type: text/x-patch Size: 26902 bytes Desc: not available URL: From mkosek at redhat.com Mon Dec 7 12:53:15 2015 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 7 Dec 2015 13:53:15 +0100 Subject: [Freeipa-devel] [PATCH] 0046 Create server certs with DNS altname In-Reply-To: <20151207052611.GG23644@dhcp-40-8.bne.redhat.com> References: <20151207052611.GG23644@dhcp-40-8.bne.redhat.com> Message-ID: <5665813B.90406@redhat.com> On 12/07/2015 06:26 AM, Fraser Tweedale wrote: > The attached patch fixes > https://fedorahosted.org/freeipa/ticket/4970. > > Note that the problem is addressed by adding the appropriate request > extension to the CSR; the fix does not involve changing the default > profile behaviour, which is complicated (see ticket for details). Thanks for the patch! This is something we should really fix, I already get warnings in my Python scripts when I hit sites protected by such HTTPS cert: /usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:264: SubjectAltNameWarning: Certificate for projects.engineering.redhat.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.) Should we split ticket 4970, for the FreeIPA server part and then for cert profile part? As it looks like the FreeIPA server will be fixed even in FreeIPA 4.3.x and the other part later. How difficult do you see the general FreeIPA Certificate Profile part of this request? Is it a too big task to handle in 4.4 time frame? From mbabinsk at redhat.com Mon Dec 7 12:53:27 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Mon, 7 Dec 2015 13:53:27 +0100 Subject: [Freeipa-devel] [PATCH 0112] CI tests: ignore disconnected domain level 1 topology on IPA master teardown In-Reply-To: <56656879.2050007@redhat.com> References: <566559E4.9020506@redhat.com> <56656879.2050007@redhat.com> Message-ID: <56658147.1030508@redhat.com> On 12/07/2015 12:07 PM, Oleg Fayans wrote: > Hi Martin, > > CONFIGURED_DOMAIN_LEVEL is declared, but not used. The rest looks fine to me > > On 12/07/2015 11:05 AM, Martin Babinsky wrote: >> This patch should fix teardown methods in replication-related CI tests >> ran at non-zero domain level. >> >> >> > Ah that was a leftover from previous implementation. Here's updated patch. -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0112.1-CI-tests-ignore-disconnected-domain-level-1-topology.patch Type: text/x-patch Size: 2597 bytes Desc: not available URL: From lslebodn at redhat.com Mon Dec 7 12:54:34 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Mon, 7 Dec 2015 13:54:34 +0100 Subject: [Freeipa-devel] [PATCH 0392] tests: Fix incorrect uninstall method invocation In-Reply-To: <56655BA2.1000001@redhat.com> References: <56655834.7000006@redhat.com> <5665589C.7080808@redhat.com> <56655BA2.1000001@redhat.com> Message-ID: <20151207125433.GC16627@mail.corp.redhat.com> On (07/12/15 11:12), Oleg Fayans wrote: >Hi Toma?, > >Could you please review my patch regarding caless tests. I's been on the >list since ages. It contains this particular change together with a lot >more of them. > We are used to sending "bump" mails on sssd-devel. It's better then sending mail to another thread. Personally, I do not have a problem to send "bump" mail even every day :-) LS From dkupka at redhat.com Mon Dec 7 13:05:29 2015 From: dkupka at redhat.com (David Kupka) Date: Mon, 7 Dec 2015 14:05:29 +0100 Subject: [Freeipa-devel] [PATCH 0070] install: Run all validators at once. Message-ID: <56658419.3080100@redhat.com> Running validators after all Knobs are set allows use of other Knob value during validation. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0070-install-Run-all-validators-at-once.patch Type: text/x-patch Size: 2013 bytes Desc: not available URL: From dkupka at redhat.com Mon Dec 7 13:06:04 2015 From: dkupka at redhat.com (David Kupka) Date: Mon, 7 Dec 2015 14:06:04 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <55F01A60.9010506@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> Message-ID: <5665843C.2090507@redhat.com> On 09/09/15 13:39, Petr Spacek wrote: > On 8.9.2015 16:30, David Kupka wrote: >> On 28/08/15 13:36, Martin Basti wrote: >>> >>> >>> On 08/28/2015 10:03 AM, Petr Spacek wrote: >>>> On 27.8.2015 14:22, David Kupka wrote: >>>>> @@ -2101,11 +2101,25 @@ class DNSZoneBase(LDAPObject): >>>>> class DNSZoneBase_add(LDAPCreate): >>>>> + takes_options = LDAPCreate.takes_options + ( >>>>> + Flag('force', >>>>> + label=_('Force'), >>>>> + doc=_('Force DNS zone creation.') >>>>> + ), >>>>> + Flag('skip_overlap_check', >>>>> + doc=_('Force DNS zone creation even if it will overlap >>>>> with ' >>>>> + 'existing zone.') >>>>> + ), >>>>> + ) >>>>> + >>>>> has_output_params = LDAPCreate.has_output_params + >>>>> dnszone_output_params >>>>> def pre_callback(self, ldap, dn, entry_attrs, attrs_list, >>>>> *keys, **options): >>>>> assert isinstance(dn, DN) >>>>> + if options['force']: >>>>> + options['skip_overlap_check'] = True >>>>> + >>>>> try: >>>>> entry = ldap.get_entry(dn) >>>>> except errors.NotFound: >>>>> @@ -2120,6 +2134,12 @@ class DNSZoneBase_add(LDAPCreate): >>>>> entry_attrs['idnszoneactive'] = 'TRUE' >>>>> + if not options['skip_overlap_check']: >>>>> + try: >>>>> + check_zone_overlap(keys[-1]) >>>>> + except RuntimeError as e: >>>>> + raise errors.InvocationError(e.message) >>>>> + >>>>> return dn >>>>> @@ -2673,9 +2693,9 @@ class dnszone_add(DNSZoneBase_add): >>>>> __doc__ = _('Create new DNS zone (SOA record).') >>>>> takes_options = DNSZoneBase_add.takes_options + ( >>>>> - Flag('force', >>>>> - label=_('Force'), >>>>> - doc=_('Force DNS zone creation even if nameserver is >>>>> not resolvable.'), >>>>> + Flag('skip_nameserver_check', >>>>> + doc=_('Force DNS zone creation even if nameserver is not ' >>>>> + 'resolvable.') >>>>> ), >>>>> # Deprecated >>>>> @@ -2699,6 +2719,9 @@ class dnszone_add(DNSZoneBase_add): >>>>> def pre_callback(self, ldap, dn, entry_attrs, attrs_list, >>>>> *keys, **options): >>>>> assert isinstance(dn, DN) >>>>> + if options['force']: >>>>> + options['skip_nameserver_check'] = True >>>> Why is it in DNSZoneBase_add.pre_callback? >>>> >>>> Shouldn't the equation force = (skip_nameserver_check + >>>> skip_nameserver_check) >>>> be handled in parameter parsing & validation? (Again, I do not know >>>> the IPA >>>> framework :-)) >>>> >>>>> + >>>>> dn = super(dnszone_add, self).pre_callback( >>>>> ldap, dn, entry_attrs, attrs_list, *keys, **options) >>>>> @@ -2713,7 +2736,7 @@ class dnszone_add(DNSZoneBase_add): >>>>> error=_("Nameserver for reverse zone cannot be >>>>> a relative DNS name")) >>>>> # verify if user specified server is resolvable >>>>> - if not options['force']: >>>>> + if not options['skip_nameserver_check']: >>>>> check_ns_rec_resolvable(keys[0], >>>>> entry_attrs['idnssoamname']) >>>>> # show warning about --name-server option >>>>> context.show_warning_nameserver_option = True >>>>> diff --git a/ipapython/ipautil.py b/ipapython/ipautil.py >>>>> index >>>>> d959bb369d946217acd080e78483cc9013dda4c7..18f477d4fb6620090b7073689c8df51b65a8307a >>>>> >>>>> 100644 >>>>> --- a/ipapython/ipautil.py >>>>> +++ b/ipapython/ipautil.py >>>>> @@ -924,6 +924,20 @@ def host_exists(host): >>>>> else: >>>>> return True >>>>> +def check_zone_overlap(zone): >>>>> + if resolver.zone_for_name(zone) == zone: >>>>> + try: >>>>> + ns = [ans.to_text() for ans in resolver.query(zone, 'NS')] >>>>> + except DNSException as e: >>>>> + root_logger.debug("Failed to resolve nameserver(s) for >>>>> domain" >>>>> + " {0}: {1}".format(zone, e)) >>>>> + ns = [] >>>>> + >>>>> + msg = u"DNS zone {0} already exists".format(zone) >>>> Nitpick: I would say "already exists in DNS" to make it absolutely >>>> clear. Just >>>> 'already exists' might be confusing because ipa dnszone-show will say >>>> that the >>>> zone does not exist ... >>>> >>>>> + if ns: >>>>> + msg += u" and is handled by server(s): {0}".format(', >>>>> '.join(ns)) >>>>> + raise RuntimeError(msg) >>>>> + >>>>> def get_ipa_basedn(conn): >>>>> """ >>>>> Get base DN of IPA suffix in given LDAP server. >>> 0064 >>> NACK >>> >>> ipa-replica-install should have the --skip-overlap-check too, because >>> any replica can be the first DNS server. >> >> Thanks for the catch, added. >> >>> >>> 0064+0058 >>> Can be the options --allow-zone-overlap and --skip-overlap-check merged >>> into an one name, to have just one name for same thing? >>> >> >> Each option has bit different behavior: >> The '--skip-overlap-check' option in API call prevent the check to be >> performed and thus no error or warning is raised. This is the way '--force' >> option was originally working. >> >> The '--allow-zone-overlap' options in installers do not skip the check but >> change the error to warning instead and let the installation continue. >> >> If you think that this can confuse users we need to change the names or even >> the logic. >> >> Updated patches attached. > > Hello, > > thank you very much for updating the patch. > > Unfortunately it is not yet ready, but we are getting there. > > > * Serious problems: > > a) ipa-server/replica/dns-install by default creates reverse zones even if > these zones exist. The check which is done for main IPA domain should be done > for all reverse zones, too. If a zone exists user should use --no-reverse or > --allow-zone-overlap if necessary. > > I believe that this is in scope of https://fedorahosted.org/freeipa/ticket/3681 . > > > b) ipa-server/replica/dns-install by default always add DNS zone which > contains hostname of the IPA server, even if the IPA domain is different and > even if the DNS zone already exists. > > If the hostname of the server does not belong to IPA domain we should not add > anything. Just check that the hostname is resolvable. Theoretically we could > add an option which explicitly asks for creating zone which encloses the > hostnmame. > > I believe that this is in scope of https://fedorahosted.org/freeipa/ticket/5087 . > > > c) I did not realize that checks in ipa dns*zone-add will break zone addition > for automatic empty zones. > > There is list of special-case DNS zones which need special handling: > https://source.isc.org/cgi-bin/gitweb.cgi?p=bind9.git;a=blob;f=bin/named/server.c;h=1a25531df354f6f0593bfb86e5aa3e3c9b9c80e5;hb=HEAD#l272 > > Feel free to copy&paste list of the domains to IPA code - list comes from > RFCs, so it should be pretty stable. It would be good > > If one of these zones is detected as "already existing" we need to further > check values in SOA record and automatically override the check if: > (SOA mname = zone name) & (SOA rname = .) > > In that case we should print informational message "automatic empty zone will > be overridden by the zone you defined now". > > > d) ipa-server/replica/dns-install fails when attempt to resolve main IPA > domain name fails with DNS timeout. This might easily happen if the domain > name is already delegated to IPA server which is being installed. In that case > we should print a warning and allow to continue. > > """ > Warning: DNS check for domain failed with error . > Please make sure that the domain is properly delegated to this IPA server. > """ > > > * Nitpicks - can be handled in separate patches: > 1) Interactive ipa-server-install tells the user that the zone name he entered > exists too late in the process. > > I.e. user has to enter DM and admin password (twice!) and only after that he > will receive an error, which may be frustrating :-) > > Please move the check right after DNS zone name input so user does not waste > his time and nerves :-) Even better, in an interactive mode it could make > sense to ask the user again if the previous input was not acceptable. > > > 2) I found out that ipa-server-install prints message "Warning: skipping DNS > resolution of host vm-206.abc.idm.lab.eng.brq.redhat.com" even if the host > name does not belong to IPA domain. That is a mistake - the check should be > skipped only in case when IPA server is part of the IPA domain. > > > 3) When adding an forward as an override for automatic empty zones we should > throw a warning if forward policy is not 'only'. It does not make sense to use > policy 'forward' in these cases. > """ > Warning: It is strongly recommended to use forward policy 'only' for the > forward zones defined for private use. > """ > Hi! I was finally able to get back to this patches. I've (hopefully) addressed all the items from "Serious problems" section plus the first "nitpick" as I found it really annoying. Updated patches attached. Patch freeipa-dkupka-0070 is needed for this to work. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0058.6-dns-do-not-add-forward-zone-if-it-is-already-resolva.patch Type: text/x-patch Size: 11794 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0064.1-dns-Check-if-domain-already-exists.patch Type: text/x-patch Size: 3834 bytes Desc: not available URL: From tbabej at redhat.com Mon Dec 7 13:17:40 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 7 Dec 2015 14:17:40 +0100 Subject: [Freeipa-devel] [PATCH 0391] replicainstall: Add check for domain if server is specified In-Reply-To: <5661E810.3060503@redhat.com> References: <5661D8A0.7030303@redhat.com> <5661E661.5090601@redhat.com> <5661E810.3060503@redhat.com> Message-ID: <566586F4.30201@redhat.com> On 12/04/2015 08:22 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 12/04/2015 07:17 PM, Tomas Babej wrote: >>> Hi, >>> >>> Avoids failing in the later stages during the ipa-client-install >>> command. >>> >>> Tomas >> >> Is this change needed? Wouldn't it be better to update >> ipa-client-install or ipa-replica-install to not require the --domain >> option? I would hope that --domain can be figured out during >> installation and not passed to ipa-replica-install manually by the admin. >> >> I just think that calling >> # ipa-replica-install --server=master.example.com >> is better than >> # ipa-replica-install --server=master.example.com --domain example.com >> if possible. > > IIRC this is for service discovery when using a specific server and not > LDAP. This is the domain used to search for the kerberos realm, for > example. > > That isn't to say this isn't discoverable but it would require another > function in discovery to query what the IPA domain is from the given > master but it gets tricky if anonymous search is disabled, for example. > > rob > Needed or not, this is the behaviour that ipa-client-install has now. Adding a domain detection method would be a RFE for ipa-client-install (and imho not something we should be adding at this point). This patch only focuses on making the ipa-replica-install work more smoothly. Tomas From tbabej at redhat.com Mon Dec 7 13:22:34 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 7 Dec 2015 14:22:34 +0100 Subject: [Freeipa-devel] You cannot specify '--admin-password' option(s) with replica file In-Reply-To: <5665532B.6070004@redhat.com> References: <5665532B.6070004@redhat.com> Message-ID: <5665881A.3050301@redhat.com> On 12/07/2015 10:36 AM, Oleg Fayans wrote: > This is an error message that I received at the attempt to install > replica with the following command: > > ipa-replica-install --setup-ca -p -w > /var/lib/ipa/replica-info-replica2.justfor.test.gpg > > However, if I remove the '-w ', then I get the password > prompt for admin password interactively. The domain level is 0. The > packages are built last Friday from upstream code. > This is a legitimate issue, can you file a ticket? Thanks, Tomas From ofayans at redhat.com Mon Dec 7 13:25:31 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Mon, 7 Dec 2015 14:25:31 +0100 Subject: [Freeipa-devel] You cannot specify '--admin-password' option(s) with replica file In-Reply-To: <5665881A.3050301@redhat.com> References: <5665532B.6070004@redhat.com> <5665881A.3050301@redhat.com> Message-ID: <566588CB.6090408@redhat.com> Hi, On 12/07/2015 02:22 PM, Tomas Babej wrote: > > > On 12/07/2015 10:36 AM, Oleg Fayans wrote: >> This is an error message that I received at the attempt to install >> replica with the following command: >> >> ipa-replica-install --setup-ca -p -w >> /var/lib/ipa/replica-info-replica2.justfor.test.gpg >> >> However, if I remove the '-w ', then I get the password >> prompt for admin password interactively. The domain level is 0. The >> packages are built last Friday from upstream code. >> > > This is a legitimate issue, can you file a ticket? Already done this https://fedorahosted.org/freeipa/ticket/5517 > > Thanks, > Tomas > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From ofayans at redhat.com Mon Dec 7 13:27:27 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Mon, 7 Dec 2015 14:27:27 +0100 Subject: [Freeipa-devel] [PATCH] ca-less tests updated - POC In-Reply-To: <563CA575.1030808@redhat.com> References: <56337745.109@redhat.com> <563B091B.3010501@redhat.com> <563B6A96.8090606@redhat.com> <563BABEC.8080304@redhat.com> <563C5B1A.1090803@redhat.com> <563C5E6D.4060307@redhat.com> <563CA575.1030808@redhat.com> Message-ID: <5665893F.40004@redhat.com> Anyone to review it guys? On 11/06/2015 02:04 PM, Oleg Fayans wrote: > Hi Jan, > > On 11/06/2015 09:01 AM, Jan Cholasta wrote: >> Actually it might be better to keep them, but fix them to expect >> ipa-server-certinstall to success. > > Done. Updated patch attached. > Also in the patch 0013 I removed a trailing whitespace which caused lint > to complain > > Now with domain level 0 the test output looks like this: > > [11:40:51]ofayans at vm-076:~]$ ipa-run-tests test_integration/test_caless.py > ==================================================================================== > test session starts > ===================================================================================== > > platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.6.4 > plugins: multihost, sourceorder > collected 88 items > > test_integration/test_caless.py > ......xx......ss...............xx........ss....................xx......ss............... > > > ===================================================================== 76 > passed, 6 skipped, 6 xfailed in 7871.10 seconds > ===================================================================== > > >> >> On 6.11.2015 08:47, Jan Cholasta wrote: >>> Hi Oleg, >>> >>> I think you can just remove >>> TestCertinstall.test_{http,ds}_intermediate_ca, the certificates are >>> imported correctly in this case and I didn't see anything break. >>> >>> Honza >>> >>> On 5.11.2015 20:20, Oleg Fayans wrote: >>>> Patch 0014 updated and passes lint >>>> >>>> On 11/05/2015 03:41 PM, Oleg Fayans wrote: >>>>> Wait a bit, the patch has problems with pylint: it does not build :) >>>>> The updated version (without the setupmaster nonsense) is being tested >>>>> now. >>>>> >>>>> On 11/05/2015 08:45 AM, Oleg Fayans wrote: >>>>>> Hi Jan, >>>>>> >>>>>> Could you take a look at these, whenever you are free? >>>>>> >>>>>> On 10/30/2015 02:57 PM, Oleg Fayans wrote: >>>>>>> Hi, >>>>>>> >>>>>>> The following patches contain updates to ca-less integration tests. >>>>>>> It's still a proof of concept: 2 tests still fail seemingly due to >>>>>>> the >>>>>>> change in target system logic (marked as xfail with "ask jcholast >>>>>>> comment") >>>>>>> >>>>>>> The test output looks like this: >>>>>>> >>>>>>> $ ipa-run-tests test_integration/test_caless.py --pdb >>>>>>> ==================================================================================== >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> test session starts >>>>>>> ===================================================================================== >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.6.4 >>>>>>> plugins: multihost, sourceorder >>>>>>> collected 88 items >>>>>>> >>>>>>> test_integration/test_caless.py >>>>>>> ......xx......ss............sssssssssssssssssss.ssssss.........xx......ssxx............. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> ==================================================================== >>>>>>> 53 >>>>>>> >>>>>>> passed, 29 skipped, 6 xfailed in 5620.17 seconds >>>>>>> ===================================================================== >>>>>>> >>>>>>> >>>>>>> Numerous skips correspond to the tests related to >>>>>>> ipa-replica-prepare >>>>>>> (unsupported under domain level 1) >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>>> >>> >>> >> >> > > > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From mbabinsk at redhat.com Mon Dec 7 13:31:25 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Mon, 7 Dec 2015 14:31:25 +0100 Subject: [Freeipa-devel] [PATCH 0364, 0367] ipa-kra-install: allow first KRA to be installed on replica In-Reply-To: <565F1B3B.5080905@redhat.com> References: <56587E92.7060008@redhat.com> <565C4C31.4070509@redhat.com> <565C877B.3050405@redhat.com> <565EF7A8.7050304@redhat.com> <565F1B3B.5080905@redhat.com> Message-ID: <56658A2D.2010803@redhat.com> On 12/02/2015 05:24 PM, Martin Basti wrote: > > > On 02.12.2015 14:52, Martin Babinsky wrote: >> On 11/30/2015 06:29 PM, Martin Basti wrote: >>> >>> >>> On 30.11.2015 14:16, Martin Babinsky wrote: >>>> On 11/27/2015 05:02 PM, Martin Basti wrote: >>>>> https://fedorahosted.org/freeipa/ticket/5460 >>>>> >>>>> I tested just master, I will test ipa-4-2 later. >>>>> patch attached. >>>>> >>>>> >>>> >>>> ACK for the master branch. >>>> >>> Thanks, additional patch improves error message when ipa-replica-install >>> --setup-ca --setup-kra is executed and KRA is not installed anywhere >>> yet. >>> >>> I'm working on patches for ipa-4-2 branch >>> >>> Martin >> >> ACK for patch 367. >> > Pushed to master: bbbe411f357b7fbad533b5211a90bb0558b1abbe > > IPA 4.2 patches attached. ACK. -- Martin^3 Babinsky From mbasti at redhat.com Mon Dec 7 13:32:14 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 7 Dec 2015 14:32:14 +0100 Subject: [Freeipa-devel] [PATCH 0372] CI: installation tests In-Reply-To: <56657A83.6020702@redhat.com> References: <5664A718.6080101@redhat.com> <56657A83.6020702@redhat.com> Message-ID: <56658A5E.8010603@redhat.com> On 07.12.2015 13:24, Oleg Fayans wrote: > Hi Martin, > > I would prefer both install_kra and install_ca methods to have > raiseonerr parameter set to True by default. We need a way to test > negatives and analyze results. > Mine looks like this: > > def install_kra(host, domain_level=None, > first_instance=False, raiseonerr=True): > if not domain_level: > domain_level = host.config.domain_level > command = ["ipa-kra-install", "-U", "-p", > host.config.dirman_password] > if domain_level == DOMAIN_LEVEL_0 and not first_instance: > replica_file = get_replica_filename(host) > command.append(replica_file) > return host.run_command(command, raiseonerr=raiseonerr) > > The rest looks good to me, but I did not run the tests yet. Sounds good, I will amend the patche later. > > On 12/06/2015 10:22 PM, Martin Basti wrote: >> My favorite today \o/ --> 67 <-- \o/ test cases, no more manual testing >> of installers \o/. >> >> Test suite contains: 6 combination how to install components on replica >> X 4 combinations of server installation + 3 extra server tests >> >> To save time tests install 1 master and 3 replicas per test class >> (except extra server tests): >> Class name specifies what is installed on master. >> >> Remember, option "-k" is your friend >> $ ipa-run-tests -k '' >> otherwise you will need a lot of time until tests finish. >> >> To list all tests: >> $ ipa-run-tests test_integration/test_installation.py --collect-only >> >> >> Patch attached, it needs to have attached all patches I sent today and >> also attached workaround patch (Martin3 will provide proper fix) >> >> I haven't had time/mood/resources to test this patch with domain level >> 0, so testing this patch with domain level 0 is appreciated. >> >> >> Oleg, I added some methods to tasks.py which you may want to reuse. >> >> Martin^2 >> From mbabinsk at redhat.com Mon Dec 7 13:32:30 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Mon, 7 Dec 2015 14:32:30 +0100 Subject: [Freeipa-devel] [PATCH 0364, 0367] ipa-kra-install: allow first KRA to be installed on replica In-Reply-To: <1448907874.3747.91.camel@redhat.com> References: <56587E92.7060008@redhat.com> <565C4C31.4070509@redhat.com> <565C877B.3050405@redhat.com> <1448907653.3747.90.camel@redhat.com> <565C93D8.4050007@redhat.com> <1448907874.3747.91.camel@redhat.com> Message-ID: <56658A6E.1040205@redhat.com> On 11/30/2015 07:24 PM, Simo Sorce wrote: > On Mon, 2015-11-30 at 19:22 +0100, Martin Basti wrote: >> >> On 30.11.2015 19:20, Simo Sorce wrote: >>> On Mon, 2015-11-30 at 18:29 +0100, Martin Basti wrote: >>>> On 30.11.2015 14:16, Martin Babinsky wrote: >>>>> On 11/27/2015 05:02 PM, Martin Basti wrote: >>>>>> https://fedorahosted.org/freeipa/ticket/5460 >>>>>> >>>>>> I tested just master, I will test ipa-4-2 later. >>>>>> patch attached. >>>>>> >>>>>> >>>>> ACK for the master branch. >>>>> >>>> Thanks, additional patch improves error message when ipa-replica-install >>>> --setup-ca --setup-kra is executed and KRA is not installed anywhere yet. >>>> >>>> I'm working on patches for ipa-4-2 branch >>> Question, shouldn't we just go ahead an install the first KRA ? >>> >>> Simo. >>> >> On meeting today Alexander told that this we do not want. >> Also Honza do not want to have KRA installed by default. > > Ok, I wasn't sure, fine by me. > > Simo. > The pacth 367 for master has been forgotten somehow. ACK. -- Martin^3 Babinsky From tbabej at redhat.com Mon Dec 7 13:33:41 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 7 Dec 2015 14:33:41 +0100 Subject: [Freeipa-devel] [PATCH 0393] replicainstall: Admin password should not conflict with Message-ID: <56658AB5.4020108@redhat.com> Hi, The --admin-password (-w) has its use both in domain level 0 and 1. https://fedorahosted.org/freeipa/ticket/5517 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0393-replicainstall-Admin-password-should-not-conflict-wi.patch Type: text/x-patch Size: 1093 bytes Desc: not available URL: From tbabej at redhat.com Mon Dec 7 13:35:44 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 7 Dec 2015 14:35:44 +0100 Subject: [Freeipa-devel] [PATCH 0393] replicainstall: Admin password should not conflict with In-Reply-To: <56658AB5.4020108@redhat.com> References: <56658AB5.4020108@redhat.com> Message-ID: <56658B30.20409@redhat.com> On 12/07/2015 02:33 PM, Tomas Babej wrote: > Hi, > > The --admin-password (-w) has its use both in domain level 0 and 1. > > https://fedorahosted.org/freeipa/ticket/5517 > > > ACK. Pushed to master: dcb6626e870bcededb62d801720721d5d6c9795f From mbasti at redhat.com Mon Dec 7 13:37:57 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 7 Dec 2015 14:37:57 +0100 Subject: [Freeipa-devel] [PATCH 0372] CI: installation tests In-Reply-To: <56658A5E.8010603@redhat.com> References: <5664A718.6080101@redhat.com> <56657A83.6020702@redhat.com> <56658A5E.8010603@redhat.com> Message-ID: <56658BB5.5060700@redhat.com> On 07.12.2015 14:32, Martin Basti wrote: > > > On 07.12.2015 13:24, Oleg Fayans wrote: >> Hi Martin, >> >> I would prefer both install_kra and install_ca methods to have >> raiseonerr parameter set to True by default. We need a way to test >> negatives and analyze results. >> Mine looks like this: >> >> def install_kra(host, domain_level=None, >> first_instance=False, raiseonerr=True): >> if not domain_level: >> domain_level = host.config.domain_level >> command = ["ipa-kra-install", "-U", "-p", >> host.config.dirman_password] >> if domain_level == DOMAIN_LEVEL_0 and not first_instance: >> replica_file = get_replica_filename(host) >> command.append(replica_file) >> return host.run_command(command, raiseonerr=raiseonerr) >> >> The rest looks good to me, but I did not run the tests yet. > Sounds good, I will amend the patche later. I changed my mind, should not be the domain_level value get from function domainlevel(host)? Martin^2 > >> >> On 12/06/2015 10:22 PM, Martin Basti wrote: >>> My favorite today \o/ --> 67 <-- \o/ test cases, no more manual testing >>> of installers \o/. >>> >>> Test suite contains: 6 combination how to install components on replica >>> X 4 combinations of server installation + 3 extra server tests >>> >>> To save time tests install 1 master and 3 replicas per test class >>> (except extra server tests): >>> Class name specifies what is installed on master. >>> >>> Remember, option "-k" is your friend >>> $ ipa-run-tests -k '' >>> otherwise you will need a lot of time until tests finish. >>> >>> To list all tests: >>> $ ipa-run-tests test_integration/test_installation.py --collect-only >>> >>> >>> Patch attached, it needs to have attached all patches I sent today and >>> also attached workaround patch (Martin3 will provide proper fix) >>> >>> I haven't had time/mood/resources to test this patch with domain level >>> 0, so testing this patch with domain level 0 is appreciated. >>> >>> >>> Oleg, I added some methods to tasks.py which you may want to reuse. >>> >>> Martin^2 >>> > From dkupka at redhat.com Mon Dec 7 13:41:29 2015 From: dkupka at redhat.com (David Kupka) Date: Mon, 7 Dec 2015 14:41:29 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <5665843C.2090507@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> <5665843C.2090507@redhat.com> Message-ID: <56658C89.8050103@redhat.com> On 07/12/15 14:06, David Kupka wrote: > On 09/09/15 13:39, Petr Spacek wrote: >> On 8.9.2015 16:30, David Kupka wrote: >>> On 28/08/15 13:36, Martin Basti wrote: >>>> >>>> >>>> On 08/28/2015 10:03 AM, Petr Spacek wrote: >>>>> On 27.8.2015 14:22, David Kupka wrote: >>>>>> @@ -2101,11 +2101,25 @@ class DNSZoneBase(LDAPObject): >>>>>> class DNSZoneBase_add(LDAPCreate): >>>>>> + takes_options = LDAPCreate.takes_options + ( >>>>>> + Flag('force', >>>>>> + label=_('Force'), >>>>>> + doc=_('Force DNS zone creation.') >>>>>> + ), >>>>>> + Flag('skip_overlap_check', >>>>>> + doc=_('Force DNS zone creation even if it will overlap >>>>>> with ' >>>>>> + 'existing zone.') >>>>>> + ), >>>>>> + ) >>>>>> + >>>>>> has_output_params = LDAPCreate.has_output_params + >>>>>> dnszone_output_params >>>>>> def pre_callback(self, ldap, dn, entry_attrs, attrs_list, >>>>>> *keys, **options): >>>>>> assert isinstance(dn, DN) >>>>>> + if options['force']: >>>>>> + options['skip_overlap_check'] = True >>>>>> + >>>>>> try: >>>>>> entry = ldap.get_entry(dn) >>>>>> except errors.NotFound: >>>>>> @@ -2120,6 +2134,12 @@ class DNSZoneBase_add(LDAPCreate): >>>>>> entry_attrs['idnszoneactive'] = 'TRUE' >>>>>> + if not options['skip_overlap_check']: >>>>>> + try: >>>>>> + check_zone_overlap(keys[-1]) >>>>>> + except RuntimeError as e: >>>>>> + raise errors.InvocationError(e.message) >>>>>> + >>>>>> return dn >>>>>> @@ -2673,9 +2693,9 @@ class dnszone_add(DNSZoneBase_add): >>>>>> __doc__ = _('Create new DNS zone (SOA record).') >>>>>> takes_options = DNSZoneBase_add.takes_options + ( >>>>>> - Flag('force', >>>>>> - label=_('Force'), >>>>>> - doc=_('Force DNS zone creation even if nameserver is >>>>>> not resolvable.'), >>>>>> + Flag('skip_nameserver_check', >>>>>> + doc=_('Force DNS zone creation even if nameserver is >>>>>> not ' >>>>>> + 'resolvable.') >>>>>> ), >>>>>> # Deprecated >>>>>> @@ -2699,6 +2719,9 @@ class dnszone_add(DNSZoneBase_add): >>>>>> def pre_callback(self, ldap, dn, entry_attrs, attrs_list, >>>>>> *keys, **options): >>>>>> assert isinstance(dn, DN) >>>>>> + if options['force']: >>>>>> + options['skip_nameserver_check'] = True >>>>> Why is it in DNSZoneBase_add.pre_callback? >>>>> >>>>> Shouldn't the equation force = (skip_nameserver_check + >>>>> skip_nameserver_check) >>>>> be handled in parameter parsing & validation? (Again, I do not know >>>>> the IPA >>>>> framework :-)) >>>>> >>>>>> + >>>>>> dn = super(dnszone_add, self).pre_callback( >>>>>> ldap, dn, entry_attrs, attrs_list, *keys, **options) >>>>>> @@ -2713,7 +2736,7 @@ class dnszone_add(DNSZoneBase_add): >>>>>> error=_("Nameserver for reverse zone cannot be >>>>>> a relative DNS name")) >>>>>> # verify if user specified server is resolvable >>>>>> - if not options['force']: >>>>>> + if not options['skip_nameserver_check']: >>>>>> check_ns_rec_resolvable(keys[0], >>>>>> entry_attrs['idnssoamname']) >>>>>> # show warning about --name-server option >>>>>> context.show_warning_nameserver_option = True >>>>>> diff --git a/ipapython/ipautil.py b/ipapython/ipautil.py >>>>>> index >>>>>> d959bb369d946217acd080e78483cc9013dda4c7..18f477d4fb6620090b7073689c8df51b65a8307a >>>>>> >>>>>> >>>>>> 100644 >>>>>> --- a/ipapython/ipautil.py >>>>>> +++ b/ipapython/ipautil.py >>>>>> @@ -924,6 +924,20 @@ def host_exists(host): >>>>>> else: >>>>>> return True >>>>>> +def check_zone_overlap(zone): >>>>>> + if resolver.zone_for_name(zone) == zone: >>>>>> + try: >>>>>> + ns = [ans.to_text() for ans in resolver.query(zone, >>>>>> 'NS')] >>>>>> + except DNSException as e: >>>>>> + root_logger.debug("Failed to resolve nameserver(s) for >>>>>> domain" >>>>>> + " {0}: {1}".format(zone, e)) >>>>>> + ns = [] >>>>>> + >>>>>> + msg = u"DNS zone {0} already exists".format(zone) >>>>> Nitpick: I would say "already exists in DNS" to make it absolutely >>>>> clear. Just >>>>> 'already exists' might be confusing because ipa dnszone-show will say >>>>> that the >>>>> zone does not exist ... >>>>> >>>>>> + if ns: >>>>>> + msg += u" and is handled by server(s): {0}".format(', >>>>>> '.join(ns)) >>>>>> + raise RuntimeError(msg) >>>>>> + >>>>>> def get_ipa_basedn(conn): >>>>>> """ >>>>>> Get base DN of IPA suffix in given LDAP server. >>>> 0064 >>>> NACK >>>> >>>> ipa-replica-install should have the --skip-overlap-check too, because >>>> any replica can be the first DNS server. >>> >>> Thanks for the catch, added. >>> >>>> >>>> 0064+0058 >>>> Can be the options --allow-zone-overlap and --skip-overlap-check merged >>>> into an one name, to have just one name for same thing? >>>> >>> >>> Each option has bit different behavior: >>> The '--skip-overlap-check' option in API call prevent the check to be >>> performed and thus no error or warning is raised. This is the way >>> '--force' >>> option was originally working. >>> >>> The '--allow-zone-overlap' options in installers do not skip the >>> check but >>> change the error to warning instead and let the installation continue. >>> >>> If you think that this can confuse users we need to change the names >>> or even >>> the logic. >>> >>> Updated patches attached. >> >> Hello, >> >> thank you very much for updating the patch. >> >> Unfortunately it is not yet ready, but we are getting there. >> >> >> * Serious problems: >> >> a) ipa-server/replica/dns-install by default creates reverse zones >> even if >> these zones exist. The check which is done for main IPA domain should >> be done >> for all reverse zones, too. If a zone exists user should use >> --no-reverse or >> --allow-zone-overlap if necessary. >> >> I believe that this is in scope of >> https://fedorahosted.org/freeipa/ticket/3681 . >> >> >> b) ipa-server/replica/dns-install by default always add DNS zone which >> contains hostname of the IPA server, even if the IPA domain is >> different and >> even if the DNS zone already exists. >> >> If the hostname of the server does not belong to IPA domain we should >> not add >> anything. Just check that the hostname is resolvable. Theoretically we >> could >> add an option which explicitly asks for creating zone which encloses the >> hostnmame. >> >> I believe that this is in scope of >> https://fedorahosted.org/freeipa/ticket/5087 . >> >> >> c) I did not realize that checks in ipa dns*zone-add will break zone >> addition >> for automatic empty zones. >> >> There is list of special-case DNS zones which need special handling: >> https://source.isc.org/cgi-bin/gitweb.cgi?p=bind9.git;a=blob;f=bin/named/server.c;h=1a25531df354f6f0593bfb86e5aa3e3c9b9c80e5;hb=HEAD#l272 >> >> >> Feel free to copy&paste list of the domains to IPA code - list comes from >> RFCs, so it should be pretty stable. It would be good >> >> If one of these zones is detected as "already existing" we need to >> further >> check values in SOA record and automatically override the check if: >> (SOA mname = zone name) & (SOA rname = .) >> >> In that case we should print informational message "automatic empty >> zone will >> be overridden by the zone you defined now". >> >> >> d) ipa-server/replica/dns-install fails when attempt to resolve main IPA >> domain name fails with DNS timeout. This might easily happen if the >> domain >> name is already delegated to IPA server which is being installed. In >> that case >> we should print a warning and allow to continue. >> >> """ >> Warning: DNS check for domain failed with error . >> Please make sure that the domain is properly delegated to this IPA >> server. >> """ >> >> >> * Nitpicks - can be handled in separate patches: >> 1) Interactive ipa-server-install tells the user that the zone name he >> entered >> exists too late in the process. >> >> I.e. user has to enter DM and admin password (twice!) and only after >> that he >> will receive an error, which may be frustrating :-) >> >> Please move the check right after DNS zone name input so user does not >> waste >> his time and nerves :-) Even better, in an interactive mode it could make >> sense to ask the user again if the previous input was not acceptable. >> >> >> 2) I found out that ipa-server-install prints message "Warning: >> skipping DNS >> resolution of host vm-206.abc.idm.lab.eng.brq.redhat.com" even if the >> host >> name does not belong to IPA domain. That is a mistake - the check >> should be >> skipped only in case when IPA server is part of the IPA domain. >> >> >> 3) When adding an forward as an override for automatic empty zones we >> should >> throw a warning if forward policy is not 'only'. It does not make >> sense to use >> policy 'forward' in these cases. >> """ >> Warning: It is strongly recommended to use forward policy 'only' for the >> forward zones defined for private use. >> """ >> > > Hi! > I was finally able to get back to this patches. I've (hopefully) > addressed all the items from "Serious problems" section plus the first > "nitpick" as I found it really annoying. > Updated patches attached. Patch freeipa-dkupka-0070 is needed for this > to work. > > Oh, stupid me, I just dropped is_host_resolvable() during patch rebase. Updated patch attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0058.7-dns-do-not-add-forward-zone-if-it-is-already-resolva.patch Type: text/x-patch Size: 12374 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0064.4-dns-Check-if-domain-already-exists.patch Type: text/x-patch Size: 12344 bytes Desc: not available URL: From redhatrises at gmail.com Mon Dec 7 13:55:10 2015 From: redhatrises at gmail.com (Gabe Alford) Date: Mon, 7 Dec 2015 06:55:10 -0700 Subject: [Freeipa-devel] [PATCH 0065] In-Reply-To: References: Message-ID: Bump for review. On Mon, Nov 30, 2015 at 7:31 PM, Gabe Alford wrote: > Hello, > > Patch fix for the following tickets: > > https://fedorahosted.org/freeipa/ticket/5022 > https://fedorahosted.org/freeipa/ticket/5320 > > Thanks, > > Gabe > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbasti at redhat.com Mon Dec 7 13:58:47 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 7 Dec 2015 14:58:47 +0100 Subject: [Freeipa-devel] [PATCH 0364, 0367] ipa-kra-install: allow first KRA to be installed on replica In-Reply-To: <56658A6E.1040205@redhat.com> References: <56587E92.7060008@redhat.com> <565C4C31.4070509@redhat.com> <565C877B.3050405@redhat.com> <1448907653.3747.90.camel@redhat.com> <565C93D8.4050007@redhat.com> <1448907874.3747.91.camel@redhat.com> <56658A6E.1040205@redhat.com> Message-ID: <56659097.5030302@redhat.com> On 07.12.2015 14:32, Martin Babinsky wrote: > On 11/30/2015 07:24 PM, Simo Sorce wrote: >> On Mon, 2015-11-30 at 19:22 +0100, Martin Basti wrote: >>> >>> On 30.11.2015 19:20, Simo Sorce wrote: >>>> On Mon, 2015-11-30 at 18:29 +0100, Martin Basti wrote: >>>>> On 30.11.2015 14:16, Martin Babinsky wrote: >>>>>> On 11/27/2015 05:02 PM, Martin Basti wrote: >>>>>>> https://fedorahosted.org/freeipa/ticket/5460 >>>>>>> >>>>>>> I tested just master, I will test ipa-4-2 later. >>>>>>> patch attached. >>>>>>> >>>>>>> >>>>>> ACK for the master branch. >>>>>> >>>>> Thanks, additional patch improves error message when >>>>> ipa-replica-install >>>>> --setup-ca --setup-kra is executed and KRA is not installed >>>>> anywhere yet. >>>>> >>>>> I'm working on patches for ipa-4-2 branch >>>> Question, shouldn't we just go ahead an install the first KRA ? >>>> >>>> Simo. >>>> >>> On meeting today Alexander told that this we do not want. >>> Also Honza do not want to have KRA installed by default. >> >> Ok, I wasn't sure, fine by me. >> >> Simo. >> > The pacth 367 for master has been forgotten somehow. ACK. > Nope. Both has been pushed. git show bbbe411f357b7fbad533b5211a90bb0558b1abbe git show bbbe411f357b7fbad533b5211a90bb0558b1abbe^ From mbasti at redhat.com Mon Dec 7 14:20:05 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 7 Dec 2015 15:20:05 +0100 Subject: [Freeipa-devel] [PATCH 0065] In-Reply-To: References: Message-ID: <56659595.8050708@redhat.com> On 07.12.2015 14:55, Gabe Alford wrote: > Bump for review. > > On Mon, Nov 30, 2015 at 7:31 PM, Gabe Alford > wrote: > > Hello, > > Patch fix for the following tickets: > > https://fedorahosted.org/freeipa/ticket/5022 > https://fedorahosted.org/freeipa/ticket/5320 > > Thanks, > > Gabe > > > > Hello, IIRC you said that we should ignore this in thread [PATCH 0065] ipa-replica-install prints incorrect error message when replica is already installed It is on my TODO list. Martin^2 -------------- next part -------------- An HTML attachment was scrubbed... URL: From amarecek at redhat.com Mon Dec 7 14:29:31 2015 From: amarecek at redhat.com (=?utf-8?Q?Ale=C5=A1_Mare=C4=8Dek?=) Date: Mon, 7 Dec 2015 09:29:31 -0500 (EST) Subject: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin In-Reply-To: <20151203173843.37179230@vor2.netbox.priv> References: <20151106113201.5e6a7042@dhcp-24-123.brq.redhat.com> <564092F6.9000800@redhat.com> <20151109163512.3d3beb5f@redhat.com> <5641BE6D.4060608@redhat.com> <20151110121350.06295a68@dhcp-24-103.brq.redhat.com> <56448E17.2010409@redhat.com> <20151203173843.37179230@vor2.netbox.priv> Message-ID: <869710129.24860544.1449498571599.JavaMail.zimbra@redhat.com> NACK. $ ./make-lint ************* Module ipatests.test_xmlrpc.test_user_plugin ipatests/test_xmlrpc/test_user_plugin.py:42: [E0611(no-name-in-module), ] No name 'ldaptracker' in module 'ipatests.test_xmlrpc') $ grep ldaptracker ipatests/test_xmlrpc/test_user_plugin.py from ipatests.test_xmlrpc.ldaptracker import Tracker $ ls ipatests/test_xmlrpc/ldaptracker* ls: cannot access ipatests/test_xmlrpc/ldaptracker*: No such file or directory ----- Original Message ----- > From: "Filip ?kola" > To: "Milan Kub?k" > Cc: freeipa-devel at redhat.com > Sent: Thursday, December 3, 2015 5:38:43 PM > Subject: Re: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin > > Hi, > > sending corrected version. > > F. > > On Thu, 12 Nov 2015 14:03:19 +0100 > Milan Kub?k wrote: > > > On 11/10/2015 12:13 PM, Filip ?kola wrote: > > > Hi, > > > > > > fixed. > > > > > > F. > > > > > > On Tue, 10 Nov 2015 10:52:45 +0100 > > > Milan Kub?k wrote: > > > > > >> On 11/09/2015 04:35 PM, Filip ?kola wrote: > > >>> Another patch was applied in the meantime. > > >>> > > >>> Attaching an updated version. > > >>> > > >>> F. > > >>> > > >>> On Mon, 9 Nov 2015 13:35:02 +0100 > > >>> Milan Kub?k wrote: > > >>> > > >>>> On 11/06/2015 11:32 AM, Filip ?kola wrote: > > >>>> Hi, > > >>>> the patch doesn't apply. > > >>>> > > >> Please fix this. > > >> > > >> ipatests/test_xmlrpc/test_user_plugin.py:1419: > > >> [E0602(undefined-variable), > > >> TestDeniedBindWithExpiredPrincipal.teardown_class] Undefined > > >> variable 'user1') > > >> > > >> Also, use the version numbers for your changed patches. > > >> > > > > > > > > Thanks for the patch. Several issues: > > > > 1. Use dict.items instead of dict.iteritems, for python3 compatibility > > > > 2. What is the purpose of TestPrepare class? The 'purge' methods do > > not call any ipa commands. > > Tracker.make_fixture should be used to make the Tracked resources > > clean themselves up when they're out of scope. > > > > 3. Why reference the resources by hardcoded name if they have a > > fixture representation? > > > > 4. Rewrite {create,delete}_test_group to a fixture. You may want to > > use different scope (or not). > > > > 5. In `def atest_rename_to_invalid_login(self, user):` - use > > pytest.skipif decorator and provide a reason if you must, > > do not obfuscate method name in order not to run it. > > > > > > > -- > Manage your subscription for the Freeipa-devel mailing list: > https://www.redhat.com/mailman/listinfo/freeipa-devel > Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code From cheimes at redhat.com Mon Dec 7 14:37:50 2015 From: cheimes at redhat.com (Christian Heimes) Date: Mon, 7 Dec 2015 15:37:50 +0100 Subject: [Freeipa-devel] [PATCH 0026] Workarounds for SELinux execmem violations in cryptography Message-ID: <566599BE.1020008@redhat.com> The patch fixes SELinux violations in Fedora 23. Background: Recent versions of cryptography cause SELinux violation which will lead to a segfault, see https://bugzilla.redhat.com/show_bug.cgi?id=1277224 . The segfault only occurs in the context of Apache HTTPD (FreeIPA web ui) when cryptography.hazmat.backends.default_backend() is initialized. I'm working on a fix for cryptography but it will take a while. First I have to wait for a new upstream release of python-cffi. Armin Ronacher plans to release cffi 1.4 in two weeks. ipaserver.dcerpc uses M2Crypto again on Python 2.7 and Dogtag's pki.client no longer tries to use PyOpenSSL instead of Python's ssl module. Some dependencies like Dogtag's pki.client library and custodia use python-requsts to make HTTPS connection. python-requests prefers PyOpenSSL over Python's stdlib ssl module. PyOpenSSL is build on top of python-cryptography which trigger a execmem SELinux violation in the context of Apache HTTPD (httpd_execmem). When requests is imported, it always tries to import pyopenssl glue code from urllib3's contrib directory. The import of PyOpenSSL is enough to trigger the SELinux denial. A hack in wsgi.py prevents the import by raising an ImportError. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-cheimes-0026-Workarounds-for-SELinux-execmem-violations-in-crypto.patch Type: text/x-patch Size: 4967 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From redhatrises at gmail.com Mon Dec 7 14:40:01 2015 From: redhatrises at gmail.com (Gabe Alford) Date: Mon, 7 Dec 2015 07:40:01 -0700 Subject: [Freeipa-devel] [PATCH 0065] In-Reply-To: <56659595.8050708@redhat.com> References: <56659595.8050708@redhat.com> Message-ID: Yup you are right. I meant to bump the other one. > It is on my TODO list. Awesome. On Mon, Dec 7, 2015 at 7:20 AM, Martin Basti wrote: > > > On 07.12.2015 14:55, Gabe Alford wrote: > > Bump for review. > > On Mon, Nov 30, 2015 at 7:31 PM, Gabe Alford > wrote: > >> Hello, >> >> Patch fix for the following tickets: >> >> https://fedorahosted.org/freeipa/ticket/5022 >> https://fedorahosted.org/freeipa/ticket/5320 >> >> Thanks, >> >> Gabe >> > > > > Hello, IIRC you said that we should ignore this in thread > [PATCH 0065] ipa-replica-install prints incorrect error message when > replica is already installed > > It is on my TODO list. > > Martin^2 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ofayans at redhat.com Mon Dec 7 14:49:17 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Mon, 7 Dec 2015 15:49:17 +0100 Subject: [Freeipa-devel] [PATCH 0372] CI: installation tests In-Reply-To: <56658BB5.5060700@redhat.com> References: <5664A718.6080101@redhat.com> <56657A83.6020702@redhat.com> <56658A5E.8010603@redhat.com> <56658BB5.5060700@redhat.com> Message-ID: <56659C6D.4040600@redhat.com> Hi, On 12/07/2015 02:37 PM, Martin Basti wrote: > > > On 07.12.2015 14:32, Martin Basti wrote: >> >> >> On 07.12.2015 13:24, Oleg Fayans wrote: >>> Hi Martin, >>> >>> I would prefer both install_kra and install_ca methods to have >>> raiseonerr parameter set to True by default. We need a way to test >>> negatives and analyze results. >>> Mine looks like this: >>> >>> def install_kra(host, domain_level=None, >>> first_instance=False, raiseonerr=True): >>> if not domain_level: >>> domain_level = host.config.domain_level >>> command = ["ipa-kra-install", "-U", "-p", >>> host.config.dirman_password] >>> if domain_level == DOMAIN_LEVEL_0 and not first_instance: >>> replica_file = get_replica_filename(host) >>> command.append(replica_file) >>> return host.run_command(command, raiseonerr=raiseonerr) >>> >>> The rest looks good to me, but I did not run the tests yet. >> Sounds good, I will amend the patche later. > > I changed my mind, should not be the domain_level value get from > function domainlevel(host)? We should have a way to test negatives, like providing replica file at domain level=1 and not providing at domain level=0. So these functions should either accept replica file as a parameter, or arbitrary domain level > Martin^2 > >> >>> >>> On 12/06/2015 10:22 PM, Martin Basti wrote: >>>> My favorite today \o/ --> 67 <-- \o/ test cases, no more manual testing >>>> of installers \o/. >>>> >>>> Test suite contains: 6 combination how to install components on replica >>>> X 4 combinations of server installation + 3 extra server tests >>>> >>>> To save time tests install 1 master and 3 replicas per test class >>>> (except extra server tests): >>>> Class name specifies what is installed on master. >>>> >>>> Remember, option "-k" is your friend >>>> $ ipa-run-tests -k '' >>>> otherwise you will need a lot of time until tests finish. >>>> >>>> To list all tests: >>>> $ ipa-run-tests test_integration/test_installation.py --collect-only >>>> >>>> >>>> Patch attached, it needs to have attached all patches I sent today and >>>> also attached workaround patch (Martin3 will provide proper fix) >>>> >>>> I haven't had time/mood/resources to test this patch with domain level >>>> 0, so testing this patch with domain level 0 is appreciated. >>>> >>>> >>>> Oleg, I added some methods to tasks.py which you may want to reuse. >>>> >>>> Martin^2 >>>> >> > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From mbasti at redhat.com Mon Dec 7 14:51:14 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 7 Dec 2015 15:51:14 +0100 Subject: [Freeipa-devel] [PATCH 0372] CI: installation tests In-Reply-To: <56659C6D.4040600@redhat.com> References: <5664A718.6080101@redhat.com> <56657A83.6020702@redhat.com> <56658A5E.8010603@redhat.com> <56658BB5.5060700@redhat.com> <56659C6D.4040600@redhat.com> Message-ID: <56659CE2.1060304@redhat.com> On 07.12.2015 15:49, Oleg Fayans wrote: > Hi, > > On 12/07/2015 02:37 PM, Martin Basti wrote: >> >> On 07.12.2015 14:32, Martin Basti wrote: >>> >>> On 07.12.2015 13:24, Oleg Fayans wrote: >>>> Hi Martin, >>>> >>>> I would prefer both install_kra and install_ca methods to have >>>> raiseonerr parameter set to True by default. We need a way to test >>>> negatives and analyze results. >>>> Mine looks like this: >>>> >>>> def install_kra(host, domain_level=None, >>>> first_instance=False, raiseonerr=True): >>>> if not domain_level: >>>> domain_level = host.config.domain_level >>>> command = ["ipa-kra-install", "-U", "-p", >>>> host.config.dirman_password] >>>> if domain_level == DOMAIN_LEVEL_0 and not first_instance: >>>> replica_file = get_replica_filename(host) >>>> command.append(replica_file) >>>> return host.run_command(command, raiseonerr=raiseonerr) >>>> >>>> The rest looks good to me, but I did not run the tests yet. >>> Sounds good, I will amend the patche later. >> I changed my mind, should not be the domain_level value get from >> function domainlevel(host)? > We should have a way to test negatives, like providing replica file at > domain level=1 and not providing at domain level=0. So these functions > should either accept replica file as a parameter, or arbitrary domain level Agree, but I meant this if not domain_level: - domain_level = host.config.domain_level + domain_level = domainlevel(host) > >> Martin^2 >> >>>> On 12/06/2015 10:22 PM, Martin Basti wrote: >>>>> My favorite today \o/ --> 67 <-- \o/ test cases, no more manual testing >>>>> of installers \o/. >>>>> >>>>> Test suite contains: 6 combination how to install components on replica >>>>> X 4 combinations of server installation + 3 extra server tests >>>>> >>>>> To save time tests install 1 master and 3 replicas per test class >>>>> (except extra server tests): >>>>> Class name specifies what is installed on master. >>>>> >>>>> Remember, option "-k" is your friend >>>>> $ ipa-run-tests -k '' >>>>> otherwise you will need a lot of time until tests finish. >>>>> >>>>> To list all tests: >>>>> $ ipa-run-tests test_integration/test_installation.py --collect-only >>>>> >>>>> >>>>> Patch attached, it needs to have attached all patches I sent today and >>>>> also attached workaround patch (Martin3 will provide proper fix) >>>>> >>>>> I haven't had time/mood/resources to test this patch with domain level >>>>> 0, so testing this patch with domain level 0 is appreciated. >>>>> >>>>> >>>>> Oleg, I added some methods to tasks.py which you may want to reuse. >>>>> >>>>> Martin^2 >>>>> From ofayans at redhat.com Mon Dec 7 14:51:58 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Mon, 7 Dec 2015 15:51:58 +0100 Subject: [Freeipa-devel] [PATCH 0372] CI: installation tests In-Reply-To: <56659CE2.1060304@redhat.com> References: <5664A718.6080101@redhat.com> <56657A83.6020702@redhat.com> <56658A5E.8010603@redhat.com> <56658BB5.5060700@redhat.com> <56659C6D.4040600@redhat.com> <56659CE2.1060304@redhat.com> Message-ID: <56659D0E.5060806@redhat.com> On 12/07/2015 03:51 PM, Martin Basti wrote: > > > On 07.12.2015 15:49, Oleg Fayans wrote: >> Hi, >> >> On 12/07/2015 02:37 PM, Martin Basti wrote: >>> >>> On 07.12.2015 14:32, Martin Basti wrote: >>>> >>>> On 07.12.2015 13:24, Oleg Fayans wrote: >>>>> Hi Martin, >>>>> >>>>> I would prefer both install_kra and install_ca methods to have >>>>> raiseonerr parameter set to True by default. We need a way to test >>>>> negatives and analyze results. >>>>> Mine looks like this: >>>>> >>>>> def install_kra(host, domain_level=None, >>>>> first_instance=False, raiseonerr=True): >>>>> if not domain_level: >>>>> domain_level = host.config.domain_level >>>>> command = ["ipa-kra-install", "-U", "-p", >>>>> host.config.dirman_password] >>>>> if domain_level == DOMAIN_LEVEL_0 and not first_instance: >>>>> replica_file = get_replica_filename(host) >>>>> command.append(replica_file) >>>>> return host.run_command(command, raiseonerr=raiseonerr) >>>>> >>>>> The rest looks good to me, but I did not run the tests yet. >>>> Sounds good, I will amend the patche later. >>> I changed my mind, should not be the domain_level value get from >>> function domainlevel(host)? >> We should have a way to test negatives, like providing replica file at >> domain level=1 and not providing at domain level=0. So these functions >> should either accept replica file as a parameter, or arbitrary domain >> level > Agree, but I meant this > > if not domain_level: > - domain_level = host.config.domain_level > + domain_level = domainlevel(host) Oh, yes, you are right, that's better. > > >> >>> Martin^2 >>> >>>>> On 12/06/2015 10:22 PM, Martin Basti wrote: >>>>>> My favorite today \o/ --> 67 <-- \o/ test cases, no more manual >>>>>> testing >>>>>> of installers \o/. >>>>>> >>>>>> Test suite contains: 6 combination how to install components on >>>>>> replica >>>>>> X 4 combinations of server installation + 3 extra server tests >>>>>> >>>>>> To save time tests install 1 master and 3 replicas per test class >>>>>> (except extra server tests): >>>>>> Class name specifies what is installed on master. >>>>>> >>>>>> Remember, option "-k" is your friend >>>>>> $ ipa-run-tests -k '' >>>>>> otherwise you will need a lot of time until tests finish. >>>>>> >>>>>> To list all tests: >>>>>> $ ipa-run-tests test_integration/test_installation.py --collect-only >>>>>> >>>>>> >>>>>> Patch attached, it needs to have attached all patches I sent today >>>>>> and >>>>>> also attached workaround patch (Martin3 will provide proper fix) >>>>>> >>>>>> I haven't had time/mood/resources to test this patch with domain >>>>>> level >>>>>> 0, so testing this patch with domain level 0 is appreciated. >>>>>> >>>>>> >>>>>> Oleg, I added some methods to tasks.py which you may want to reuse. >>>>>> >>>>>> Martin^2 >>>>>> > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From ofayans at redhat.com Mon Dec 7 14:55:01 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Mon, 7 Dec 2015 15:55:01 +0100 Subject: [Freeipa-devel] [PATCH 0370] CI: test various topologies with 3 replicas In-Reply-To: <5664A21D.7040802@redhat.com> References: <5664A21D.7040802@redhat.com> Message-ID: <56659DC5.9040901@redhat.com> ACK On 12/06/2015 10:01 PM, Martin Basti wrote: > Patch attached, to work properly it requires all patches I sent today + > workaround patch attached (Martin3 will provide proper fix). > > The last two test are failing due to bug in test framework > (ipa-replica-manage should not be used with domain level1) > > Martin^2 -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From mbabinsk at redhat.com Mon Dec 7 15:05:12 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Mon, 7 Dec 2015 16:05:12 +0100 Subject: [Freeipa-devel] [PATCH 0111] prevent crashes of server uninstall check caused by failed, 5 LDAP connections In-Reply-To: <5661EE41.1090900@redhat.com> References: <5661EA3A.3070504@redhat.com> <5661EE41.1090900@redhat.com> Message-ID: <5665A028.3010905@redhat.com> On 12/04/2015 08:49 PM, Rob Crittenden wrote: > Martin Babinsky wrote: >> https://fedorahosted.org/freeipa/ticket/5409 > > Should it also warn about the potential loss of the DNSSEC master? > > rob > Probably, but that warrants a separate ticket IMHO. IIRC these checks are a part of replica deletion code at domain level 1 in ipa-replica-manage, so it should be possible to transplant them on server uninstaller. -- Martin^3 Babinsky From abokovoy at redhat.com Mon Dec 7 15:17:47 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 7 Dec 2015 17:17:47 +0200 Subject: [Freeipa-devel] [PATCH 0026] Workarounds for SELinux execmem violations in cryptography In-Reply-To: <566599BE.1020008@redhat.com> References: <566599BE.1020008@redhat.com> Message-ID: <20151207151747.GZ8374@redhat.com> On Mon, 07 Dec 2015, Christian Heimes wrote: >The patch fixes SELinux violations in Fedora 23. > >Background: Recent versions of cryptography cause SELinux violation >which will lead to a segfault, see >https://bugzilla.redhat.com/show_bug.cgi?id=1277224 . The segfault only >occurs in the context of Apache HTTPD (FreeIPA web ui) when >cryptography.hazmat.backends.default_backend() is initialized. I'm >working on a fix for cryptography but it will take a while. First I have >to wait for a new upstream release of python-cffi. Armin Ronacher plans >to release cffi 1.4 in two weeks. > > >ipaserver.dcerpc uses M2Crypto again on Python 2.7 and Dogtag's >pki.client no longer tries to use PyOpenSSL instead of Python's ssl >module. > >Some dependencies like Dogtag's pki.client library and custodia use >python-requsts to make HTTPS connection. python-requests prefers >PyOpenSSL over Python's stdlib ssl module. PyOpenSSL is build on top >of python-cryptography which trigger a execmem SELinux violation >in the context of Apache HTTPD (httpd_execmem). >When requests is imported, it always tries to import pyopenssl glue >code from urllib3's contrib directory. The import of PyOpenSSL is >enough to trigger the SELinux denial. >A hack in wsgi.py prevents the import by raising an ImportError. ACK. Thanks for these patches. Note to Debian/Ubuntu maintainers: AppArmor 'support' in python-cffi already detects apparmor by looking into /proc and disabling the use of writeable and executable memory. On those platforms I suspect recent enough python-cryptography would work without problem by downgrading own feature set. The code in this patches should be harmless, though. -- / Alexander Bokovoy From jcholast at redhat.com Mon Dec 7 15:21:39 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 7 Dec 2015 16:21:39 +0100 Subject: [Freeipa-devel] [PATCHES 516-517] spec file: put Python modules into standalone packages Message-ID: <5665A403.9090604@redhat.com> Hi, the attached patches partially fix . This is done to allow the addition of Python 3 packages, see . See commit messages for more information. In order to test: 1. make rpms 2. 3. Test with both dnf and yum-deprecated. Beware that when you run "yum-deprecated clean all", it does not remove cache for the on-disk repository created in step 2, you have to remove the /var/cache/yum/$basearch/$releasever/$reponame directory manually. Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-516.1-spec-file-remove-config-files-from-freeipa-python.patch Type: text/x-patch Size: 3642 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-517.1-spec-file-put-Python-modules-into-standalone-package.patch Type: text/x-patch Size: 27868 bytes Desc: not available URL: From jcholast at redhat.com Mon Dec 7 15:22:54 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 7 Dec 2015 16:22:54 +0100 Subject: [Freeipa-devel] [PATCH] 0749 Package ipapython, ipalib, ipaplatform, ipatests for Python 3 In-Reply-To: <565EE63E.6070207@redhat.com> References: <563C9810.1050604@redhat.com> <5656E487.8050703@redhat.com> <5658508D.1020105@redhat.com> <565C0FF3.8040406@redhat.com> <565D80B0.9000500@redhat.com> <565DA287.6030403@redhat.com> <565EE63E.6070207@redhat.com> Message-ID: <5665A44E.3060203@redhat.com> On 2.12.2015 13:38, Petr Viktorin wrote: > On 12/01/2015 02:37 PM, Jan Cholasta wrote: > [...] >> /etc/ipa/default.conf is managed by freeipa-client and thus should be owned by it. >> >> This is a common pattern in other packages (even other FreeIPA >> sub-packages) and I don't see any reason not to follow it here as well. > > OK. After your patch is applied this won't be a problem, though. Turns out we forgot about translation files, so -common subpackage is actually necessary. > > [...] >>>>> - python-ipap11helper has compiled code: with this pulled out, >>>>> python3-ipalib can be noarch >>>> >>>> This is not the goal here, but if you insist on doing it, do it for >>>> Python 2 as well. >>> >>> It is definitely the goal of this patch to make the py3 packages as good >>> as possible. That includes making them noarch. >> >> It is completely unnecessary for the initial py3 support. >> >> I would rather maintain internal consistency than make the py3 packages >> "perfect". >> >>> On the other hand improving the py2 packages is not a goal of this >>> particular patch. >> >> Which is exactly the reason I have provided patches for py2 packages >> myself. > > As far as I'm concerned, the patches look good, except for consistency > the package name should be "python2-ipalib". Fixed. > Unless the process changed, you still need them reviewed by a core > FreeIPA developer. > >>>>> - python3-ipatests is needed if we want to start testing the py3 >>>>> packages in CI >>>> >>>> Right. >>>> >>>>> >>>>> As for new provides, Fedora's Python packaging guidelines say: >>>>> >>>>> """ >>>>> Using a fictional module named "example", the subpackage containing >>>>> the python2 version must provide python2-example. This is of course >>>>> always the case if the subpackage is named python2-example [...] >>>>> If the subpackage has some other name then then Provides: >>>>> python2-example >>>>> must be added explicitly (but see the %python_provide macro below). >>>>> >>>>> The python3 subpackage must provide python3-example. However, as the >>>>> naming guidelines mandate that the python3 subpackage be named >>>>> python3-example, this will happen automatically. >>>>> """ >>>>> >>>>> so I'm now adding Provides for the top-level modules. >>>> >>>> The goal of this work is to add support for Python 3, not to comply with >>>> Fedora packaging guidelines. FreeIPA on Fedora uses its own spec file >>>> anyway. >>> >>> The goal of this patch is to add new packages that support Python 3. >>> Yes, the Fedora spec is different, but it's heavily based on the >>> upstream one, and this is a good thing. I consider the Fedora guidelines >>> the standard in Python RPM packaging. If IPA uses different packaging >>> guidelines, can you point me to them? >> >> FreeIPA never fully complied to Fedora packaging guidelines AFAIK and I >> don't see any reason to start now, since nobody seemed to care so far. >> Following them in just py3 sub-packages does not improve the state of >> FreeIPA as a whole and only brings inconsistency into it, so there's no >> benefit in doing it at all. >> >>>> Again, if you insist on doing this, do it for Python 2 as well. > > OK, when your patches are ACKed I'll send patches to both improve py2 > packaging and add the new packages. > See above. -- Jan Cholasta From cheimes at redhat.com Mon Dec 7 15:26:13 2015 From: cheimes at redhat.com (Christian Heimes) Date: Mon, 7 Dec 2015 16:26:13 +0100 Subject: [Freeipa-devel] [PATCH 0026] Workarounds for SELinux execmem violations in cryptography In-Reply-To: <20151207151747.GZ8374@redhat.com> References: <566599BE.1020008@redhat.com> <20151207151747.GZ8374@redhat.com> Message-ID: <5665A515.30507@redhat.com> On 2015-12-07 16:17, Alexander Bokovoy wrote: > On Mon, 07 Dec 2015, Christian Heimes wrote: >> The patch fixes SELinux violations in Fedora 23. >> >> Background: Recent versions of cryptography cause SELinux violation >> which will lead to a segfault, see >> https://bugzilla.redhat.com/show_bug.cgi?id=1277224 . The segfault only >> occurs in the context of Apache HTTPD (FreeIPA web ui) when >> cryptography.hazmat.backends.default_backend() is initialized. I'm >> working on a fix for cryptography but it will take a while. First I have >> to wait for a new upstream release of python-cffi. Armin Ronacher plans >> to release cffi 1.4 in two weeks. >> >> >> ipaserver.dcerpc uses M2Crypto again on Python 2.7 and Dogtag's >> pki.client no longer tries to use PyOpenSSL instead of Python's ssl >> module. >> >> Some dependencies like Dogtag's pki.client library and custodia use >> python-requsts to make HTTPS connection. python-requests prefers >> PyOpenSSL over Python's stdlib ssl module. PyOpenSSL is build on top >> of python-cryptography which trigger a execmem SELinux violation >> in the context of Apache HTTPD (httpd_execmem). >> When requests is imported, it always tries to import pyopenssl glue >> code from urllib3's contrib directory. The import of PyOpenSSL is >> enough to trigger the SELinux denial. >> A hack in wsgi.py prevents the import by raising an ImportError. > ACK. Thanks for these patches. > > Note to Debian/Ubuntu maintainers: AppArmor 'support' in python-cffi > already detects apparmor by looking into /proc and disabling the use of > writeable and executable memory. On those platforms I suspect recent > enough python-cryptography would work without problem by downgrading own > feature set. The code in this patches should be harmless, though. Cryptography's core depends on dynamic callbacks. There is no "downgrade feature-set" feature. I guess the libffi uses the broken and potential dangerous workaround with two shared mmap() with file backend. (http://www.akkadia.org/drepper/selinux-mem.html). The approach requires a writeable, executable temp file and breaks isolation between a parent process and all its forked child processes. Christian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From mkosek at redhat.com Mon Dec 7 15:43:53 2015 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 7 Dec 2015 16:43:53 +0100 Subject: [Freeipa-devel] [PATCH 0391] replicainstall: Add check for domain if server is specified In-Reply-To: <566586F4.30201@redhat.com> References: <5661D8A0.7030303@redhat.com> <5661E661.5090601@redhat.com> <5661E810.3060503@redhat.com> <566586F4.30201@redhat.com> Message-ID: <5665A939.1060807@redhat.com> On 12/07/2015 02:17 PM, Tomas Babej wrote: > > > On 12/04/2015 08:22 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On 12/04/2015 07:17 PM, Tomas Babej wrote: >>>> Hi, >>>> >>>> Avoids failing in the later stages during the ipa-client-install >>>> command. >>>> >>>> Tomas >>> >>> Is this change needed? Wouldn't it be better to update >>> ipa-client-install or ipa-replica-install to not require the --domain >>> option? I would hope that --domain can be figured out during >>> installation and not passed to ipa-replica-install manually by the admin. >>> >>> I just think that calling >>> # ipa-replica-install --server=master.example.com >>> is better than >>> # ipa-replica-install --server=master.example.com --domain example.com >>> if possible. >> >> IIRC this is for service discovery when using a specific server and not >> LDAP. This is the domain used to search for the kerberos realm, for >> example. >> >> That isn't to say this isn't discoverable but it would require another >> function in discovery to query what the IPA domain is from the given >> master but it gets tricky if anonymous search is disabled, for example. >> >> rob >> > > Needed or not, this is the behaviour that ipa-client-install has now. > Adding a domain detection method would be a RFE for ipa-client-install > (and imho not something we should be adding at this point). > > This patch only focuses on making the ipa-replica-install work more > smoothly. I am just thinking that client promotion (ipa-replica-install) and ipa-client-install are a bit different use cases. While ipa-client-install should be typically run in auto-discovery and you thus do not use --server option much, while with ipa-replica-install, you want to make sure you have the expected topology and should use --server all the time without gambling on it. But I do not think it has to be there since 4.3 GA, can you please file a ticket for this gap? From mbasti at redhat.com Mon Dec 7 15:46:34 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 7 Dec 2015 16:46:34 +0100 Subject: [Freeipa-devel] [PATCH 0370] CI: test various topologies with 3 replicas In-Reply-To: <56659DC5.9040901@redhat.com> References: <5664A21D.7040802@redhat.com> <56659DC5.9040901@redhat.com> Message-ID: <5665A9DA.8010903@redhat.com> On 07.12.2015 15:55, Oleg Fayans wrote: > ACK > > On 12/06/2015 10:01 PM, Martin Basti wrote: >> Patch attached, to work properly it requires all patches I sent today + >> workaround patch attached (Martin3 will provide proper fix). >> >> The last two test are failing due to bug in test framework >> (ipa-replica-manage should not be used with domain level1) >> >> Martin^2 Pushed to master: bee222372aef6d8dbc353458392efc6b0ad225ea From mkubik at redhat.com Mon Dec 7 16:49:18 2015 From: mkubik at redhat.com (=?UTF-8?Q?Milan_Kub=c3=adk?=) Date: Mon, 7 Dec 2015 17:49:18 +0100 Subject: [Freeipa-devel] [PATCH 0002] Refactor test_group_plugin In-Reply-To: <20151203201556.4ff69dba@vor2.netbox.priv> References: <20151120135636.71171d5c@vor2.netbox.priv> <20151123145934.36901470@dhcp-25-21.brq.redhat.com> <20151123164249.4a576706@dhcp-25-21.brq.redhat.com> <565C76D6.3010406@redhat.com> <20151203201556.4ff69dba@vor2.netbox.priv> Message-ID: <5665B88E.4000507@redhat.com> On 12/03/2015 08:15 PM, Filip ?kola wrote: > On Mon, 30 Nov 2015 17:18:30 +0100 > Milan Kub?k wrote: > >> On 11/23/2015 04:42 PM, Filip ?kola wrote: >>> Sending updated patch. >>> >>> F. >>> >>> On Mon, 23 Nov 2015 14:59:34 +0100 >>> Filip ?kola wrote: >>> >>>> Found couple of issues (broke some dependencies). >>>> >>>> NACK >>>> >>>> F. >>>> >>>> On Fri, 20 Nov 2015 13:56:36 +0100 >>>> Filip ?kola wrote: >>>> >>>>> Another one. >>>>> >>>>> F. >>> >> Hi, the tests look good. Few remarks, though. >> >> 1. Please, use the shortes copyright notice in new modules. >> >> # >> # Copyright (C) 2015 FreeIPA Contributors see COPYING for >> license # >> >> 2. The tests `test_group_remove_group_from_protected_group` and >> `test_group_full_set_of_objectclass_not_available_post_detach` >> were not ported. Please, include them in the patch. >> >> Also, for less hassle, please rebase your patches on top of >> freeipa-mkubik-0025-3-Separated-Tracker-implementations-into-standalone-pa.patch >> Which changes the location of tracker implementations and prevents >> circular imports. >> >> Thanks. >> > > > Hi, > > these cases are there, in corresponding classes. They are marked with > the original comments. (However I can move them to separate class if > desirable.) > > The copyright notice is changed. Also included a few changes in the > test with user without private group. > > Filip NACK linter: ************* Module tracker.group_plugin ipatests/test_xmlrpc/tracker/group_plugin.py:257: [E0102(function-redefined), GroupTracker.check_remove_member] method already defined line 253) Probably a leftover after the rebase made on top of my patch. Please fix it. You can check youch changes by make-lint script before sending them. Thanks -- Milan Kubik From mbabinsk at redhat.com Mon Dec 7 17:27:02 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Mon, 7 Dec 2015 18:27:02 +0100 Subject: [Freeipa-devel] [PATCH 0369] Force creation of service during replica install In-Reply-To: <56649E85.1000303@redhat.com> References: <56649E85.1000303@redhat.com> Message-ID: <5665C166.9020109@redhat.com> On 12/06/2015 09:45 PM, Martin Basti wrote: > Replica install should not fail due a missing A record, if there are > proper entries in hosts. > > Patch attached. > > ACK. -- Martin^3 Babinsky From mbasti at redhat.com Mon Dec 7 18:13:05 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 7 Dec 2015 19:13:05 +0100 Subject: [Freeipa-devel] [PATCH 0369] Force creation of service during replica install In-Reply-To: <5665C166.9020109@redhat.com> References: <56649E85.1000303@redhat.com> <5665C166.9020109@redhat.com> Message-ID: <5665CC31.7040809@redhat.com> On 07.12.2015 18:27, Martin Babinsky wrote: > On 12/06/2015 09:45 PM, Martin Basti wrote: >> Replica install should not fail due a missing A record, if there are >> proper entries in hosts. >> >> Patch attached. >> >> > ACK. > Pushed to master: cac756b87d2eb521f038d0fb2ddb2a98569cf1af From pvoborni at redhat.com Mon Dec 7 18:59:02 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 7 Dec 2015 19:59:02 +0100 Subject: [Freeipa-devel] [PATCH 0026] Workarounds for SELinux execmem violations in cryptography In-Reply-To: <5665A515.30507@redhat.com> References: <566599BE.1020008@redhat.com> <20151207151747.GZ8374@redhat.com> <5665A515.30507@redhat.com> Message-ID: <5665D6F6.80905@redhat.com> On 7.12.2015 16:26, Christian Heimes wrote: > On 2015-12-07 16:17, Alexander Bokovoy wrote: >> On Mon, 07 Dec 2015, Christian Heimes wrote: >>> The patch fixes SELinux violations in Fedora 23. >>> >>> Background: Recent versions of cryptography cause SELinux violation >>> which will lead to a segfault, see >>> https://bugzilla.redhat.com/show_bug.cgi?id=1277224 . The segfault only >>> occurs in the context of Apache HTTPD (FreeIPA web ui) when >>> cryptography.hazmat.backends.default_backend() is initialized. I'm >>> working on a fix for cryptography but it will take a while. First I have >>> to wait for a new upstream release of python-cffi. Armin Ronacher plans >>> to release cffi 1.4 in two weeks. >>> >>> >>> ipaserver.dcerpc uses M2Crypto again on Python 2.7 and Dogtag's >>> pki.client no longer tries to use PyOpenSSL instead of Python's ssl >>> module. >>> >>> Some dependencies like Dogtag's pki.client library and custodia use >>> python-requsts to make HTTPS connection. python-requests prefers >>> PyOpenSSL over Python's stdlib ssl module. PyOpenSSL is build on top >>> of python-cryptography which trigger a execmem SELinux violation >>> in the context of Apache HTTPD (httpd_execmem). >>> When requests is imported, it always tries to import pyopenssl glue >>> code from urllib3's contrib directory. The import of PyOpenSSL is >>> enough to trigger the SELinux denial. >>> A hack in wsgi.py prevents the import by raising an ImportError. >> ACK. Thanks for these patches. >> >> Note to Debian/Ubuntu maintainers: AppArmor 'support' in python-cffi >> already detects apparmor by looking into /proc and disabling the use of >> writeable and executable memory. On those platforms I suspect recent >> enough python-cryptography would work without problem by downgrading own >> feature set. The code in this patches should be harmless, though. > > Cryptography's core depends on dynamic callbacks. There is no "downgrade > feature-set" feature. > > I guess the libffi uses the broken and potential dangerous workaround > with two shared mmap() with file backend. > (http://www.akkadia.org/drepper/selinux-mem.html). The approach requires > a writeable, executable temp file and breaks isolation between a parent > process and all its forked child processes. > > Christian > The patch needs to be rebased to 4-2 branch to be usable on Fedora 23 - FreeIPA 4.2.3. -- Petr Vobornik From mbasti at redhat.com Mon Dec 7 20:11:51 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 7 Dec 2015 21:11:51 +0100 Subject: [Freeipa-devel] [PATCH 522] replica promotion: allow OTP bulk client enrollment In-Reply-To: <56653386.5020703@redhat.com> References: <565F0D00.6050805@redhat.com> <56653386.5020703@redhat.com> Message-ID: <5665E807.3070600@redhat.com> On 07.12.2015 08:21, Jan Cholasta wrote: > On 2.12.2015 16:23, Jan Cholasta wrote: >> Hi, >> >> the attached patch fixes . >> >> Note that you still have to provide admin password in >> ipa-replica-install, either using --admin-password or interactively, >> because: >> >> a) Admin password is required for replica promotion. This will be fixed >> with . >> >> Patches are on the list: >> . >> > > Pushed. > >> >> >> b) Admin password is required for connection check. This will be fixed >> with . > > Martin Basti pointed out that admin password should not be asked > interactively during OTP replica promotion. Fixed. > > Updated and rebased patch attached. > > > 1) [root at vm-058-138 ~]# ipa-replica-install --server vm-058-137.abc.idm.lab.eng.brq.redhat.com --domain abc.idm.lab.eng.brq.redhat.com --password=bubak --setup-ca Configuring client side components Password for admin at ABC.IDM.LAB.ENG.BRQ.REDHAT.COM: IMO password should be asked first, before any installation begins (IMO this is for conncheck) 2) When host is not in ipaservers hostgroup. Also I would expect different error message ipa-replica-install --server vm-058-137.abc.idm.lab.eng.brq.redhat.com --domain abc.idm.lab.eng.brq.redhat.com --password=bubak --setup-ca --skip-conncheck .... step() File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 352, in step = lambda: next(self.__gen) File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from six.reraise(*exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from value = gen.send(prev_value) File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line 63, in _install for nothing in self._installer(self.parent): File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 1507, in main promote_check(self) File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 374, in decorated func(installer) File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 1002, in promote_check conn.connect(ccache=installer._ccache) File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 66, in connect conn = self.create_connection(*args, **kw) File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", line 199, in create_connection principal = krb_utils.get_principal(ccache_name=ccache) File "/usr/lib/python2.7/site-packages/ipalib/krb_utils.py", line 184, in get_principal raise errors.CCacheError(message=unicode(e)) 2015-12-07T16:23:40Z DEBUG The ipa-replica-install command failed, exception: CCacheError: Major (851968): Unspecified GSS failure. Minor code may provide more information, Minor (2529639053): No Kerberos credentials available 2015-12-07T16:23:40Z ERROR Major (851968): Unspecified GSS failure. Minor code may provide more information, Minor (2529639053): No Kerberos credentials available 3) This case is not handle very well: a) install client with OTP password b) install replica with the same OTP password (when host is no in ipaservers group, if host is in ipaservers group it works) ipa.ipapython.install.cli.install_tool(Replica): ERROR Major (851968): Unspecified GSS failure. Minor code may provide more information, Minor (2529639053): No Kerberos credentials available ipa.ipapython.install.cli.install_tool(Replica): ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information 4) This is not user friendly I used wrong OTP password, can we somehow propagate the actual error from client install to stderr? ipa.ipapython.install.cli.install_tool(Replica): ERROR Configuration of client side components failed! ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' '--unattended' '--domain' 'abc.idm.lab.eng.brq.redhat.com' '--server' 'vm-058-137.abc.idm.lab.eng.brq.redhat.com' '--password' 'buba'' returned non-zero exit status 1 ipa.ipapython.install.cli.install_tool(Replica): ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information -------------- next part -------------- An HTML attachment was scrubbed... URL: From simo at redhat.com Mon Dec 7 20:13:30 2015 From: simo at redhat.com (Simo Sorce) Date: Mon, 07 Dec 2015 15:13:30 -0500 Subject: [Freeipa-devel] [PATCH 564] Implement pwd policy iteration in the kdb driver Message-ID: <1449519210.17418.18.camel@redhat.com> Subject says it all. Tested via kadmin.local list_policies Ticket: 3015 Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-564-1-Implement-pwd-policy-iterator.patch Type: text/x-patch Size: 6711 bytes Desc: not available URL: From mbasti at redhat.com Mon Dec 7 21:44:32 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 7 Dec 2015 22:44:32 +0100 Subject: [Freeipa-devel] [PATCH 0372] CI: installation tests In-Reply-To: <56659D0E.5060806@redhat.com> References: <5664A718.6080101@redhat.com> <56657A83.6020702@redhat.com> <56658A5E.8010603@redhat.com> <56658BB5.5060700@redhat.com> <56659C6D.4040600@redhat.com> <56659CE2.1060304@redhat.com> <56659D0E.5060806@redhat.com> Message-ID: <5665FDC0.7020907@redhat.com> On 07.12.2015 15:51, Oleg Fayans wrote: > > On 12/07/2015 03:51 PM, Martin Basti wrote: >> >> On 07.12.2015 15:49, Oleg Fayans wrote: >>> Hi, >>> >>> On 12/07/2015 02:37 PM, Martin Basti wrote: >>>> On 07.12.2015 14:32, Martin Basti wrote: >>>>> On 07.12.2015 13:24, Oleg Fayans wrote: >>>>>> Hi Martin, >>>>>> >>>>>> I would prefer both install_kra and install_ca methods to have >>>>>> raiseonerr parameter set to True by default. We need a way to test >>>>>> negatives and analyze results. >>>>>> Mine looks like this: >>>>>> >>>>>> def install_kra(host, domain_level=None, >>>>>> first_instance=False, raiseonerr=True): >>>>>> if not domain_level: >>>>>> domain_level = host.config.domain_level >>>>>> command = ["ipa-kra-install", "-U", "-p", >>>>>> host.config.dirman_password] >>>>>> if domain_level == DOMAIN_LEVEL_0 and not first_instance: >>>>>> replica_file = get_replica_filename(host) >>>>>> command.append(replica_file) >>>>>> return host.run_command(command, raiseonerr=raiseonerr) >>>>>> >>>>>> The rest looks good to me, but I did not run the tests yet. >>>>> Sounds good, I will amend the patche later. >>>> I changed my mind, should not be the domain_level value get from >>>> function domainlevel(host)? >>> We should have a way to test negatives, like providing replica file at >>> domain level=1 and not providing at domain level=0. So these functions >>> should either accept replica file as a parameter, or arbitrary domain >>> level >> Agree, but I meant this >> >> if not domain_level: >> - domain_level = host.config.domain_level >> + domain_level = domainlevel(host) > Oh, yes, you are right, that's better. > >> >>>> Martin^2 >>>> >>>>>> On 12/06/2015 10:22 PM, Martin Basti wrote: >>>>>>> My favorite today \o/ --> 67 <-- \o/ test cases, no more manual >>>>>>> testing >>>>>>> of installers \o/. >>>>>>> >>>>>>> Test suite contains: 6 combination how to install components on >>>>>>> replica >>>>>>> X 4 combinations of server installation + 3 extra server tests >>>>>>> >>>>>>> To save time tests install 1 master and 3 replicas per test class >>>>>>> (except extra server tests): >>>>>>> Class name specifies what is installed on master. >>>>>>> >>>>>>> Remember, option "-k" is your friend >>>>>>> $ ipa-run-tests -k '' >>>>>>> otherwise you will need a lot of time until tests finish. >>>>>>> >>>>>>> To list all tests: >>>>>>> $ ipa-run-tests test_integration/test_installation.py --collect-only >>>>>>> >>>>>>> >>>>>>> Patch attached, it needs to have attached all patches I sent today >>>>>>> and >>>>>>> also attached workaround patch (Martin3 will provide proper fix) >>>>>>> >>>>>>> I haven't had time/mood/resources to test this patch with domain >>>>>>> level >>>>>>> 0, so testing this patch with domain level 0 is appreciated. >>>>>>> >>>>>>> >>>>>>> Oleg, I added some methods to tasks.py which you may want to reuse. >>>>>>> >>>>>>> Martin^2 >>>>>>> Updated patch attached. Still some tests are failing, not sure if it is test issue or bugs. I will test later. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0372.2-CI-installation-tests.patch Type: text/x-patch Size: 7937 bytes Desc: not available URL: From ftweedal at redhat.com Mon Dec 7 22:46:39 2015 From: ftweedal at redhat.com (Fraser Tweedale) Date: Tue, 8 Dec 2015 08:46:39 +1000 Subject: [Freeipa-devel] [PATCH] 0046 Create server certs with DNS altname In-Reply-To: <5665813B.90406@redhat.com> References: <20151207052611.GG23644@dhcp-40-8.bne.redhat.com> <5665813B.90406@redhat.com> Message-ID: <20151207224639.GL23644@dhcp-40-8.bne.redhat.com> On Mon, Dec 07, 2015 at 01:53:15PM +0100, Martin Kosek wrote: > On 12/07/2015 06:26 AM, Fraser Tweedale wrote: > > The attached patch fixes > > https://fedorahosted.org/freeipa/ticket/4970. > > > > Note that the problem is addressed by adding the appropriate request > > extension to the CSR; the fix does not involve changing the default > > profile behaviour, which is complicated (see ticket for details). > > Thanks for the patch! This is something we should really fix, I already get > warnings in my Python scripts when I hit sites protected by such HTTPS cert: > > /usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:264: > SubjectAltNameWarning: Certificate for projects.engineering.redhat.com has no > `subjectAltName`, falling back to check for a `commonName` for now. This > feature is being removed by major browsers and deprecated by RFC 2818. (See > https://github.com/shazow/urllib3/issues/497 for details.) > > Should we split ticket 4970, for the FreeIPA server part and then for cert > profile part? As it looks like the FreeIPA server will be fixed even in FreeIPA > 4.3.x and the other part later. > > How difficult do you see the general FreeIPA Certificate Profile part of this > request? Is it a too big task to handle in 4.4 time frame? > I will split the ticket and would suggest 4.4 Backlog - it might be doable but is a lower priority than e.g. Sub-CAs. Cheers, Fraser From rcritten at redhat.com Mon Dec 7 22:50:05 2015 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 7 Dec 2015 17:50:05 -0500 Subject: [Freeipa-devel] [PATCH] 0046 Create server certs with DNS altname In-Reply-To: <20151207224639.GL23644@dhcp-40-8.bne.redhat.com> References: <20151207052611.GG23644@dhcp-40-8.bne.redhat.com> <5665813B.90406@redhat.com> <20151207224639.GL23644@dhcp-40-8.bne.redhat.com> Message-ID: <56660D1D.6000109@redhat.com> Fraser Tweedale wrote: > On Mon, Dec 07, 2015 at 01:53:15PM +0100, Martin Kosek wrote: >> On 12/07/2015 06:26 AM, Fraser Tweedale wrote: >>> The attached patch fixes >>> https://fedorahosted.org/freeipa/ticket/4970. >>> >>> Note that the problem is addressed by adding the appropriate request >>> extension to the CSR; the fix does not involve changing the default >>> profile behaviour, which is complicated (see ticket for details). >> >> Thanks for the patch! This is something we should really fix, I already get >> warnings in my Python scripts when I hit sites protected by such HTTPS cert: >> >> /usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:264: >> SubjectAltNameWarning: Certificate for projects.engineering.redhat.com has no >> `subjectAltName`, falling back to check for a `commonName` for now. This >> feature is being removed by major browsers and deprecated by RFC 2818. (See >> https://github.com/shazow/urllib3/issues/497 for details.) >> >> Should we split ticket 4970, for the FreeIPA server part and then for cert >> profile part? As it looks like the FreeIPA server will be fixed even in FreeIPA >> 4.3.x and the other part later. >> >> How difficult do you see the general FreeIPA Certificate Profile part of this >> request? Is it a too big task to handle in 4.4 time frame? >> > I will split the ticket and would suggest 4.4 Backlog - it might be > doable but is a lower priority than e.g. Sub-CAs. If you are going to defer the profile part then you should probably update the client to also include a SAN if --request-cert is provided. rob From ftweedal at redhat.com Tue Dec 8 01:22:54 2015 From: ftweedal at redhat.com (Fraser Tweedale) Date: Tue, 8 Dec 2015 11:22:54 +1000 Subject: [Freeipa-devel] [PATCH] 0046 Create server certs with DNS altname In-Reply-To: <20151207224639.GL23644@dhcp-40-8.bne.redhat.com> References: <20151207052611.GG23644@dhcp-40-8.bne.redhat.com> <5665813B.90406@redhat.com> <20151207224639.GL23644@dhcp-40-8.bne.redhat.com> Message-ID: <20151208012254.GO23644@dhcp-40-8.bne.redhat.com> On Tue, Dec 08, 2015 at 08:46:39AM +1000, Fraser Tweedale wrote: > On Mon, Dec 07, 2015 at 01:53:15PM +0100, Martin Kosek wrote: > > On 12/07/2015 06:26 AM, Fraser Tweedale wrote: > > > The attached patch fixes > > > https://fedorahosted.org/freeipa/ticket/4970. > > > > > > Note that the problem is addressed by adding the appropriate request > > > extension to the CSR; the fix does not involve changing the default > > > profile behaviour, which is complicated (see ticket for details). > > > > Thanks for the patch! This is something we should really fix, I already get > > warnings in my Python scripts when I hit sites protected by such HTTPS cert: > > > > /usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:264: > > SubjectAltNameWarning: Certificate for projects.engineering.redhat.com has no > > `subjectAltName`, falling back to check for a `commonName` for now. This > > feature is being removed by major browsers and deprecated by RFC 2818. (See > > https://github.com/shazow/urllib3/issues/497 for details.) > > > > Should we split ticket 4970, for the FreeIPA server part and then for cert > > profile part? As it looks like the FreeIPA server will be fixed even in FreeIPA > > 4.3.x and the other part later. > > > > How difficult do you see the general FreeIPA Certificate Profile part of this > > request? Is it a too big task to handle in 4.4 time frame? > > > I will split the ticket and would suggest 4.4 Backlog - it might be > doable but is a lower priority than e.g. Sub-CAs. > PKI ticket: https://fedorahosted.org/pki/ticket/1710 IPA tracker: https://fedorahosted.org/freeipa/ticket/5523 > Cheers, > Fraser > > -- > Manage your subscription for the Freeipa-devel mailing list: > https://www.redhat.com/mailman/listinfo/freeipa-devel > Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code From dkupka at redhat.com Tue Dec 8 06:51:37 2015 From: dkupka at redhat.com (David Kupka) Date: Tue, 8 Dec 2015 07:51:37 +0100 Subject: [Freeipa-devel] [PATCH 0070] install: Run all validators at once. In-Reply-To: <56658419.3080100@redhat.com> References: <56658419.3080100@redhat.com> Message-ID: <56667DF9.5080709@redhat.com> On 07/12/15 14:05, David Kupka wrote: > Running validators after all Knobs are set allows use of other Knob > value during validation. Updated patch attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0070.1-install-Run-all-validators-at-once.patch Type: text/x-patch Size: 1923 bytes Desc: not available URL: From jcholast at redhat.com Tue Dec 8 06:57:16 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 8 Dec 2015 07:57:16 +0100 Subject: [Freeipa-devel] [PATCH 0391] replicainstall: Add check for domain if server is specified In-Reply-To: <5665A939.1060807@redhat.com> References: <5661D8A0.7030303@redhat.com> <5661E661.5090601@redhat.com> <5661E810.3060503@redhat.com> <566586F4.30201@redhat.com> <5665A939.1060807@redhat.com> Message-ID: <56667F4C.9070305@redhat.com> On 7.12.2015 16:43, Martin Kosek wrote: > On 12/07/2015 02:17 PM, Tomas Babej wrote: >> >> >> On 12/04/2015 08:22 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> On 12/04/2015 07:17 PM, Tomas Babej wrote: >>>>> Hi, >>>>> >>>>> Avoids failing in the later stages during the ipa-client-install >>>>> command. >>>>> >>>>> Tomas >>>> >>>> Is this change needed? Wouldn't it be better to update >>>> ipa-client-install or ipa-replica-install to not require the --domain >>>> option? I would hope that --domain can be figured out during >>>> installation and not passed to ipa-replica-install manually by the admin. >>>> >>>> I just think that calling >>>> # ipa-replica-install --server=master.example.com >>>> is better than >>>> # ipa-replica-install --server=master.example.com --domain example.com >>>> if possible. >>> >>> IIRC this is for service discovery when using a specific server and not >>> LDAP. This is the domain used to search for the kerberos realm, for >>> example. >>> >>> That isn't to say this isn't discoverable but it would require another >>> function in discovery to query what the IPA domain is from the given >>> master but it gets tricky if anonymous search is disabled, for example. >>> >>> rob >>> >> >> Needed or not, this is the behaviour that ipa-client-install has now. >> Adding a domain detection method would be a RFE for ipa-client-install >> (and imho not something we should be adding at this point). >> >> This patch only focuses on making the ipa-replica-install work more >> smoothly. > > I am just thinking that client promotion (ipa-replica-install) and > ipa-client-install are a bit different use cases. While ipa-client-install > should be typically run in auto-discovery and you thus do not use --server > option much, while with ipa-replica-install, you want to make sure you have the > expected topology and should use --server all the time without gambling on it. > > But I do not think it has to be there since 4.3 GA, can you please file a > ticket for this gap? I would rather do it now, because the change from optional to mandatory is backward incompatible. (We don't want to break users' scripts, right?) -- Jan Cholasta From jcholast at redhat.com Tue Dec 8 07:12:40 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 8 Dec 2015 08:12:40 +0100 Subject: [Freeipa-devel] [PATCH 0070] install: Run all validators at once. In-Reply-To: <56667DF9.5080709@redhat.com> References: <56658419.3080100@redhat.com> <56667DF9.5080709@redhat.com> Message-ID: <566682E8.7070003@redhat.com> On 8.12.2015 07:51, David Kupka wrote: > On 07/12/15 14:05, David Kupka wrote: >> Running validators after all Knobs are set allows use of other Knob >> value during validation. > > Updated patch attached. Works for me, ACK. Pushed to master: 2c5a662fd80f7152834dfebf45628d3a7b8a68bf -- Jan Cholasta From mkosek at redhat.com Tue Dec 8 07:23:12 2015 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 8 Dec 2015 08:23:12 +0100 Subject: [Freeipa-devel] [PATCH 0391] replicainstall: Add check for domain if server is specified In-Reply-To: <56667F4C.9070305@redhat.com> References: <5661D8A0.7030303@redhat.com> <5661E661.5090601@redhat.com> <5661E810.3060503@redhat.com> <566586F4.30201@redhat.com> <5665A939.1060807@redhat.com> <56667F4C.9070305@redhat.com> Message-ID: <56668560.1040508@redhat.com> On 12/08/2015 07:57 AM, Jan Cholasta wrote: > On 7.12.2015 16:43, Martin Kosek wrote: >> On 12/07/2015 02:17 PM, Tomas Babej wrote: >>> >>> >>> On 12/04/2015 08:22 PM, Rob Crittenden wrote: >>>> Martin Kosek wrote: >>>>> On 12/04/2015 07:17 PM, Tomas Babej wrote: >>>>>> Hi, >>>>>> >>>>>> Avoids failing in the later stages during the ipa-client-install >>>>>> command. >>>>>> >>>>>> Tomas >>>>> >>>>> Is this change needed? Wouldn't it be better to update >>>>> ipa-client-install or ipa-replica-install to not require the --domain >>>>> option? I would hope that --domain can be figured out during >>>>> installation and not passed to ipa-replica-install manually by the admin. >>>>> >>>>> I just think that calling >>>>> # ipa-replica-install --server=master.example.com >>>>> is better than >>>>> # ipa-replica-install --server=master.example.com --domain example.com >>>>> if possible. >>>> >>>> IIRC this is for service discovery when using a specific server and not >>>> LDAP. This is the domain used to search for the kerberos realm, for >>>> example. >>>> >>>> That isn't to say this isn't discoverable but it would require another >>>> function in discovery to query what the IPA domain is from the given >>>> master but it gets tricky if anonymous search is disabled, for example. >>>> >>>> rob >>>> >>> >>> Needed or not, this is the behaviour that ipa-client-install has now. >>> Adding a domain detection method would be a RFE for ipa-client-install >>> (and imho not something we should be adding at this point). >>> >>> This patch only focuses on making the ipa-replica-install work more >>> smoothly. >> >> I am just thinking that client promotion (ipa-replica-install) and >> ipa-client-install are a bit different use cases. While ipa-client-install >> should be typically run in auto-discovery and you thus do not use --server >> option much, while with ipa-replica-install, you want to make sure you have the >> expected topology and should use --server all the time without gambling on it. >> >> But I do not think it has to be there since 4.3 GA, can you please file a >> ticket for this gap? > > I would rather do it now, because the change from optional to mandatory is > backward incompatible. (We don't want to break users' scripts, right?) I think it is the other way around - with the change I was suggesting (autodetecting --domain option instead of always requesting it, as in Tomas' patch which we can merge if my proposal is not doable for 4.3 GA). From jcholast at redhat.com Tue Dec 8 07:28:13 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 8 Dec 2015 08:28:13 +0100 Subject: [Freeipa-devel] [PATCH 0391] replicainstall: Add check for domain if server is specified In-Reply-To: <56668560.1040508@redhat.com> References: <5661D8A0.7030303@redhat.com> <5661E661.5090601@redhat.com> <5661E810.3060503@redhat.com> <566586F4.30201@redhat.com> <5665A939.1060807@redhat.com> <56667F4C.9070305@redhat.com> <56668560.1040508@redhat.com> Message-ID: <5666868D.6010702@redhat.com> On 8.12.2015 08:23, Martin Kosek wrote: > On 12/08/2015 07:57 AM, Jan Cholasta wrote: >> On 7.12.2015 16:43, Martin Kosek wrote: >>> On 12/07/2015 02:17 PM, Tomas Babej wrote: >>>> >>>> >>>> On 12/04/2015 08:22 PM, Rob Crittenden wrote: >>>>> Martin Kosek wrote: >>>>>> On 12/04/2015 07:17 PM, Tomas Babej wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Avoids failing in the later stages during the ipa-client-install >>>>>>> command. >>>>>>> >>>>>>> Tomas >>>>>> >>>>>> Is this change needed? Wouldn't it be better to update >>>>>> ipa-client-install or ipa-replica-install to not require the --domain >>>>>> option? I would hope that --domain can be figured out during >>>>>> installation and not passed to ipa-replica-install manually by the admin. >>>>>> >>>>>> I just think that calling >>>>>> # ipa-replica-install --server=master.example.com >>>>>> is better than >>>>>> # ipa-replica-install --server=master.example.com --domain example.com >>>>>> if possible. >>>>> >>>>> IIRC this is for service discovery when using a specific server and not >>>>> LDAP. This is the domain used to search for the kerberos realm, for >>>>> example. >>>>> >>>>> That isn't to say this isn't discoverable but it would require another >>>>> function in discovery to query what the IPA domain is from the given >>>>> master but it gets tricky if anonymous search is disabled, for example. >>>>> >>>>> rob >>>>> >>>> >>>> Needed or not, this is the behaviour that ipa-client-install has now. >>>> Adding a domain detection method would be a RFE for ipa-client-install >>>> (and imho not something we should be adding at this point). >>>> >>>> This patch only focuses on making the ipa-replica-install work more >>>> smoothly. >>> >>> I am just thinking that client promotion (ipa-replica-install) and >>> ipa-client-install are a bit different use cases. While ipa-client-install >>> should be typically run in auto-discovery and you thus do not use --server >>> option much, while with ipa-replica-install, you want to make sure you have the >>> expected topology and should use --server all the time without gambling on it. >>> >>> But I do not think it has to be there since 4.3 GA, can you please file a >>> ticket for this gap? >> >> I would rather do it now, because the change from optional to mandatory is >> backward incompatible. (We don't want to break users' scripts, right?) > > I think it is the other way around - with the change I was suggesting > (autodetecting --domain option instead of always requesting it, as in Tomas' > patch which we can merge if my proposal is not doable for 4.3 GA). > "with ipa-replica-install, you want to make sure you have the expected topology and should use --server all the time" sounds like you want to make --server mandatory for ipa-replica-install, which should be done either before 4.3 GA or never. -- Jan Cholasta From jcholast at redhat.com Tue Dec 8 07:52:35 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 8 Dec 2015 08:52:35 +0100 Subject: [Freeipa-devel] [PATCH 522] replica promotion: allow OTP bulk client enrollment In-Reply-To: <5665E807.3070600@redhat.com> References: <565F0D00.6050805@redhat.com> <56653386.5020703@redhat.com> <5665E807.3070600@redhat.com> Message-ID: <56668C43.3000908@redhat.com> On 7.12.2015 21:11, Martin Basti wrote: > > > On 07.12.2015 08:21, Jan Cholasta wrote: >> On 2.12.2015 16:23, Jan Cholasta wrote: >>> Hi, >>> >>> the attached patch fixes . >>> >>> Note that you still have to provide admin password in >>> ipa-replica-install, either using --admin-password or interactively, >>> because: >>> >>> a) Admin password is required for replica promotion. This will be fixed >>> with . >>> >>> Patches are on the list: >>> . >>> >> >> Pushed. >> >>> >>> >>> b) Admin password is required for connection check. This will be fixed >>> with . >> >> Martin Basti pointed out that admin password should not be asked >> interactively during OTP replica promotion. Fixed. >> >> Updated and rebased patch attached. >> >> >> > > 1) > [root at vm-058-138 ~]# ipa-replica-install --server > vm-058-137.abc.idm.lab.eng.brq.redhat.com --domain > abc.idm.lab.eng.brq.redhat.com --password=bubak --setup-ca > Configuring client side components > Password for admin at ABC.IDM.LAB.ENG.BRQ.REDHAT.COM: > > IMO password should be asked first, before any installation begins (IMO > this is for conncheck) The same thing happens without my patch. Could you file a ticket? > > 2) > When host is not in ipaservers hostgroup. Also I would expect different > error message > ipa-replica-install --server vm-058-137.abc.idm.lab.eng.brq.redhat.com > --domain abc.idm.lab.eng.brq.redhat.com --password=bubak --setup-ca > --skip-conncheck > > .... > step() > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", > line 352, in > step = lambda: next(self.__gen) > File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", > line 81, in run_generator_with_yield_from > six.reraise(*exc_info) > File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", > line 59, in run_generator_with_yield_from > value = gen.send(prev_value) > File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", > line 63, in _install > for nothing in self._installer(self.parent): > File > "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", > line 1507, in main > promote_check(self) > File > "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", > line 374, in decorated > func(installer) > File > "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", > line 1002, in promote_check > conn.connect(ccache=installer._ccache) > File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 66, > in connect > conn = self.create_connection(*args, **kw) > File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", > line 199, in create_connection > principal = krb_utils.get_principal(ccache_name=ccache) > File "/usr/lib/python2.7/site-packages/ipalib/krb_utils.py", line > 184, in get_principal > raise errors.CCacheError(message=unicode(e)) > > 2015-12-07T16:23:40Z DEBUG The ipa-replica-install command failed, > exception: CCacheError: Major (851968): Unspecified GSS failure. Minor > code may provide more information, Minor (2529639053): No Kerberos > credentials available > 2015-12-07T16:23:40Z ERROR Major (851968): Unspecified GSS failure. > Minor code may provide more information, Minor (2529639053): No Kerberos > credentials available Fixed. > > > 3) > This case is not handle very well: > a) install client with OTP password > b) install replica with the same OTP password (when host is no in > ipaservers group, if host is in ipaservers group it works) > > ipa.ipapython.install.cli.install_tool(Replica): ERROR Major > (851968): Unspecified GSS failure. Minor code may provide more > information, Minor (2529639053): No Kerberos credentials available > ipa.ipapython.install.cli.install_tool(Replica): ERROR The > ipa-replica-install command failed. See /var/log/ipareplica-install.log > for more information This is the same as 2). > > 4) > This is not user friendly > I used wrong OTP password, can we somehow propagate the actual error > from client install to stderr? > > ipa.ipapython.install.cli.install_tool(Replica): ERROR Configuration of > client side components failed! > ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' > '--unattended' '--domain' 'abc.idm.lab.eng.brq.redhat.com' '--server' > 'vm-058-137.abc.idm.lab.eng.brq.redhat.com' '--password' 'buba'' > returned non-zero exit status 1 > ipa.ipapython.install.cli.install_tool(Replica): ERROR The > ipa-replica-install command failed. See /var/log/ipareplica-install.log > for more information The same thing happens without my patch for any other error. Could you file a ticket? Updated patch attached. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-522.2-replica-promotion-allow-OTP-bulk-client-enrollment.patch Type: text/x-patch Size: 4741 bytes Desc: not available URL: From pspacek at redhat.com Tue Dec 8 07:56:32 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 8 Dec 2015 08:56:32 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <56658C89.8050103@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> <5665843C.2090507@redhat.com> <56658C89.8050103@redhat.com> Message-ID: <56668D30.8060308@redhat.com> On 7.12.2015 14:41, David Kupka wrote: > +def is_host_resolvable(fqdn): > + if not isinstance(fqdn, DNSName): > + fqdn = DNSName(fqdn) > + for rdtype in (rdatatype.A, rdatatype.AAAA): > + try: > + resolver.query(fqdn.make_absolute(), rdtype) > + except DNSException: > + continue > + else: > + return True > + > + return False > NACK, you are re-introducing duplicate function which was removed in 498471e4aed1367b72cd74d15811d0584a6ee268. Please amend the patch ASAP to use new verify_host_resolvable() function so I can test it and get it into 4.3. -- Petr^2 Spacek From mkosek at redhat.com Tue Dec 8 08:00:20 2015 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 8 Dec 2015 09:00:20 +0100 Subject: [Freeipa-devel] [PATCH] 0046 Create server certs with DNS altname In-Reply-To: <20151208012254.GO23644@dhcp-40-8.bne.redhat.com> References: <20151207052611.GG23644@dhcp-40-8.bne.redhat.com> <5665813B.90406@redhat.com> <20151207224639.GL23644@dhcp-40-8.bne.redhat.com> <20151208012254.GO23644@dhcp-40-8.bne.redhat.com> Message-ID: <56668E14.3060103@redhat.com> On 12/08/2015 02:22 AM, Fraser Tweedale wrote: > On Tue, Dec 08, 2015 at 08:46:39AM +1000, Fraser Tweedale wrote: >> On Mon, Dec 07, 2015 at 01:53:15PM +0100, Martin Kosek wrote: >>> On 12/07/2015 06:26 AM, Fraser Tweedale wrote: >>>> The attached patch fixes >>>> https://fedorahosted.org/freeipa/ticket/4970. >>>> >>>> Note that the problem is addressed by adding the appropriate request >>>> extension to the CSR; the fix does not involve changing the default >>>> profile behaviour, which is complicated (see ticket for details). >>> >>> Thanks for the patch! This is something we should really fix, I already get >>> warnings in my Python scripts when I hit sites protected by such HTTPS cert: >>> >>> /usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:264: >>> SubjectAltNameWarning: Certificate for projects.engineering.redhat.com has no >>> `subjectAltName`, falling back to check for a `commonName` for now. This >>> feature is being removed by major browsers and deprecated by RFC 2818. (See >>> https://github.com/shazow/urllib3/issues/497 for details.) >>> >>> Should we split ticket 4970, for the FreeIPA server part and then for cert >>> profile part? As it looks like the FreeIPA server will be fixed even in FreeIPA >>> 4.3.x and the other part later. >>> >>> How difficult do you see the general FreeIPA Certificate Profile part of this >>> request? Is it a too big task to handle in 4.4 time frame? >>> >> I will split the ticket and would suggest 4.4 Backlog - it might be >> doable but is a lower priority than e.g. Sub-CAs. >> > PKI ticket: https://fedorahosted.org/pki/ticket/1710 > IPA tracker: https://fedorahosted.org/freeipa/ticket/5523 Thanks. I updated the ticket and added more information. I increased priority as I do not want us to overlook it, as it has potential to break FreeIPA certificates when the major browsers remove support for such certificates. Right? From ofayans at redhat.com Tue Dec 8 09:04:12 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Tue, 8 Dec 2015 10:04:12 +0100 Subject: [Freeipa-devel] [PATCH 0372] CI: installation tests In-Reply-To: <5665FDC0.7020907@redhat.com> References: <5664A718.6080101@redhat.com> <56657A83.6020702@redhat.com> <56658A5E.8010603@redhat.com> <56658BB5.5060700@redhat.com> <56659C6D.4040600@redhat.com> <56659CE2.1060304@redhat.com> <56659D0E.5060806@redhat.com> <5665FDC0.7020907@redhat.com> Message-ID: <56669D0C.4050201@redhat.com> ACK On 12/07/2015 10:44 PM, Martin Basti wrote: > > > On 07.12.2015 15:51, Oleg Fayans wrote: >> >> On 12/07/2015 03:51 PM, Martin Basti wrote: >>> >>> On 07.12.2015 15:49, Oleg Fayans wrote: >>>> Hi, >>>> >>>> On 12/07/2015 02:37 PM, Martin Basti wrote: >>>>> On 07.12.2015 14:32, Martin Basti wrote: >>>>>> On 07.12.2015 13:24, Oleg Fayans wrote: >>>>>>> Hi Martin, >>>>>>> >>>>>>> I would prefer both install_kra and install_ca methods to have >>>>>>> raiseonerr parameter set to True by default. We need a way to test >>>>>>> negatives and analyze results. >>>>>>> Mine looks like this: >>>>>>> >>>>>>> def install_kra(host, domain_level=None, >>>>>>> first_instance=False, raiseonerr=True): >>>>>>> if not domain_level: >>>>>>> domain_level = host.config.domain_level >>>>>>> command = ["ipa-kra-install", "-U", "-p", >>>>>>> host.config.dirman_password] >>>>>>> if domain_level == DOMAIN_LEVEL_0 and not first_instance: >>>>>>> replica_file = get_replica_filename(host) >>>>>>> command.append(replica_file) >>>>>>> return host.run_command(command, raiseonerr=raiseonerr) >>>>>>> >>>>>>> The rest looks good to me, but I did not run the tests yet. >>>>>> Sounds good, I will amend the patche later. >>>>> I changed my mind, should not be the domain_level value get from >>>>> function domainlevel(host)? >>>> We should have a way to test negatives, like providing replica file at >>>> domain level=1 and not providing at domain level=0. So these functions >>>> should either accept replica file as a parameter, or arbitrary domain >>>> level >>> Agree, but I meant this >>> >>> if not domain_level: >>> - domain_level = host.config.domain_level >>> + domain_level = domainlevel(host) >> Oh, yes, you are right, that's better. >> >>> >>>>> Martin^2 >>>>> >>>>>>> On 12/06/2015 10:22 PM, Martin Basti wrote: >>>>>>>> My favorite today \o/ --> 67 <-- \o/ test cases, no more manual >>>>>>>> testing >>>>>>>> of installers \o/. >>>>>>>> >>>>>>>> Test suite contains: 6 combination how to install components on >>>>>>>> replica >>>>>>>> X 4 combinations of server installation + 3 extra server tests >>>>>>>> >>>>>>>> To save time tests install 1 master and 3 replicas per test class >>>>>>>> (except extra server tests): >>>>>>>> Class name specifies what is installed on master. >>>>>>>> >>>>>>>> Remember, option "-k" is your friend >>>>>>>> $ ipa-run-tests -k '>>>>>>> regexp)>' >>>>>>>> otherwise you will need a lot of time until tests finish. >>>>>>>> >>>>>>>> To list all tests: >>>>>>>> $ ipa-run-tests test_integration/test_installation.py >>>>>>>> --collect-only >>>>>>>> >>>>>>>> >>>>>>>> Patch attached, it needs to have attached all patches I sent today >>>>>>>> and >>>>>>>> also attached workaround patch (Martin3 will provide proper fix) >>>>>>>> >>>>>>>> I haven't had time/mood/resources to test this patch with domain >>>>>>>> level >>>>>>>> 0, so testing this patch with domain level 0 is appreciated. >>>>>>>> >>>>>>>> >>>>>>>> Oleg, I added some methods to tasks.py which you may want to reuse. >>>>>>>> >>>>>>>> Martin^2 >>>>>>>> > Updated patch attached. > > Still some tests are failing, not sure if it is test issue or bugs. I > will test later. -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From ftweedal at redhat.com Tue Dec 8 09:05:10 2015 From: ftweedal at redhat.com (Fraser Tweedale) Date: Tue, 8 Dec 2015 19:05:10 +1000 Subject: [Freeipa-devel] [PATCH] 0046 Create server certs with DNS altname In-Reply-To: <56668E14.3060103@redhat.com> References: <20151207052611.GG23644@dhcp-40-8.bne.redhat.com> <5665813B.90406@redhat.com> <20151207224639.GL23644@dhcp-40-8.bne.redhat.com> <20151208012254.GO23644@dhcp-40-8.bne.redhat.com> <56668E14.3060103@redhat.com> Message-ID: <20151208090510.GP23644@dhcp-40-8.bne.redhat.com> On Tue, Dec 08, 2015 at 09:00:20AM +0100, Martin Kosek wrote: > On 12/08/2015 02:22 AM, Fraser Tweedale wrote: > > On Tue, Dec 08, 2015 at 08:46:39AM +1000, Fraser Tweedale wrote: > >> On Mon, Dec 07, 2015 at 01:53:15PM +0100, Martin Kosek wrote: > >>> On 12/07/2015 06:26 AM, Fraser Tweedale wrote: > >>>> The attached patch fixes > >>>> https://fedorahosted.org/freeipa/ticket/4970. > >>>> > >>>> Note that the problem is addressed by adding the appropriate request > >>>> extension to the CSR; the fix does not involve changing the default > >>>> profile behaviour, which is complicated (see ticket for details). > >>> > >>> Thanks for the patch! This is something we should really fix, I already get > >>> warnings in my Python scripts when I hit sites protected by such HTTPS cert: > >>> > >>> /usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:264: > >>> SubjectAltNameWarning: Certificate for projects.engineering.redhat.com has no > >>> `subjectAltName`, falling back to check for a `commonName` for now. This > >>> feature is being removed by major browsers and deprecated by RFC 2818. (See > >>> https://github.com/shazow/urllib3/issues/497 for details.) > >>> > >>> Should we split ticket 4970, for the FreeIPA server part and then for cert > >>> profile part? As it looks like the FreeIPA server will be fixed even in FreeIPA > >>> 4.3.x and the other part later. > >>> > >>> How difficult do you see the general FreeIPA Certificate Profile part of this > >>> request? Is it a too big task to handle in 4.4 time frame? > >>> > >> I will split the ticket and would suggest 4.4 Backlog - it might be > >> doable but is a lower priority than e.g. Sub-CAs. > >> > > PKI ticket: https://fedorahosted.org/pki/ticket/1710 > > IPA tracker: https://fedorahosted.org/freeipa/ticket/5523 > > Thanks. I updated the ticket and added more information. I increased priority > as I do not want us to overlook it, as it has potential to break FreeIPA > certificates when the major browsers remove support for such certificates. Right? > Yes. With my (updated) patch the IPA HTTP/LDAP certs issued during ipa-server-install or ipa-replica-prepare and IPA client host certs issued during ipa-client-install will be OK. But for service and host certs issued due to user requests this is the case. Cheers, Fraser From ftweedal at redhat.com Tue Dec 8 09:06:39 2015 From: ftweedal at redhat.com (Fraser Tweedale) Date: Tue, 8 Dec 2015 19:06:39 +1000 Subject: [Freeipa-devel] [PATCH] 0046 Create server certs with DNS altname In-Reply-To: <56660D1D.6000109@redhat.com> References: <20151207052611.GG23644@dhcp-40-8.bne.redhat.com> <5665813B.90406@redhat.com> <20151207224639.GL23644@dhcp-40-8.bne.redhat.com> <56660D1D.6000109@redhat.com> Message-ID: <20151208090638.GQ23644@dhcp-40-8.bne.redhat.com> On Mon, Dec 07, 2015 at 05:50:05PM -0500, Rob Crittenden wrote: > Fraser Tweedale wrote: > > On Mon, Dec 07, 2015 at 01:53:15PM +0100, Martin Kosek wrote: > >> On 12/07/2015 06:26 AM, Fraser Tweedale wrote: > >>> The attached patch fixes > >>> https://fedorahosted.org/freeipa/ticket/4970. > >>> > >>> Note that the problem is addressed by adding the appropriate request > >>> extension to the CSR; the fix does not involve changing the default > >>> profile behaviour, which is complicated (see ticket for details). > >> > >> Thanks for the patch! This is something we should really fix, I already get > >> warnings in my Python scripts when I hit sites protected by such HTTPS cert: > >> > >> /usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:264: > >> SubjectAltNameWarning: Certificate for projects.engineering.redhat.com has no > >> `subjectAltName`, falling back to check for a `commonName` for now. This > >> feature is being removed by major browsers and deprecated by RFC 2818. (See > >> https://github.com/shazow/urllib3/issues/497 for details.) > >> > >> Should we split ticket 4970, for the FreeIPA server part and then for cert > >> profile part? As it looks like the FreeIPA server will be fixed even in FreeIPA > >> 4.3.x and the other part later. > >> > >> How difficult do you see the general FreeIPA Certificate Profile part of this > >> request? Is it a too big task to handle in 4.4 time frame? > >> > > I will split the ticket and would suggest 4.4 Backlog - it might be > > doable but is a lower priority than e.g. Sub-CAs. > > If you are going to defer the profile part then you should probably > update the client to also include a SAN if --request-cert is provided. > > rob > Yes, good idea. Updated patch attached. Cheers, Fraser -------------- next part -------------- From 72e24bb90fbb331644f0509371872a17f86007cb Mon Sep 17 00:00:00 2001 From: Fraser Tweedale Date: Mon, 7 Dec 2015 16:14:28 +1100 Subject: [PATCH] Create server and host certs with DNS altname Currently server (HTTP / LDAP) certs are created without a Subject Alternative Name extension during server install, replica prepare and host enrolment, a potentially problematic violation of RFC 2818. Add the hostname as a SAN dNSName when these certs are created. (Certmonger adds an appropriate request extension when renewing the certificate, so nothing needs to be done for renewal). Fixes: https://fedorahosted.org/freeipa/ticket/4970 --- ipa-client/ipa-install/ipa-client-install | 2 +- ipapython/certmonger.py | 9 ++++++++- ipaserver/install/certs.py | 8 ++++++-- 3 files changed, 15 insertions(+), 4 deletions(-) diff --git a/ipa-client/ipa-install/ipa-client-install b/ipa-client/ipa-install/ipa-client-install index 974dd1da8bf3f5836170ca67d2f4c298e7ec6844..fd273597944b8d07a2c9bdb96f6a32566085747f 100755 --- a/ipa-client/ipa-install/ipa-client-install +++ b/ipa-client/ipa-install/ipa-client-install @@ -1167,7 +1167,7 @@ def configure_certmonger(fstore, subject_base, cli_realm, hostname, options, try: certmonger.request_cert(nssdb=paths.IPA_NSSDB_DIR, nickname='Local IPA host', - subject=subject, + subject=subject, dns=[hostname], principal=principal, passwd_fname=passwd_fname) except Exception: diff --git a/ipapython/certmonger.py b/ipapython/certmonger.py index 2a4e43d3c5d5746134fc5b11a2d01d05f67a2e26..8901d3bb068cc1e0c94ea6c5a093d054ce0557e6 100644 --- a/ipapython/certmonger.py +++ b/ipapython/certmonger.py @@ -299,9 +299,14 @@ def add_subject(request_id, subject): add_request_value(request_id, 'template-subject', subject) -def request_cert(nssdb, nickname, subject, principal, passwd_fname=None): +def request_cert( + nssdb, nickname, subject, principal, passwd_fname=None, + dns=None): """ Execute certmonger to request a server certificate. + + ``dns`` + A sequence of DNS names to appear in SAN request extension. """ cm = _certmonger() ca_path = cm.obj_if.find_ca_by_nickname('IPA') @@ -312,6 +317,8 @@ def request_cert(nssdb, nickname, subject, principal, passwd_fname=None): KEY_LOCATION=nssdb, KEY_NICKNAME=nickname, SUBJECT=subject, PRINCIPAL=[principal], CA=ca_path) + if dns is not None and len(dns) > 0: + request_parameters['DNS'] = dns if passwd_fname: request_parameters['KEY_PIN_FILE'] = passwd_fname result = cm.obj_if.add_request(request_parameters) diff --git a/ipaserver/install/certs.py b/ipaserver/install/certs.py index c918791f0be7a17e20123fe6f94c4ac0bbf09d7b..bd1792d32246bc3034c5403f1d868e0966ec0014 100644 --- a/ipaserver/install/certs.py +++ b/ipaserver/install/certs.py @@ -335,7 +335,7 @@ class CertDB(object): cdb = self if subject is None: subject=DN(('CN', hostname), self.subject_base) - self.request_cert(subject) + self.request_cert(subject, san_dnsnames=[hostname]) cdb.issue_server_cert(self.certreq_fname, self.certder_fname) self.import_cert(self.certder_fname, nickname) fd = open(self.certder_fname, "r") @@ -359,7 +359,9 @@ class CertDB(object): os.unlink(self.certreq_fname) os.unlink(self.certder_fname) - def request_cert(self, subject, certtype="rsa", keysize="2048"): + def request_cert( + self, subject, certtype="rsa", keysize="2048", + san_dnsnames=None): assert isinstance(subject, DN) self.create_noise_file() self.setup_cert_request() @@ -370,6 +372,8 @@ class CertDB(object): "-z", self.noise_fname, "-f", self.passwd_fname, "-a"] + if san_dnsnames is not None and len(san_dnsnames) > 0: + args += ['-8', ','.join(san_dnsnames)] (stdout, stderr, returncode) = self.run_certutil(args) os.remove(self.noise_fname) return (stdout, stderr) -- 2.4.3 From mbasti at redhat.com Tue Dec 8 09:09:14 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 8 Dec 2015 10:09:14 +0100 Subject: [Freeipa-devel] [PATCH 0372] CI: installation tests In-Reply-To: <56669D0C.4050201@redhat.com> References: <5664A718.6080101@redhat.com> <56657A83.6020702@redhat.com> <56658A5E.8010603@redhat.com> <56658BB5.5060700@redhat.com> <56659C6D.4040600@redhat.com> <56659CE2.1060304@redhat.com> <56659D0E.5060806@redhat.com> <5665FDC0.7020907@redhat.com> <56669D0C.4050201@redhat.com> Message-ID: <56669E3A.8000208@redhat.com> On 08.12.2015 10:04, Oleg Fayans wrote: > ACK Pushed to master: a11cddd75b4e887998ad6fd52a05f87e0354ea30 How about patch mbasti-371, this will not work without it. > > On 12/07/2015 10:44 PM, Martin Basti wrote: >> >> On 07.12.2015 15:51, Oleg Fayans wrote: >>> On 12/07/2015 03:51 PM, Martin Basti wrote: >>>> On 07.12.2015 15:49, Oleg Fayans wrote: >>>>> Hi, >>>>> >>>>> On 12/07/2015 02:37 PM, Martin Basti wrote: >>>>>> On 07.12.2015 14:32, Martin Basti wrote: >>>>>>> On 07.12.2015 13:24, Oleg Fayans wrote: >>>>>>>> Hi Martin, >>>>>>>> >>>>>>>> I would prefer both install_kra and install_ca methods to have >>>>>>>> raiseonerr parameter set to True by default. We need a way to test >>>>>>>> negatives and analyze results. >>>>>>>> Mine looks like this: >>>>>>>> >>>>>>>> def install_kra(host, domain_level=None, >>>>>>>> first_instance=False, raiseonerr=True): >>>>>>>> if not domain_level: >>>>>>>> domain_level = host.config.domain_level >>>>>>>> command = ["ipa-kra-install", "-U", "-p", >>>>>>>> host.config.dirman_password] >>>>>>>> if domain_level == DOMAIN_LEVEL_0 and not first_instance: >>>>>>>> replica_file = get_replica_filename(host) >>>>>>>> command.append(replica_file) >>>>>>>> return host.run_command(command, raiseonerr=raiseonerr) >>>>>>>> >>>>>>>> The rest looks good to me, but I did not run the tests yet. >>>>>>> Sounds good, I will amend the patche later. >>>>>> I changed my mind, should not be the domain_level value get from >>>>>> function domainlevel(host)? >>>>> We should have a way to test negatives, like providing replica file at >>>>> domain level=1 and not providing at domain level=0. So these functions >>>>> should either accept replica file as a parameter, or arbitrary domain >>>>> level >>>> Agree, but I meant this >>>> >>>> if not domain_level: >>>> - domain_level = host.config.domain_level >>>> + domain_level = domainlevel(host) >>> Oh, yes, you are right, that's better. >>> >>>>>> Martin^2 >>>>>> >>>>>>>> On 12/06/2015 10:22 PM, Martin Basti wrote: >>>>>>>>> My favorite today \o/ --> 67 <-- \o/ test cases, no more manual >>>>>>>>> testing >>>>>>>>> of installers \o/. >>>>>>>>> >>>>>>>>> Test suite contains: 6 combination how to install components on >>>>>>>>> replica >>>>>>>>> X 4 combinations of server installation + 3 extra server tests >>>>>>>>> >>>>>>>>> To save time tests install 1 master and 3 replicas per test class >>>>>>>>> (except extra server tests): >>>>>>>>> Class name specifies what is installed on master. >>>>>>>>> >>>>>>>>> Remember, option "-k" is your friend >>>>>>>>> $ ipa-run-tests -k '>>>>>>>> regexp)>' >>>>>>>>> otherwise you will need a lot of time until tests finish. >>>>>>>>> >>>>>>>>> To list all tests: >>>>>>>>> $ ipa-run-tests test_integration/test_installation.py >>>>>>>>> --collect-only >>>>>>>>> >>>>>>>>> >>>>>>>>> Patch attached, it needs to have attached all patches I sent today >>>>>>>>> and >>>>>>>>> also attached workaround patch (Martin3 will provide proper fix) >>>>>>>>> >>>>>>>>> I haven't had time/mood/resources to test this patch with domain >>>>>>>>> level >>>>>>>>> 0, so testing this patch with domain level 0 is appreciated. >>>>>>>>> >>>>>>>>> >>>>>>>>> Oleg, I added some methods to tasks.py which you may want to reuse. >>>>>>>>> >>>>>>>>> Martin^2 >>>>>>>>> >> Updated patch attached. >> >> Still some tests are failing, not sure if it is test issue or bugs. I >> will test later. From ofayans at redhat.com Tue Dec 8 09:22:55 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Tue, 8 Dec 2015 10:22:55 +0100 Subject: [Freeipa-devel] [PATCH 0371] CI: fix function that prepares /etc/hosts In-Reply-To: <5664A016.5060900@redhat.com> References: <5664A016.5060900@redhat.com> Message-ID: <5666A16F.9000601@redhat.com> ACK On 12/06/2015 09:52 PM, Martin Basti wrote: > Without this fix, function removes more entries from /etc/host than is > required, and it causes installation failure in tests without DNS > > Patch attached. -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From ofayans at redhat.com Tue Dec 8 09:23:31 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Tue, 8 Dec 2015 10:23:31 +0100 Subject: [Freeipa-devel] [PATCH 0372] CI: installation tests In-Reply-To: <56669E3A.8000208@redhat.com> References: <5664A718.6080101@redhat.com> <56657A83.6020702@redhat.com> <56658A5E.8010603@redhat.com> <56658BB5.5060700@redhat.com> <56659C6D.4040600@redhat.com> <56659CE2.1060304@redhat.com> <56659D0E.5060806@redhat.com> <5665FDC0.7020907@redhat.com> <56669D0C.4050201@redhat.com> <56669E3A.8000208@redhat.com> Message-ID: <5666A193.6030603@redhat.com> On 12/08/2015 10:09 AM, Martin Basti wrote: > > > On 08.12.2015 10:04, Oleg Fayans wrote: >> ACK > Pushed to master: a11cddd75b4e887998ad6fd52a05f87e0354ea30 > > How about patch mbasti-371, this will not work without it. Looks safe. Could be pushed too :) >> >> On 12/07/2015 10:44 PM, Martin Basti wrote: >>> >>> On 07.12.2015 15:51, Oleg Fayans wrote: >>>> On 12/07/2015 03:51 PM, Martin Basti wrote: >>>>> On 07.12.2015 15:49, Oleg Fayans wrote: >>>>>> Hi, >>>>>> >>>>>> On 12/07/2015 02:37 PM, Martin Basti wrote: >>>>>>> On 07.12.2015 14:32, Martin Basti wrote: >>>>>>>> On 07.12.2015 13:24, Oleg Fayans wrote: >>>>>>>>> Hi Martin, >>>>>>>>> >>>>>>>>> I would prefer both install_kra and install_ca methods to have >>>>>>>>> raiseonerr parameter set to True by default. We need a way to test >>>>>>>>> negatives and analyze results. >>>>>>>>> Mine looks like this: >>>>>>>>> >>>>>>>>> def install_kra(host, domain_level=None, >>>>>>>>> first_instance=False, raiseonerr=True): >>>>>>>>> if not domain_level: >>>>>>>>> domain_level = host.config.domain_level >>>>>>>>> command = ["ipa-kra-install", "-U", "-p", >>>>>>>>> host.config.dirman_password] >>>>>>>>> if domain_level == DOMAIN_LEVEL_0 and not first_instance: >>>>>>>>> replica_file = get_replica_filename(host) >>>>>>>>> command.append(replica_file) >>>>>>>>> return host.run_command(command, raiseonerr=raiseonerr) >>>>>>>>> >>>>>>>>> The rest looks good to me, but I did not run the tests yet. >>>>>>>> Sounds good, I will amend the patche later. >>>>>>> I changed my mind, should not be the domain_level value get from >>>>>>> function domainlevel(host)? >>>>>> We should have a way to test negatives, like providing replica >>>>>> file at >>>>>> domain level=1 and not providing at domain level=0. So these >>>>>> functions >>>>>> should either accept replica file as a parameter, or arbitrary domain >>>>>> level >>>>> Agree, but I meant this >>>>> >>>>> if not domain_level: >>>>> - domain_level = host.config.domain_level >>>>> + domain_level = domainlevel(host) >>>> Oh, yes, you are right, that's better. >>>> >>>>>>> Martin^2 >>>>>>> >>>>>>>>> On 12/06/2015 10:22 PM, Martin Basti wrote: >>>>>>>>>> My favorite today \o/ --> 67 <-- \o/ test cases, no more manual >>>>>>>>>> testing >>>>>>>>>> of installers \o/. >>>>>>>>>> >>>>>>>>>> Test suite contains: 6 combination how to install components on >>>>>>>>>> replica >>>>>>>>>> X 4 combinations of server installation + 3 extra server tests >>>>>>>>>> >>>>>>>>>> To save time tests install 1 master and 3 replicas per test class >>>>>>>>>> (except extra server tests): >>>>>>>>>> Class name specifies what is installed on master. >>>>>>>>>> >>>>>>>>>> Remember, option "-k" is your friend >>>>>>>>>> $ ipa-run-tests -k '>>>>>>>>> regexp)>' >>>>>>>>>> otherwise you will need a lot of time until tests finish. >>>>>>>>>> >>>>>>>>>> To list all tests: >>>>>>>>>> $ ipa-run-tests test_integration/test_installation.py >>>>>>>>>> --collect-only >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Patch attached, it needs to have attached all patches I sent >>>>>>>>>> today >>>>>>>>>> and >>>>>>>>>> also attached workaround patch (Martin3 will provide proper fix) >>>>>>>>>> >>>>>>>>>> I haven't had time/mood/resources to test this patch with domain >>>>>>>>>> level >>>>>>>>>> 0, so testing this patch with domain level 0 is appreciated. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Oleg, I added some methods to tasks.py which you may want to >>>>>>>>>> reuse. >>>>>>>>>> >>>>>>>>>> Martin^2 >>>>>>>>>> >>> Updated patch attached. >>> >>> Still some tests are failing, not sure if it is test issue or bugs. I >>> will test later. > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From mbasti at redhat.com Tue Dec 8 09:26:24 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 8 Dec 2015 10:26:24 +0100 Subject: [Freeipa-devel] [PATCH 0371] CI: fix function that prepares /etc/hosts In-Reply-To: <5666A16F.9000601@redhat.com> References: <5664A016.5060900@redhat.com> <5666A16F.9000601@redhat.com> Message-ID: <5666A240.70203@redhat.com> On 08.12.2015 10:22, Oleg Fayans wrote: > ACK > > On 12/06/2015 09:52 PM, Martin Basti wrote: >> Without this fix, function removes more entries from /etc/host than is >> required, and it causes installation failure in tests without DNS >> >> Patch attached. Pushed to master: e4259d5b49a6f501f0a6f1b020bf492cac06e1c7 From mbasti at redhat.com Tue Dec 8 09:31:09 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 8 Dec 2015 10:31:09 +0100 Subject: [Freeipa-devel] [PATCH 522] replica promotion: allow OTP bulk client enrollment In-Reply-To: <56668C43.3000908@redhat.com> References: <565F0D00.6050805@redhat.com> <56653386.5020703@redhat.com> <5665E807.3070600@redhat.com> <56668C43.3000908@redhat.com> Message-ID: <5666A35D.8040203@redhat.com> On 08.12.2015 08:52, Jan Cholasta wrote: > On 7.12.2015 21:11, Martin Basti wrote: >> >> >> On 07.12.2015 08:21, Jan Cholasta wrote: >>> On 2.12.2015 16:23, Jan Cholasta wrote: >>>> Hi, >>>> >>>> the attached patch fixes >>>> . >>>> >>>> Note that you still have to provide admin password in >>>> ipa-replica-install, either using --admin-password or interactively, >>>> because: >>>> >>>> a) Admin password is required for replica promotion. This will be >>>> fixed >>>> with . >>>> >>>> Patches are on the list: >>>> . >>>> >>>> >>> >>> Pushed. >>> >>>> >>>> >>>> b) Admin password is required for connection check. This will be fixed >>>> with . >>> >>> Martin Basti pointed out that admin password should not be asked >>> interactively during OTP replica promotion. Fixed. >>> >>> Updated and rebased patch attached. >>> >>> >>> >> >> 1) >> [root at vm-058-138 ~]# ipa-replica-install --server >> vm-058-137.abc.idm.lab.eng.brq.redhat.com --domain >> abc.idm.lab.eng.brq.redhat.com --password=bubak --setup-ca >> Configuring client side components >> Password for admin at ABC.IDM.LAB.ENG.BRQ.REDHAT.COM: >> >> IMO password should be asked first, before any installation begins (IMO >> this is for conncheck) > > The same thing happens without my patch. Could you file a ticket? https://fedorahosted.org/freeipa/ticket/5525 > >> >> 2) >> When host is not in ipaservers hostgroup. Also I would expect different >> error message >> ipa-replica-install --server vm-058-137.abc.idm.lab.eng.brq.redhat.com >> --domain abc.idm.lab.eng.brq.redhat.com --password=bubak --setup-ca >> --skip-conncheck >> >> .... >> step() >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >> line 352, in >> step = lambda: next(self.__gen) >> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >> line 81, in run_generator_with_yield_from >> six.reraise(*exc_info) >> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >> line 59, in run_generator_with_yield_from >> value = gen.send(prev_value) >> File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >> line 63, in _install >> for nothing in self._installer(self.parent): >> File >> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >> >> line 1507, in main >> promote_check(self) >> File >> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >> >> line 374, in decorated >> func(installer) >> File >> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >> >> line 1002, in promote_check >> conn.connect(ccache=installer._ccache) >> File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 66, >> in connect >> conn = self.create_connection(*args, **kw) >> File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", >> line 199, in create_connection >> principal = krb_utils.get_principal(ccache_name=ccache) >> File "/usr/lib/python2.7/site-packages/ipalib/krb_utils.py", line >> 184, in get_principal >> raise errors.CCacheError(message=unicode(e)) >> >> 2015-12-07T16:23:40Z DEBUG The ipa-replica-install command failed, >> exception: CCacheError: Major (851968): Unspecified GSS failure. Minor >> code may provide more information, Minor (2529639053): No Kerberos >> credentials available >> 2015-12-07T16:23:40Z ERROR Major (851968): Unspecified GSS failure. >> Minor code may provide more information, Minor (2529639053): No Kerberos >> credentials available > > Fixed. > >> >> >> 3) >> This case is not handle very well: >> a) install client with OTP password >> b) install replica with the same OTP password (when host is no in >> ipaservers group, if host is in ipaservers group it works) >> >> ipa.ipapython.install.cli.install_tool(Replica): ERROR Major >> (851968): Unspecified GSS failure. Minor code may provide more >> information, Minor (2529639053): No Kerberos credentials available >> ipa.ipapython.install.cli.install_tool(Replica): ERROR The >> ipa-replica-install command failed. See /var/log/ipareplica-install.log >> for more information > > This is the same as 2). > >> >> 4) >> This is not user friendly >> I used wrong OTP password, can we somehow propagate the actual error >> from client install to stderr? >> >> ipa.ipapython.install.cli.install_tool(Replica): ERROR Configuration of >> client side components failed! >> ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' >> '--unattended' '--domain' 'abc.idm.lab.eng.brq.redhat.com' '--server' >> 'vm-058-137.abc.idm.lab.eng.brq.redhat.com' '--password' 'buba'' >> returned non-zero exit status 1 >> ipa.ipapython.install.cli.install_tool(Replica): ERROR The >> ipa-replica-install command failed. See /var/log/ipareplica-install.log >> for more information > > The same thing happens without my patch for any other error. Could you > file a ticket? https://fedorahosted.org/freeipa/ticket/5527 > > Updated patch attached. > Working on review From mbabinsk at redhat.com Tue Dec 8 09:45:27 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Tue, 8 Dec 2015 10:45:27 +0100 Subject: [Freeipa-devel] [PATCH 0113] properly add ACIs to custodia container during IPA upgrade Message-ID: <5666A6B7.8010705@redhat.com> fixes https://fedorahosted.org/freeipa/ticket/5524 -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0113-properly-add-ACIs-to-custodia-container-during-IPA-u.patch Type: text/x-patch Size: 3241 bytes Desc: not available URL: From cheimes at redhat.com Tue Dec 8 10:20:13 2015 From: cheimes at redhat.com (Christian Heimes) Date: Tue, 8 Dec 2015 11:20:13 +0100 Subject: [Freeipa-devel] [PATCH 0026] Workarounds for SELinux execmem violations in cryptography In-Reply-To: <5665D6F6.80905@redhat.com> References: <566599BE.1020008@redhat.com> <20151207151747.GZ8374@redhat.com> <5665A515.30507@redhat.com> <5665D6F6.80905@redhat.com> Message-ID: <5666AEDD.80505@redhat.com> On 2015-12-07 19:59, Petr Vobornik wrote: > On 7.12.2015 16:26, Christian Heimes wrote: >> On 2015-12-07 16:17, Alexander Bokovoy wrote: >>> On Mon, 07 Dec 2015, Christian Heimes wrote: >>>> The patch fixes SELinux violations in Fedora 23. >>>> >>>> Background: Recent versions of cryptography cause SELinux violation >>>> which will lead to a segfault, see >>>> https://bugzilla.redhat.com/show_bug.cgi?id=1277224 . The segfault only >>>> occurs in the context of Apache HTTPD (FreeIPA web ui) when >>>> cryptography.hazmat.backends.default_backend() is initialized. I'm >>>> working on a fix for cryptography but it will take a while. First I >>>> have >>>> to wait for a new upstream release of python-cffi. Armin Ronacher plans >>>> to release cffi 1.4 in two weeks. >>>> >>>> >>>> ipaserver.dcerpc uses M2Crypto again on Python 2.7 and Dogtag's >>>> pki.client no longer tries to use PyOpenSSL instead of Python's ssl >>>> module. >>>> >>>> Some dependencies like Dogtag's pki.client library and custodia use >>>> python-requsts to make HTTPS connection. python-requests prefers >>>> PyOpenSSL over Python's stdlib ssl module. PyOpenSSL is build on top >>>> of python-cryptography which trigger a execmem SELinux violation >>>> in the context of Apache HTTPD (httpd_execmem). >>>> When requests is imported, it always tries to import pyopenssl glue >>>> code from urllib3's contrib directory. The import of PyOpenSSL is >>>> enough to trigger the SELinux denial. >>>> A hack in wsgi.py prevents the import by raising an ImportError. >>> ACK. Thanks for these patches. >>> >>> Note to Debian/Ubuntu maintainers: AppArmor 'support' in python-cffi >>> already detects apparmor by looking into /proc and disabling the use of >>> writeable and executable memory. On those platforms I suspect recent >>> enough python-cryptography would work without problem by downgrading own >>> feature set. The code in this patches should be harmless, though. >> >> Cryptography's core depends on dynamic callbacks. There is no "downgrade >> feature-set" feature. >> >> I guess the libffi uses the broken and potential dangerous workaround >> with two shared mmap() with file backend. >> (http://www.akkadia.org/drepper/selinux-mem.html). The approach requires >> a writeable, executable temp file and breaks isolation between a parent >> process and all its forked child processes. >> >> Christian >> > > The patch needs to be rebased to 4-2 branch to be usable on Fedora 23 - > FreeIPA 4.2.3. For FreeIPA 4.2 only the patch in wsgi.py is needed. The older version doesn't use cryptography for RC4. I've attached a patch. Christian -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-cheimes-0026-2-Workarounds-for-SELinux-execmem-violations-in-crypto.patch Type: text/x-patch Size: 1909 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From pspacek at redhat.com Tue Dec 8 10:33:23 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 8 Dec 2015 11:33:23 +0100 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <56619841.3010809@redhat.com> References: <565F34FD.5090204@redhat.com> <20151203090954.GB5209@mail.corp.redhat.com> <566058E2.6040201@redhat.com> <20151204111921.GE28768@mail.corp.redhat.com> <56619841.3010809@redhat.com> Message-ID: <5666B1F3.4070903@redhat.com> On 4.12.2015 14:42, Rob Crittenden wrote: > Lukas Slebodnik wrote: >> > On (03/12/15 09:59), Rob Crittenden wrote: >>> >> Lukas Slebodnik wrote: >>>> >>> On (02/12/15 13:14), Rob Crittenden wrote: >>>>> >>>> Is it still mandatory that tests pass the unit tests before acceptance? >>>> >>> Unit test could be executed as part of "%check" phase in spec files. >>>> >>> I recently added C-base unit tests there. >>>> >>> >>>> >>> I was not bale to run "make tests" there because many tests failed. >>>> >>> If they require real IPA server for execution ( or lite server) >>>> >>> then they are not unit test but integration tests. >>>> >>> One solution would be to skip them or to usw cwrap[1] + lite server. >>>> >>> So it can be run also in mock/koji which has many restrictions. >>> >> >>> >> It would represent quite a lot of work but it may be a good idea to >>> >> investigate. Ipsilon uses cwrap for its tests so some of the >>> >> configuration can be gleaned from that. >>> >> >>> >> I would definitely be opposed to this as part of the freeipa.spec in the >> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> > What do you mean by this part? >> > >> > Did you mean "running make tests" in spec file? >> > If yes, could you elaborate why it's not good idea? >> > many projects run tests in "%check" >> > sssd, certmonger, glibc ... >> > >> > Currently only C-based test are executed. And I added it only recently. > Because it would be overkill during development. The expectation is that > developers and reviewers run the tests before submission/acceptance. If > they fail to do that then it will be obvious. > >>> >> git tree. In Fedora it might help find problems when rawhide becomes >>> >> Fedora.next so it would provide some value there. >>> >> >>>> >>> >>>> >>> Also lint should be also part of "%check" phase and not part of >>>> >>> ordinary build. >>>> >>> BTW I could not see a lint[2] in fedora build at all. So I'm not sure >>>> >>> if it is executed with upstream spec file. >>> >> >>> >> It isn't there because the expectation is that lint already passes as >>> >> part of the release process. I don't see the value on running lint on >>> >> release tarballs. >>> >> >> > That's just an expectation. It needn't be true. Your initial mail was about >> > stricter review process. And automating things is best way how to >> > enforce it. So reviewer would just build rpms and if "%check" phase >> > will not pass then he will not continue with review. >> > If it will be part of "%check" then you can use mock and easily ensure >> > that test passes on stable fedora and fedora rawhide (and maybe centOS) > By the time downstream gets a tarball it is too late to fix lint errors. > If upstream is doing a release with lint errors then there is something > deeply wrong with the release process. If someone wants to add it to the > downstream spec files I'm not going to complain, I just find it > extremely unlikely that it will provide any value, ever. Sorry Rob, but I disagree. Lint already caught couple cases where Requires: were not properly updated so IPA code was referencing non-existing code in Dogtag/Custodia packages and so on. So clearly there is some value in it. -- Petr^2 Spacek From dkupka at redhat.com Tue Dec 8 11:19:24 2015 From: dkupka at redhat.com (David Kupka) Date: Tue, 8 Dec 2015 12:19:24 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <56668D30.8060308@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> <5665843C.2090507@redhat.com> <56658C89.8050103@redhat.com> <56668D30.8060308@redhat.com> Message-ID: <5666BCBC.9040005@redhat.com> On 08/12/15 08:56, Petr Spacek wrote: > On 7.12.2015 14:41, David Kupka wrote: >> +def is_host_resolvable(fqdn): >> + if not isinstance(fqdn, DNSName): >> + fqdn = DNSName(fqdn) >> + for rdtype in (rdatatype.A, rdatatype.AAAA): >> + try: >> + resolver.query(fqdn.make_absolute(), rdtype) >> + except DNSException: >> + continue >> + else: >> + return True >> + >> + return False >> > > NACK, you are re-introducing duplicate function which was removed in > 498471e4aed1367b72cd74d15811d0584a6ee268. > > Please amend the patch ASAP to use new verify_host_resolvable() function so I > can test it and get it into 4.3. > Updated patches attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0058.8-dns-do-not-add-forward-zone-if-it-is-already-resolva.patch Type: text/x-patch Size: 11740 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0064.5-dns-Check-if-domain-already-exists.patch Type: text/x-patch Size: 12878 bytes Desc: not available URL: From mbasti at redhat.com Tue Dec 8 11:49:27 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 8 Dec 2015 12:49:27 +0100 Subject: [Freeipa-devel] [PATCH 522] replica promotion: allow OTP bulk client enrollment In-Reply-To: <5666A35D.8040203@redhat.com> References: <565F0D00.6050805@redhat.com> <56653386.5020703@redhat.com> <5665E807.3070600@redhat.com> <56668C43.3000908@redhat.com> <5666A35D.8040203@redhat.com> Message-ID: <5666C3C7.2070505@redhat.com> On 08.12.2015 10:31, Martin Basti wrote: > > > On 08.12.2015 08:52, Jan Cholasta wrote: >> On 7.12.2015 21:11, Martin Basti wrote: >>> >>> >>> On 07.12.2015 08:21, Jan Cholasta wrote: >>>> On 2.12.2015 16:23, Jan Cholasta wrote: >>>>> Hi, >>>>> >>>>> the attached patch fixes >>>>> . >>>>> >>>>> Note that you still have to provide admin password in >>>>> ipa-replica-install, either using --admin-password or interactively, >>>>> because: >>>>> >>>>> a) Admin password is required for replica promotion. This will be >>>>> fixed >>>>> with . >>>>> >>>>> Patches are on the list: >>>>> . >>>>> >>>>> >>>> >>>> Pushed. >>>> >>>>> >>>>> >>>>> b) Admin password is required for connection check. This will be >>>>> fixed >>>>> with . >>>> >>>> Martin Basti pointed out that admin password should not be asked >>>> interactively during OTP replica promotion. Fixed. >>>> >>>> Updated and rebased patch attached. >>>> >>>> >>>> >>> >>> 1) >>> [root at vm-058-138 ~]# ipa-replica-install --server >>> vm-058-137.abc.idm.lab.eng.brq.redhat.com --domain >>> abc.idm.lab.eng.brq.redhat.com --password=bubak --setup-ca >>> Configuring client side components >>> Password for admin at ABC.IDM.LAB.ENG.BRQ.REDHAT.COM: >>> >>> IMO password should be asked first, before any installation begins (IMO >>> this is for conncheck) >> >> The same thing happens without my patch. Could you file a ticket? > https://fedorahosted.org/freeipa/ticket/5525 > >> >>> >>> 2) >>> When host is not in ipaservers hostgroup. Also I would expect different >>> error message >>> ipa-replica-install --server vm-058-137.abc.idm.lab.eng.brq.redhat.com >>> --domain abc.idm.lab.eng.brq.redhat.com --password=bubak --setup-ca >>> --skip-conncheck >>> >>> .... >>> step() >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>> line 352, in >>> step = lambda: next(self.__gen) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>> line 81, in run_generator_with_yield_from >>> six.reraise(*exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>> line 59, in run_generator_with_yield_from >>> value = gen.send(prev_value) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>> line 63, in _install >>> for nothing in self._installer(self.parent): >>> File >>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >>> >>> line 1507, in main >>> promote_check(self) >>> File >>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >>> >>> line 374, in decorated >>> func(installer) >>> File >>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >>> >>> line 1002, in promote_check >>> conn.connect(ccache=installer._ccache) >>> File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 66, >>> in connect >>> conn = self.create_connection(*args, **kw) >>> File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", >>> line 199, in create_connection >>> principal = krb_utils.get_principal(ccache_name=ccache) >>> File "/usr/lib/python2.7/site-packages/ipalib/krb_utils.py", line >>> 184, in get_principal >>> raise errors.CCacheError(message=unicode(e)) >>> >>> 2015-12-07T16:23:40Z DEBUG The ipa-replica-install command failed, >>> exception: CCacheError: Major (851968): Unspecified GSS failure. Minor >>> code may provide more information, Minor (2529639053): No Kerberos >>> credentials available >>> 2015-12-07T16:23:40Z ERROR Major (851968): Unspecified GSS failure. >>> Minor code may provide more information, Minor (2529639053): No >>> Kerberos >>> credentials available >> >> Fixed. >> >>> >>> >>> 3) >>> This case is not handle very well: >>> a) install client with OTP password >>> b) install replica with the same OTP password (when host is no in >>> ipaservers group, if host is in ipaservers group it works) >>> >>> ipa.ipapython.install.cli.install_tool(Replica): ERROR Major >>> (851968): Unspecified GSS failure. Minor code may provide more >>> information, Minor (2529639053): No Kerberos credentials available >>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The >>> ipa-replica-install command failed. See /var/log/ipareplica-install.log >>> for more information >> >> This is the same as 2). >> >>> >>> 4) >>> This is not user friendly >>> I used wrong OTP password, can we somehow propagate the actual error >>> from client install to stderr? >>> >>> ipa.ipapython.install.cli.install_tool(Replica): ERROR Configuration of >>> client side components failed! >>> ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' >>> '--unattended' '--domain' 'abc.idm.lab.eng.brq.redhat.com' '--server' >>> 'vm-058-137.abc.idm.lab.eng.brq.redhat.com' '--password' 'buba'' >>> returned non-zero exit status 1 >>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The >>> ipa-replica-install command failed. See /var/log/ipareplica-install.log >>> for more information >> >> The same thing happens without my patch for any other error. Could >> you file a ticket? > > https://fedorahosted.org/freeipa/ticket/5527 > >> >> Updated patch attached. >> > Working on review > Is this expected that client will be installed even if there is not enough privileges to install replica? # ipa-replica-install --server vm-058-137.abc.idm.lab.eng.brq.redhat.com --domain abc.idm.lab.eng.brq.redhat.com --password bubak --skip-conncheck Configuring client side components ipa.ipapython.install.cli.install_tool(Replica): ERROR Insufficient privileges to promote the server. ipa.ipapython.install.cli.install_tool(Replica): ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information From jcholast at redhat.com Tue Dec 8 12:09:53 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 8 Dec 2015 13:09:53 +0100 Subject: [Freeipa-devel] [PATCH 522] replica promotion: allow OTP bulk client enrollment In-Reply-To: <5666C3C7.2070505@redhat.com> References: <565F0D00.6050805@redhat.com> <56653386.5020703@redhat.com> <5665E807.3070600@redhat.com> <56668C43.3000908@redhat.com> <5666A35D.8040203@redhat.com> <5666C3C7.2070505@redhat.com> Message-ID: <5666C891.1020205@redhat.com> On 8.12.2015 12:49, Martin Basti wrote: > > > On 08.12.2015 10:31, Martin Basti wrote: >> >> >> On 08.12.2015 08:52, Jan Cholasta wrote: >>> On 7.12.2015 21:11, Martin Basti wrote: >>>> >>>> >>>> On 07.12.2015 08:21, Jan Cholasta wrote: >>>>> On 2.12.2015 16:23, Jan Cholasta wrote: >>>>>> Hi, >>>>>> >>>>>> the attached patch fixes >>>>>> . >>>>>> >>>>>> Note that you still have to provide admin password in >>>>>> ipa-replica-install, either using --admin-password or interactively, >>>>>> because: >>>>>> >>>>>> a) Admin password is required for replica promotion. This will be >>>>>> fixed >>>>>> with . >>>>>> >>>>>> Patches are on the list: >>>>>> . >>>>>> >>>>>> >>>>> >>>>> Pushed. >>>>> >>>>>> >>>>>> >>>>>> b) Admin password is required for connection check. This will be >>>>>> fixed >>>>>> with . >>>>> >>>>> Martin Basti pointed out that admin password should not be asked >>>>> interactively during OTP replica promotion. Fixed. >>>>> >>>>> Updated and rebased patch attached. >>>>> >>>>> >>>>> >>>> >>>> 1) >>>> [root at vm-058-138 ~]# ipa-replica-install --server >>>> vm-058-137.abc.idm.lab.eng.brq.redhat.com --domain >>>> abc.idm.lab.eng.brq.redhat.com --password=bubak --setup-ca >>>> Configuring client side components >>>> Password for admin at ABC.IDM.LAB.ENG.BRQ.REDHAT.COM: >>>> >>>> IMO password should be asked first, before any installation begins (IMO >>>> this is for conncheck) >>> >>> The same thing happens without my patch. Could you file a ticket? >> https://fedorahosted.org/freeipa/ticket/5525 >> >>> >>>> >>>> 2) >>>> When host is not in ipaservers hostgroup. Also I would expect different >>>> error message >>>> ipa-replica-install --server vm-058-137.abc.idm.lab.eng.brq.redhat.com >>>> --domain abc.idm.lab.eng.brq.redhat.com --password=bubak --setup-ca >>>> --skip-conncheck >>>> >>>> .... >>>> step() >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 352, in >>>> step = lambda: next(self.__gen) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>> line 81, in run_generator_with_yield_from >>>> six.reraise(*exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>> line 59, in run_generator_with_yield_from >>>> value = gen.send(prev_value) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>> line 63, in _install >>>> for nothing in self._installer(self.parent): >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >>>> >>>> line 1507, in main >>>> promote_check(self) >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >>>> >>>> line 374, in decorated >>>> func(installer) >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >>>> >>>> line 1002, in promote_check >>>> conn.connect(ccache=installer._ccache) >>>> File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 66, >>>> in connect >>>> conn = self.create_connection(*args, **kw) >>>> File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", >>>> line 199, in create_connection >>>> principal = krb_utils.get_principal(ccache_name=ccache) >>>> File "/usr/lib/python2.7/site-packages/ipalib/krb_utils.py", line >>>> 184, in get_principal >>>> raise errors.CCacheError(message=unicode(e)) >>>> >>>> 2015-12-07T16:23:40Z DEBUG The ipa-replica-install command failed, >>>> exception: CCacheError: Major (851968): Unspecified GSS failure. Minor >>>> code may provide more information, Minor (2529639053): No Kerberos >>>> credentials available >>>> 2015-12-07T16:23:40Z ERROR Major (851968): Unspecified GSS failure. >>>> Minor code may provide more information, Minor (2529639053): No >>>> Kerberos >>>> credentials available >>> >>> Fixed. >>> >>>> >>>> >>>> 3) >>>> This case is not handle very well: >>>> a) install client with OTP password >>>> b) install replica with the same OTP password (when host is no in >>>> ipaservers group, if host is in ipaservers group it works) >>>> >>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR Major >>>> (851968): Unspecified GSS failure. Minor code may provide more >>>> information, Minor (2529639053): No Kerberos credentials available >>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The >>>> ipa-replica-install command failed. See /var/log/ipareplica-install.log >>>> for more information >>> >>> This is the same as 2). >>> >>>> >>>> 4) >>>> This is not user friendly >>>> I used wrong OTP password, can we somehow propagate the actual error >>>> from client install to stderr? >>>> >>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR Configuration of >>>> client side components failed! >>>> ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' >>>> '--unattended' '--domain' 'abc.idm.lab.eng.brq.redhat.com' '--server' >>>> 'vm-058-137.abc.idm.lab.eng.brq.redhat.com' '--password' 'buba'' >>>> returned non-zero exit status 1 >>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The >>>> ipa-replica-install command failed. See /var/log/ipareplica-install.log >>>> for more information >>> >>> The same thing happens without my patch for any other error. Could >>> you file a ticket? >> >> https://fedorahosted.org/freeipa/ticket/5527 >> >>> >>> Updated patch attached. >>> >> Working on review >> > > Is this expected that client will be installed even if there is not > enough privileges to install replica? > > # ipa-replica-install --server > vm-058-137.abc.idm.lab.eng.brq.redhat.com --domain > abc.idm.lab.eng.brq.redhat.com --password bubak --skip-conncheck > Configuring client side components > ipa.ipapython.install.cli.install_tool(Replica): ERROR > Insufficient privileges to promote the server. > ipa.ipapython.install.cli.install_tool(Replica): ERROR The > ipa-replica-install command failed. See /var/log/ipareplica-install.log > for more information Yes. The check can't be done without the host keytab, which you get with ipa-client-install. If ipa-client-install wasn't monolithic, the check could be done earlier, but we are not there yet. The client should be probably uninstalled in case of failure, though. -- Jan Cholasta From mbasti at redhat.com Tue Dec 8 12:19:40 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 8 Dec 2015 13:19:40 +0100 Subject: [Freeipa-devel] [PATCH 522] replica promotion: allow OTP bulk client enrollment In-Reply-To: <5666C891.1020205@redhat.com> References: <565F0D00.6050805@redhat.com> <56653386.5020703@redhat.com> <5665E807.3070600@redhat.com> <56668C43.3000908@redhat.com> <5666A35D.8040203@redhat.com> <5666C3C7.2070505@redhat.com> <5666C891.1020205@redhat.com> Message-ID: <5666CADC.3070407@redhat.com> On 08.12.2015 13:09, Jan Cholasta wrote: > On 8.12.2015 12:49, Martin Basti wrote: >> >> >> On 08.12.2015 10:31, Martin Basti wrote: >>> >>> >>> On 08.12.2015 08:52, Jan Cholasta wrote: >>>> On 7.12.2015 21:11, Martin Basti wrote: >>>>> >>>>> >>>>> On 07.12.2015 08:21, Jan Cholasta wrote: >>>>>> On 2.12.2015 16:23, Jan Cholasta wrote: >>>>>>> Hi, >>>>>>> >>>>>>> the attached patch fixes >>>>>>> . >>>>>>> >>>>>>> Note that you still have to provide admin password in >>>>>>> ipa-replica-install, either using --admin-password or >>>>>>> interactively, >>>>>>> because: >>>>>>> >>>>>>> a) Admin password is required for replica promotion. This will be >>>>>>> fixed >>>>>>> with . >>>>>>> >>>>>>> Patches are on the list: >>>>>>> . >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> Pushed. >>>>>> >>>>>>> >>>>>>> >>>>>>> b) Admin password is required for connection check. This will be >>>>>>> fixed >>>>>>> with . >>>>>> >>>>>> Martin Basti pointed out that admin password should not be asked >>>>>> interactively during OTP replica promotion. Fixed. >>>>>> >>>>>> Updated and rebased patch attached. >>>>>> >>>>>> >>>>>> >>>>> >>>>> 1) >>>>> [root at vm-058-138 ~]# ipa-replica-install --server >>>>> vm-058-137.abc.idm.lab.eng.brq.redhat.com --domain >>>>> abc.idm.lab.eng.brq.redhat.com --password=bubak --setup-ca >>>>> Configuring client side components >>>>> Password for admin at ABC.IDM.LAB.ENG.BRQ.REDHAT.COM: >>>>> >>>>> IMO password should be asked first, before any installation begins >>>>> (IMO >>>>> this is for conncheck) >>>> >>>> The same thing happens without my patch. Could you file a ticket? >>> https://fedorahosted.org/freeipa/ticket/5525 >>> >>>> >>>>> >>>>> 2) >>>>> When host is not in ipaservers hostgroup. Also I would expect >>>>> different >>>>> error message >>>>> ipa-replica-install --server >>>>> vm-058-137.abc.idm.lab.eng.brq.redhat.com >>>>> --domain abc.idm.lab.eng.brq.redhat.com --password=bubak --setup-ca >>>>> --skip-conncheck >>>>> >>>>> .... >>>>> step() >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 352, in >>>>> step = lambda: next(self.__gen) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>> line 81, in run_generator_with_yield_from >>>>> six.reraise(*exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>> line 59, in run_generator_with_yield_from >>>>> value = gen.send(prev_value) >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>>> line 63, in _install >>>>> for nothing in self._installer(self.parent): >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >>>>> >>>>> >>>>> line 1507, in main >>>>> promote_check(self) >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >>>>> >>>>> >>>>> line 374, in decorated >>>>> func(installer) >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >>>>> >>>>> >>>>> line 1002, in promote_check >>>>> conn.connect(ccache=installer._ccache) >>>>> File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line >>>>> 66, >>>>> in connect >>>>> conn = self.create_connection(*args, **kw) >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", >>>>> line 199, in create_connection >>>>> principal = krb_utils.get_principal(ccache_name=ccache) >>>>> File "/usr/lib/python2.7/site-packages/ipalib/krb_utils.py", line >>>>> 184, in get_principal >>>>> raise errors.CCacheError(message=unicode(e)) >>>>> >>>>> 2015-12-07T16:23:40Z DEBUG The ipa-replica-install command failed, >>>>> exception: CCacheError: Major (851968): Unspecified GSS failure. >>>>> Minor >>>>> code may provide more information, Minor (2529639053): No Kerberos >>>>> credentials available >>>>> 2015-12-07T16:23:40Z ERROR Major (851968): Unspecified GSS failure. >>>>> Minor code may provide more information, Minor (2529639053): No >>>>> Kerberos >>>>> credentials available >>>> >>>> Fixed. >>>> >>>>> >>>>> >>>>> 3) >>>>> This case is not handle very well: >>>>> a) install client with OTP password >>>>> b) install replica with the same OTP password (when host is no in >>>>> ipaservers group, if host is in ipaservers group it works) >>>>> >>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR Major >>>>> (851968): Unspecified GSS failure. Minor code may provide more >>>>> information, Minor (2529639053): No Kerberos credentials available >>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The >>>>> ipa-replica-install command failed. See >>>>> /var/log/ipareplica-install.log >>>>> for more information >>>> >>>> This is the same as 2). >>>> >>>>> >>>>> 4) >>>>> This is not user friendly >>>>> I used wrong OTP password, can we somehow propagate the actual error >>>>> from client install to stderr? >>>>> >>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR >>>>> Configuration of >>>>> client side components failed! >>>>> ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' >>>>> '--unattended' '--domain' 'abc.idm.lab.eng.brq.redhat.com' '--server' >>>>> 'vm-058-137.abc.idm.lab.eng.brq.redhat.com' '--password' 'buba'' >>>>> returned non-zero exit status 1 >>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The >>>>> ipa-replica-install command failed. See >>>>> /var/log/ipareplica-install.log >>>>> for more information >>>> >>>> The same thing happens without my patch for any other error. Could >>>> you file a ticket? >>> >>> https://fedorahosted.org/freeipa/ticket/5527 >>> >>>> >>>> Updated patch attached. >>>> >>> Working on review >>> >> >> Is this expected that client will be installed even if there is not >> enough privileges to install replica? >> >> # ipa-replica-install --server >> vm-058-137.abc.idm.lab.eng.brq.redhat.com --domain >> abc.idm.lab.eng.brq.redhat.com --password bubak --skip-conncheck >> Configuring client side components >> ipa.ipapython.install.cli.install_tool(Replica): ERROR >> Insufficient privileges to promote the server. >> ipa.ipapython.install.cli.install_tool(Replica): ERROR The >> ipa-replica-install command failed. See /var/log/ipareplica-install.log >> for more information > > Yes. The check can't be done without the host keytab, which you get > with ipa-client-install. If ipa-client-install wasn't monolithic, the > check could be done earlier, but we are not there yet. > > The client should be probably uninstalled in case of failure, though. > ACK I will report this in a separate ticket From pspacek at redhat.com Tue Dec 8 12:26:08 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 8 Dec 2015 13:26:08 +0100 Subject: [Freeipa-devel] [PATCH 0068] add missing /ipaplatform/constants.py to .gitignore Message-ID: <5666CC60.4040303@redhat.com> Hello, add missing /ipaplatform/constants.py to .gitignore -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0068-add-missing-ipaplatform-constants.py-to-.gitignore.patch Type: text/x-patch Size: 584 bytes Desc: not available URL: From tbabej at redhat.com Tue Dec 8 12:26:50 2015 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 8 Dec 2015 13:26:50 +0100 Subject: [Freeipa-devel] [PATCH 0068] add missing /ipaplatform/constants.py to .gitignore In-Reply-To: <5666CC60.4040303@redhat.com> References: <5666CC60.4040303@redhat.com> Message-ID: <5666CC8A.3070104@redhat.com> On 12/08/2015 01:26 PM, Petr Spacek wrote: > Hello, > > add missing /ipaplatform/constants.py to .gitignore > ACK. From tbabej at redhat.com Tue Dec 8 12:29:15 2015 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 8 Dec 2015 13:29:15 +0100 Subject: [Freeipa-devel] [PATCH 0068] add missing /ipaplatform/constants.py to .gitignore In-Reply-To: <5666CC8A.3070104@redhat.com> References: <5666CC60.4040303@redhat.com> <5666CC8A.3070104@redhat.com> Message-ID: <5666CD1B.5030905@redhat.com> On 12/08/2015 01:26 PM, Tomas Babej wrote: > > > On 12/08/2015 01:26 PM, Petr Spacek wrote: >> Hello, >> >> add missing /ipaplatform/constants.py to .gitignore >> > > ACK. > Pushed to master: 848912ae31d1549d5f6bed874cc6c4541bada6f4 From mkosek at redhat.com Tue Dec 8 12:34:13 2015 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 8 Dec 2015 13:34:13 +0100 Subject: [Freeipa-devel] [PATCH 0391] replicainstall: Add check for domain if server is specified In-Reply-To: <5666868D.6010702@redhat.com> References: <5661D8A0.7030303@redhat.com> <5661E661.5090601@redhat.com> <5661E810.3060503@redhat.com> <566586F4.30201@redhat.com> <5665A939.1060807@redhat.com> <56667F4C.9070305@redhat.com> <56668560.1040508@redhat.com> <5666868D.6010702@redhat.com> Message-ID: <5666CE45.4080301@redhat.com> On 12/08/2015 08:28 AM, Jan Cholasta wrote: > On 8.12.2015 08:23, Martin Kosek wrote: >> On 12/08/2015 07:57 AM, Jan Cholasta wrote: >>> On 7.12.2015 16:43, Martin Kosek wrote: >>>> On 12/07/2015 02:17 PM, Tomas Babej wrote: >>>>> >>>>> >>>>> On 12/04/2015 08:22 PM, Rob Crittenden wrote: >>>>>> Martin Kosek wrote: >>>>>>> On 12/04/2015 07:17 PM, Tomas Babej wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Avoids failing in the later stages during the ipa-client-install >>>>>>>> command. >>>>>>>> >>>>>>>> Tomas >>>>>>> >>>>>>> Is this change needed? Wouldn't it be better to update >>>>>>> ipa-client-install or ipa-replica-install to not require the --domain >>>>>>> option? I would hope that --domain can be figured out during >>>>>>> installation and not passed to ipa-replica-install manually by the admin. >>>>>>> >>>>>>> I just think that calling >>>>>>> # ipa-replica-install --server=master.example.com >>>>>>> is better than >>>>>>> # ipa-replica-install --server=master.example.com --domain example.com >>>>>>> if possible. >>>>>> >>>>>> IIRC this is for service discovery when using a specific server and not >>>>>> LDAP. This is the domain used to search for the kerberos realm, for >>>>>> example. >>>>>> >>>>>> That isn't to say this isn't discoverable but it would require another >>>>>> function in discovery to query what the IPA domain is from the given >>>>>> master but it gets tricky if anonymous search is disabled, for example. >>>>>> >>>>>> rob >>>>>> >>>>> >>>>> Needed or not, this is the behaviour that ipa-client-install has now. >>>>> Adding a domain detection method would be a RFE for ipa-client-install >>>>> (and imho not something we should be adding at this point). >>>>> >>>>> This patch only focuses on making the ipa-replica-install work more >>>>> smoothly. >>>> >>>> I am just thinking that client promotion (ipa-replica-install) and >>>> ipa-client-install are a bit different use cases. While ipa-client-install >>>> should be typically run in auto-discovery and you thus do not use --server >>>> option much, while with ipa-replica-install, you want to make sure you have >>>> the >>>> expected topology and should use --server all the time without gambling on it. >>>> >>>> But I do not think it has to be there since 4.3 GA, can you please file a >>>> ticket for this gap? >>> >>> I would rather do it now, because the change from optional to mandatory is >>> backward incompatible. (We don't want to break users' scripts, right?) >> >> I think it is the other way around - with the change I was suggesting >> (autodetecting --domain option instead of always requesting it, as in Tomas' >> patch which we can merge if my proposal is not doable for 4.3 GA). >> > > "with ipa-replica-install, you want to make sure you have the expected topology > and should use --server all the time" sounds like you want to make --server > mandatory for ipa-replica-install, which should be done either before 4.3 GA or > never. Ah, no, this is not what I meant. I was only discussing the --domain option in this mail the the typical use cases for --server option in ipa-client-install and ipa-replica-install. If we can trust ipa-replica-install to do a good job in picking a server to replica from, the --server can stay optional. Although I am on fence here, being more explicit when creating topology may be helpful. CCing Simo, in case he has some opinions on this. From ssorce at redhat.com Tue Dec 8 13:00:20 2015 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 08 Dec 2015 08:00:20 -0500 Subject: [Freeipa-devel] [PATCH 0391] replicainstall: Add check for domain if server is specified In-Reply-To: <5666CE45.4080301@redhat.com> References: <5661D8A0.7030303@redhat.com> <5661E661.5090601@redhat.com> <5661E810.3060503@redhat.com> <566586F4.30201@redhat.com> <5665A939.1060807@redhat.com> <56667F4C.9070305@redhat.com> <56668560.1040508@redhat.com> <5666868D.6010702@redhat.com> <5666CE45.4080301@redhat.com> Message-ID: <1449579620.17418.28.camel@redhat.com> On Tue, 2015-12-08 at 13:34 +0100, Martin Kosek wrote: > On 12/08/2015 08:28 AM, Jan Cholasta wrote: > > On 8.12.2015 08:23, Martin Kosek wrote: > >> On 12/08/2015 07:57 AM, Jan Cholasta wrote: > >>> On 7.12.2015 16:43, Martin Kosek wrote: > >>>> On 12/07/2015 02:17 PM, Tomas Babej wrote: > >>>>> > >>>>> > >>>>> On 12/04/2015 08:22 PM, Rob Crittenden wrote: > >>>>>> Martin Kosek wrote: > >>>>>>> On 12/04/2015 07:17 PM, Tomas Babej wrote: > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> Avoids failing in the later stages during the ipa-client-install > >>>>>>>> command. > >>>>>>>> > >>>>>>>> Tomas > >>>>>>> > >>>>>>> Is this change needed? Wouldn't it be better to update > >>>>>>> ipa-client-install or ipa-replica-install to not require the --domain > >>>>>>> option? I would hope that --domain can be figured out during > >>>>>>> installation and not passed to ipa-replica-install manually by the admin. > >>>>>>> > >>>>>>> I just think that calling > >>>>>>> # ipa-replica-install --server=master.example.com > >>>>>>> is better than > >>>>>>> # ipa-replica-install --server=master.example.com --domain example.com > >>>>>>> if possible. > >>>>>> > >>>>>> IIRC this is for service discovery when using a specific server and not > >>>>>> LDAP. This is the domain used to search for the kerberos realm, for > >>>>>> example. > >>>>>> > >>>>>> That isn't to say this isn't discoverable but it would require another > >>>>>> function in discovery to query what the IPA domain is from the given > >>>>>> master but it gets tricky if anonymous search is disabled, for example. > >>>>>> > >>>>>> rob > >>>>>> > >>>>> > >>>>> Needed or not, this is the behaviour that ipa-client-install has now. > >>>>> Adding a domain detection method would be a RFE for ipa-client-install > >>>>> (and imho not something we should be adding at this point). > >>>>> > >>>>> This patch only focuses on making the ipa-replica-install work more > >>>>> smoothly. > >>>> > >>>> I am just thinking that client promotion (ipa-replica-install) and > >>>> ipa-client-install are a bit different use cases. While ipa-client-install > >>>> should be typically run in auto-discovery and you thus do not use --server > >>>> option much, while with ipa-replica-install, you want to make sure you have > >>>> the > >>>> expected topology and should use --server all the time without gambling on it. > >>>> > >>>> But I do not think it has to be there since 4.3 GA, can you please file a > >>>> ticket for this gap? > >>> > >>> I would rather do it now, because the change from optional to mandatory is > >>> backward incompatible. (We don't want to break users' scripts, right?) > >> > >> I think it is the other way around - with the change I was suggesting > >> (autodetecting --domain option instead of always requesting it, as in Tomas' > >> patch which we can merge if my proposal is not doable for 4.3 GA). > >> > > > > "with ipa-replica-install, you want to make sure you have the expected topology > > and should use --server all the time" sounds like you want to make --server > > mandatory for ipa-replica-install, which should be done either before 4.3 GA or > > never. > > Ah, no, this is not what I meant. I was only discussing the --domain option in > this mail the the typical use cases for --server option in ipa-client-install > and ipa-replica-install. > > If we can trust ipa-replica-install to do a good job in picking a server to > replica from, the --server can stay optional. Although I am on fence here, > being more explicit when creating topology may be helpful. CCing Simo, in case > he has some opinions on this. Leave it optional, our first order of business is making things simple, then adding optional knobs to let the admin with knowledge to tweak things. Simo. From rcritten at redhat.com Tue Dec 8 13:25:33 2015 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 8 Dec 2015 08:25:33 -0500 Subject: [Freeipa-devel] patch acceptance criteria In-Reply-To: <5666B1F3.4070903@redhat.com> References: <565F34FD.5090204@redhat.com> <20151203090954.GB5209@mail.corp.redhat.com> <566058E2.6040201@redhat.com> <20151204111921.GE28768@mail.corp.redhat.com> <56619841.3010809@redhat.com> <5666B1F3.4070903@redhat.com> Message-ID: <5666DA4D.8090701@redhat.com> Petr Spacek wrote: > On 4.12.2015 14:42, Rob Crittenden wrote: >> Lukas Slebodnik wrote: >>>> On (03/12/15 09:59), Rob Crittenden wrote: >>>>>> Lukas Slebodnik wrote: >>>>>>>> On (02/12/15 13:14), Rob Crittenden wrote: >>>>>>>>>> Is it still mandatory that tests pass the unit tests before acceptance? >>>>>>>> Unit test could be executed as part of "%check" phase in spec files. >>>>>>>> I recently added C-base unit tests there. >>>>>>>> >>>>>>>> I was not bale to run "make tests" there because many tests failed. >>>>>>>> If they require real IPA server for execution ( or lite server) >>>>>>>> then they are not unit test but integration tests. >>>>>>>> One solution would be to skip them or to usw cwrap[1] + lite server. >>>>>>>> So it can be run also in mock/koji which has many restrictions. >>>>>> >>>>>> It would represent quite a lot of work but it may be a good idea to >>>>>> investigate. Ipsilon uses cwrap for its tests so some of the >>>>>> configuration can be gleaned from that. >>>>>> >>>>>> I would definitely be opposed to this as part of the freeipa.spec in the >>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>>> What do you mean by this part? >>>> >>>> Did you mean "running make tests" in spec file? >>>> If yes, could you elaborate why it's not good idea? >>>> many projects run tests in "%check" >>>> sssd, certmonger, glibc ... >>>> >>>> Currently only C-based test are executed. And I added it only recently. >> Because it would be overkill during development. The expectation is that >> developers and reviewers run the tests before submission/acceptance. If >> they fail to do that then it will be obvious. >> >>>>>> git tree. In Fedora it might help find problems when rawhide becomes >>>>>> Fedora.next so it would provide some value there. >>>>>> >>>>>>>> >>>>>>>> Also lint should be also part of "%check" phase and not part of >>>>>>>> ordinary build. >>>>>>>> BTW I could not see a lint[2] in fedora build at all. So I'm not sure >>>>>>>> if it is executed with upstream spec file. >>>>>> >>>>>> It isn't there because the expectation is that lint already passes as >>>>>> part of the release process. I don't see the value on running lint on >>>>>> release tarballs. >>>>>> >>>> That's just an expectation. It needn't be true. Your initial mail was about >>>> stricter review process. And automating things is best way how to >>>> enforce it. So reviewer would just build rpms and if "%check" phase >>>> will not pass then he will not continue with review. >>>> If it will be part of "%check" then you can use mock and easily ensure >>>> that test passes on stable fedora and fedora rawhide (and maybe centOS) >> By the time downstream gets a tarball it is too late to fix lint errors. >> If upstream is doing a release with lint errors then there is something >> deeply wrong with the release process. If someone wants to add it to the >> downstream spec files I'm not going to complain, I just find it >> extremely unlikely that it will provide any value, ever. > > Sorry Rob, but I disagree. Lint already caught couple cases where Requires: > were not properly updated so IPA code was referencing non-existing code in > Dogtag/Custodia packages and so on. > > So clearly there is some value in it. > I'm referring to the downstream spec files (e.g. Fedora) which don't run the lint target during the build. rob From tbabej at redhat.com Tue Dec 8 13:28:06 2015 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 8 Dec 2015 14:28:06 +0100 Subject: [Freeipa-devel] [PATCH 0394] topology: Make sure the old 'realm' topology suffix is not Message-ID: <5666DAE6.3030708@redhat.com> Hi, The old 'realm' topology suffix is no longer used, however, it was being created on masters with version 4.2.3 and later. Make sure it's properly removed. Note that this is not the case for the 'ipaca' suffix, which was later removed to 'ca'. https://fedorahosted.org/freeipa/ticket/5526 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0394-topology-Make-sure-the-old-realm-topology-suffix-is-.patch Type: text/x-patch Size: 1304 bytes Desc: not available URL: From ofayans at redhat.com Tue Dec 8 13:33:12 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Tue, 8 Dec 2015 14:33:12 +0100 Subject: [Freeipa-devel] [PATCH] First part of the replica promotion tests + testplan In-Reply-To: <565F17EE.3020701@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> <565DB7D3.5040500@redhat.com> <565F0BDD.8080206@redhat.com> <565F17EE.3020701@redhat.com> Message-ID: <5666DC18.2040906@redhat.com> Hi all, The patches are rebased against the current master. On 12/02/2015 05:10 PM, Martin Basti wrote: > > > On 02.12.2015 16:18, Oleg Fayans wrote: >> Hi Martin, >> >> On 12/01/2015 04:08 PM, Martin Basti wrote: >>> >>> >>> On 27.11.2015 16:26, Oleg Fayans wrote: >>>> And patch N 16 passes lint too: >>>> >>>> On 11/27/2015 04:03 PM, Oleg Fayans wrote: >>>>> Hi, >>>>> >>>>> On 11/27/2015 03:26 PM, Martin Basti wrote: >>>>>> >>>>>> >>>>>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>>>>> Hi Martin, >>>>>>> >>>>>>> All your suggestions were taken into account. Both patches are >>>>>>> updated. Thank you for your help! >>>>>>> >>>>>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>>>>> Hi Martin, >>>>>>>>> >>>>>>>>> I agree to all your points but one. please, see my comment below >>>>>>>>> >>>>>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> 0) Note >>>>>>>>>> Please be aware of https://fedorahosted.org/freeipa/ticket/5469 >>>>>>>>>> during >>>>>>>>>> KRA testing >>>>>>>>>> >>>>>>>>>> 1) >>>>>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, this may >>>>>>>>>> change >>>>>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for domain >>>>>>>>>> level 0 >>>>>>>>>> and 1 >>>>>>>>>> >>>>>>>>>> 2) >>>>>>>>>> Why uninstall KRA then server, is not enough just uninstall >>>>>>>>>> server >>>>>>>>>> which >>>>>>>>>> covers KRA uninstall? >>>>>>>>>> >>>>>>>>>> + def teardown_method(self, method): >>>>>>>>>> + for host in self.replicas: >>>>>>>>>> + host.run_command(self.kra_uninstall, >>>>>>>>>> raiseonerr=False) >>>>>>>>>> + tasks.uninstall_master(host) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 3) >>>>>>>>>> Can be this function more generic? It should allow specify host >>>>>>>>>> where >>>>>>>>>> KRA should be installed not just master >>>>>>>>>> >>>>>>>>>> + def test_kra_install_master(self): >>>>>>>>>> + self.master.run_command(self.kra_install) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 4) >>>>>>>>>> >>>>>>>>>> TestLevel0(Dummy): >>>>>>>>>> Can be the test name more specific, something like >>>>>>>>>> TestReplicaPromotionLevel0 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 5) >>>>>>>>>> please remove this, the patch is on review and it will be pushed >>>>>>>>>> sooner >>>>>>>>>> than tests >>>>>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>>>>> >>>>>>>>>> and as I mentioned in ticket #5455, I cannot reproduce it with >>>>>>>>>> ipa-kra-install, so please provide steps to reproduce if you >>>>>>>>>> insist >>>>>>>>>> that >>>>>>>>>> this still does not work as expected with KRA. >>>>>>>>>> >>>>>>>>>> 6) This is completely wrong, it removes everything that we >>>>>>>>>> tried to >>>>>>>>>> achieve with previous patches with domain level in CI >>>>>>>>> >>>>>>>>> Actually, being able to configure domain level per class is WAY >>>>>>>>> more >>>>>>>>> convenient, than to always have to think which domain level is >>>>>>>>> appropriate for which particular test during jenkins job >>>>>>>>> configuration. In fact, I should have thought about it from the >>>>>>>>> very >>>>>>>>> beginning. For example, in test_replica_promotion.py we have on >>>>>>>>> class, >>>>>>>>> which intiates with domain level = 1, while others - with domain >>>>>>>>> level >>>>>>>>> 0. With config-based approach, we would have to implement a >>>>>>>>> separate >>>>>>>>> step that raises domain level. Overall, I am against the approach, >>>>>>>>> when you have to remember to set certain domain level in config >>>>>>>>> for >>>>>>>>> any particular test. The tests themselves should be aware of the >>>>>>>>> domain level they need. >>>>>>>> I do not say that we should not have something that overrides >>>>>>>> settings >>>>>>>> in from config in a particular test case, I say your patch is >>>>>>>> doing it >>>>>>>> wrong. >>>>>>>> >>>>>>>> I agree it is useful to have param domain_level in install_master, >>>>>>>> and >>>>>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by default, >>>>>>>> because with your current patch the domain_level in config is not >>>>>>>> used >>>>>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>>>>> >>>>>>>> For example I want to achieve this goal: >>>>>>>> test_vault.py, this test suite can run on domain level1 and on >>>>>>>> domain >>>>>>>> level0, so with one test we can test 2 domain levels just with >>>>>>>> putting >>>>>>>> domain level into config file. >>>>>>>> >>>>>>>> I agree that with extraordinary test like replica promotion test >>>>>>>> is, we >>>>>>>> need something that allows override the config file. >>>>>>>> >>>>>>>> As I said bellow, domain_level default value should be None in >>>>>>>> install_master and install_topo plugin. If domain level was >>>>>>>> specified >>>>>>>> use the specified one, if not (value is None) use the domain level >>>>>>>> from >>>>>>>> config file. >>>>>>> >>>>>>> Agreed :) >>>>>>> >>>>>>>> >>>>>>>> Martin >>>>>>>>> >>>>>>>>>> >>>>>>>>>> [PATCH] Enabled setting domain_level per class derived from >>>>>>>>>> TestIntegration >>>>>>>>>> >>>>>>>>>> When I configure domain level 0 in yaml config, how is this >>>>>>>>>> supposed to >>>>>>>>>> get into install methods when you removed that code? >>>>>>>>>> >>>>>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>>>>> + "--domain-level=%i" % domain_level >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> You always use MAX_DOMAIN_LEVEL in this case or whatever is >>>>>>>>>> specified in >>>>>>>>>> domain_level option. >>>>>>>>>> I suggest to use domain_level=None, and when it is None use >>>>>>>>>> 'host.config.domain_level', if it is not None, use 'domain_level' >>>>>>>>>> >>>>>>>>>> With this we can specify domain level in config file for test >>>>>>>>>> that >>>>>>>>>> can >>>>>>>>>> be used for both domain levels and you can manually specify >>>>>>>>>> domain >>>>>>>>>> level >>>>>>>>>> for test that requires specific domain level. >>>>>>>>>> >>>>>>>>>> Also this should go away >>>>>>>>>> >>>>>>>>>> @classmethod >>>>>>>>>> def install(cls, mh): >>>>>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>>>>> if cls.topology is None: >>>>>>>>>> return >>>>>>>>>> >>>>>>>>>> I do not see reason why test should override configuration in >>>>>>>>>> config in >>>>>>>>>> this case. >>>>>>>>>> >>>>>>>>>> Martin >>>>>>>>>> >>>>>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Here is the updated version of the patch (more tests + fixed the >>>>>>>>>>> issues of the first one) + patch 0017, that implements the >>>>>>>>>>> necessary >>>>>>>>>>> changes in the background code, i. e. patch 16 does not work >>>>>>>>>>> without >>>>>>>>>>> patch 17 >>>>>>>>>>> >>>>>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>>>>> Hi guys, >>>>>>>>>>>>> >>>>>>>>>>>>> Here are first two automated testcases from this (so far >>>>>>>>>>>>> incomplete) >>>>>>>>>>>>> testplan: >>>>>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>>>>> >>>>>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> PATCH 16: NACK >>>>>>>>>>>> >>>>>>>>>>>> 1) >>>>>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>>>>> 'domain_level' to >>>>>>>>>>>> install_topo()? >>>>>>>>>>>> Also it is good practise to add new option as the last >>>>>>>>>>>> parameter. >>>>>>>>>>>> >>>>>>>>>>>> 2) >>>>>>>>>>>> cab you in both tests specify a domain level with constant >>>>>>>>>>>> instead of >>>>>>>>>>>> number literal? >>>>>>>>>>>> >>>>>>>>>>>> 3) >>>>>>>>>>>> both test call install_topo with custom domain level, but it >>>>>>>>>>>> cannot >>>>>>>>>>>> work >>>>>>>>>>>> because 1) (did you run the test?) >>>>>>>>>>>> >>>>>>>>>>>> 4) >>>>>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>>>>> Respectively why there is call of install_topo() that installs >>>>>>>>>>>> replica. >>>>>>>>>>>> As this test just tests that ipa-replica-prepare is not working >>>>>>>>>>>> anymore, >>>>>>>>>>>> is it worth to spend 20 minutes with installing replica and >>>>>>>>>>>> then >>>>>>>>>>>> just no >>>>>>>>>>>> tot use it? IMO to install master in install step is enough. >>>>>>>>>>>> >>>>>>>>>>>> Martin^2 >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> ./make-lint >>>>>> ************* Module ipatests.test_integration.base >>>>>> ipatests/test_integration/base.py:66: [E1101(no-member), >>>>>> IntegrationTest.install] Class 'IntegrationTest' has no >>>>>> 'domain_level' >>>>>> member) >>>>>> ************* Module ipatests.test_integration.test_replica_promotion >>>>>> ipatests/test_integration/test_replica_promotion.py:16: >>>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no 'domain_level' >>>>>> member) >>>>>> ipatests/test_integration/test_replica_promotion.py:117: >>>>>> [E1101(no-member), >>>>>> TestCAInstall.test_ca_install_without_replica_file] >>>>>> Module 'ipatests.test_integration.tasks' has no 'setup_replica' >>>>>> member) >>>>>> >>>>>> >>>>>> Is it so hard to run pylint before patch is posted for review? >>>>> >>>>> Sorry, my bad. Fixed. >>>>> >>>>> >>>>> >>>> >>> >>> 1) >>> Why is this change in the patch? >>> - # Clean up the test directory >>> - host.run_command(['rm', '-rvf', host.config.test_dir]) >> >> Otherwise 2 out of 8 tests fail with IOError at line 78 of >> ipatests/test_integration/tasks.py >> >> I do not understand yet how does this happen, but if you remove >> ipatests folder once, it then fails to be created again. >> > So this should be in separated patch and investigated properly. Agree. Removed > >>> - >>> >>> >>> 2) >>> is enough to have this check only in install_master, install_topo can >>> pass None to install_master >>> + if domain_level is None: >>> + domain_level = master.config.domain_level >> >> Done >> >>> >>> >>> 3) >>> IMO replica-manage del should cleanup hosts entry, so following code >>> should not be needed. >>> + if cleanhost: >>> + kinit_admin(master) >>> + master.run_command(["ipa", "host-del", "--updatedns", >>> replica.hostname], >>> + raiseonerr=False) >> >> Well, in fact it does not. At least the corresponding dns record stays >> and causes the subsequent ipa-client-install to fail. Probably it's a >> bug. On the other hand, if I promote an existing client to replica and >> then delete this replica, then, I probably want the host record (that >> was created during client-install) to stay in the system. So, does not >> look like a bug to me. > No you don't, because replica uninstallation also removes the client. > > I tried it with ipa42, ipa-replica-manage del removes host entry, and > DNS A records, only SSHFP records stay there (I'm not sure if it is bug > or feature) > > Also I received this message > """ > Failed to cleanup replica1.ipa.test DNS entries: no matching entry found > You may need to manually remove them from the tree > """ > But, A record has been removed, so this is probably false positive and > it needs to have a ticket. Agree, that was an issue with my setup. Removed >> >>> >>> 4) >>> This variable is not used >>> + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] >> >> Removed >> >>> >>> 5) >>> Why do you need this >>> + kra_install = ["ipa-kra-install", "-U", "-p", >>> config.dirman_password] >>> when you implemented tasks.install_kra that returns the exactly the same >>> result? >> >> Right. Removed >> >>> >>> 6) >>> PEP8 >>> ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 blank lines, >>> found 1 >>> ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 blank lines, >>> found 1 >>> ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 blank lines, >>> found 1 >>> ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 blank lines, >>> found 1 >>> ./ipatests/test_integration/tasks.py:950:80: E501 line too long (80 > 79 >>> characters) >>> >>> ./ipatests/test_integration/test_replica_promotion.py:29:80: E501 line >>> too long (85 > 79 characters) >>> ./ipatests/test_integration/test_replica_promotion.py:64:80: E501 line >>> too long (85 > 79 characters) >>> ./ipatests/test_integration/test_replica_promotion.py:67:80: E501 line >>> too long (88 > 79 characters) >>> ./ipatests/test_integration/test_replica_promotion.py:93:80: E501 line >>> too long (80 > 79 characters) >>> ./ipatests/test_integration/test_replica_promotion.py:94:80: E501 line >>> too long (83 > 79 characters) >>> ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 line >>> too long (81 > 79 characters) >>> ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 line >>> too long (80 > 79 characters) >>> ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 line >>> too long (82 > 79 characters) >>> ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 line >>> too long (80 > 79 characters) >> >> Most of these complaints are unrelated to the current patches. >> It's better to create a separate patch addressing PEP8 errors. >> > I beg for your pardon, those PEP8 errors have been introduced by your > patches. Fixed >>> >>> 7) >>> Why this must be stored in instance? IMO to have it stored as local >>> variable is perfect >>> TestKRAInstall, TestCAInstall >>> self.replica1_filename = tasks.get_replica_filename(replica1) >>> self.replica2_filename = tasks.get_replica_filename(replica2) >> >> Agree. Fixed >> > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0016.5-First-part-of-replica-promotion-tests.patch Type: text/x-patch Size: 10106 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0017.4-Enabled-setting-domain_level-per-class.patch Type: text/x-patch Size: 6444 bytes Desc: not available URL: From tbabej at redhat.com Tue Dec 8 13:47:17 2015 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 8 Dec 2015 14:47:17 +0100 Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <566060E2.8030903@redhat.com> References: <565C3C6F.4040901@redhat.com> <20151130163256.GA7759@mail.corp.redhat.com> <565DD881.60401@redhat.com> <565EAAAE.8020306@redhat.com> <565F0332.9000804@redhat.com> <20151202162519.GA19962@mail.corp.redhat.com> <56601AE8.2020100@redhat.com> <5660528E.40904@redhat.com> <149447113.23167965.1449156368210.JavaMail.zimbra@redhat.com> <566060E2.8030903@redhat.com> Message-ID: <5666DF65.1030309@redhat.com> On 12/03/2015 04:33 PM, Tomas Babej wrote: > > > On 12/03/2015 04:26 PM, Ale? Mare?ek wrote: >> Hello, >> >> ACK for code >> NACK for the placing "get_client_ip_with_hostmask" function to test_sudo.py (this function should be in some more general file) >> > > What place would you propose? The task.py is not a good place, as this > is not really a task. > > Nevertheless, I'd rather have it moved when an use case outside > test_sudo.py actually arises. Right now it would lead to unnecessary > cluttering. > > Tomas > I re-discovered ipatests.test_integration.util (two years after I created it :D) - which seemed ideal for this function. Updated patch attached. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0388-6-tests-Add-hostmask-detection-for-sudo-rules-validati.patch Type: text/x-patch Size: 4486 bytes Desc: not available URL: From dkupka at redhat.com Tue Dec 8 14:03:10 2015 From: dkupka at redhat.com (David Kupka) Date: Tue, 8 Dec 2015 15:03:10 +0100 Subject: [Freeipa-devel] [PATCH 0071] replica: Fix ipa-replica-install with replica file (domain, level 0). Message-ID: <5666E31E.1030705@redhat.com> https://fedorahosted.org/freeipa/ticket/5531 -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0071.0-replica-Fix-ipa-replica-install-with-replica-file-dom.patch Type: text/x-patch Size: 2344 bytes Desc: not available URL: From pspacek at redhat.com Tue Dec 8 14:24:34 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 8 Dec 2015 15:24:34 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <5666BCBC.9040005@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> <5665843C.2090507@redhat.com> <56658C89.8050103@redhat.com> <56668D30.8060308@redhat.com> <5666BCBC.9040005@redhat.com> Message-ID: <5666E822.90103@redhat.com> On 8.12.2015 12:19, David Kupka wrote: > On 08/12/15 08:56, Petr Spacek wrote: >> On 7.12.2015 14:41, David Kupka wrote: >>> +def is_host_resolvable(fqdn): >>> + if not isinstance(fqdn, DNSName): >>> + fqdn = DNSName(fqdn) >>> + for rdtype in (rdatatype.A, rdatatype.AAAA): >>> + try: >>> + resolver.query(fqdn.make_absolute(), rdtype) >>> + except DNSException: >>> + continue >>> + else: >>> + return True >>> + >>> + return False >>> >> >> NACK, you are re-introducing duplicate function which was removed in >> 498471e4aed1367b72cd74d15811d0584a6ee268. >> >> Please amend the patch ASAP to use new verify_host_resolvable() function so I >> can test it and get it into 4.3. >> > Updated patches attached. Hmm, my review script screams: Detected base branch: origin/master Checks will be made against following base commit: 848912a add missing /ipaplatform/constants.py to .gitignore Writing API to API.txt ./ipalib/plugins/dns.py:2127:9: E124 closing bracket does not match visual indentation ./ipalib/plugins/dns.py:2711:13: E128 continuation line under-indented for visual indent ./ipalib/plugins/dns.py:2713:9: E124 closing bracket does not match visual indentation ./ipalib/plugins/dns.py:2716:13: E128 continuation line under-indented for visual indent ./ipapython/ipautil.py:928:1: E302 expected 2 blank lines, found 1 ./ipapython/ipautil.py:948:17: E128 continuation line under-indented for visual indent ./ipapython/ipautil.py:956:1: E302 expected 2 blank lines, found 1 ./ipapython/ipautil.py:997:80: E501 line too long (83 > 79 characters) ./ipapython/ipautil.py:998:80: E501 line too long (83 > 79 characters) ./ipaserver/install/bindinstance.py:49:9: E128 continuation line under-indented for visual indent ./ipaserver/install/bindinstance.py:292:80: E501 line too long (80 > 79 characters) ./ipaserver/install/bindinstance.py:438:19: E128 continuation line under-indented for visual indent ./ipaserver/install/server/common.py:271:63: E261 at least two spaces before inline comment ./ipaserver/install/server/common.py:271:80: E501 line too long (90 > 79 characters) ERROR: PEP8 --diff failed, continuing ... ERROR: API.txt was changed without a change in VERSION, continuing ... Summary of detected errors: ERROR: PEP8 --diff failed ERROR: API.txt was changed without a change in VERSION Please fix it :-) Make make this more pleasant for you, you can use (and review) my attached patch. It adds 'review' target to Makefile, it will do the same checks as I do. -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: review.log Type: text/x-log Size: 2077 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0069-Add-review-target-for-make.-It-automates-following-t.patch Type: text/x-patch Size: 3163 bytes Desc: not available URL: From mbasti at redhat.com Tue Dec 8 15:09:44 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 8 Dec 2015 16:09:44 +0100 Subject: [Freeipa-devel] [PATCH 0065] ipa-replica-install prints incorrect error message when replica is already installed In-Reply-To: References: Message-ID: <5666F2B8.1090803@redhat.com> On 01.12.2015 14:57, Gabe Alford wrote: > Sorry guys, I forgot to add a meaningful subject to this message. > Ignore the previous thread start. > > ---------- Forwarded message ---------- > From: *Gabe Alford* > > Date: Mon, Nov 30, 2015 at 7:31 PM > Subject: [PATCH 0065] > To: freeipa-devel > > > > Hello, > > Patch fix for the following tickets: > > https://fedorahosted.org/freeipa/ticket/5022 > https://fedorahosted.org/freeipa/ticket/5320 > > Thanks, > > Gabe > > > ACK -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcholast at redhat.com Tue Dec 8 15:14:03 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 8 Dec 2015 16:14:03 +0100 Subject: [Freeipa-devel] [PATCH 0065] ipa-replica-install prints incorrect error message when replica is already installed In-Reply-To: <5666F2B8.1090803@redhat.com> References: <5666F2B8.1090803@redhat.com> Message-ID: <5666F3BB.8030006@redhat.com> On 8.12.2015 16:09, Martin Basti wrote: > > > On 01.12.2015 14:57, Gabe Alford wrote: >> Sorry guys, I forgot to add a meaningful subject to this message. >> Ignore the previous thread start. >> >> ---------- Forwarded message ---------- >> From: *Gabe Alford* > >> Date: Mon, Nov 30, 2015 at 7:31 PM >> Subject: [PATCH 0065] >> To: freeipa-devel > > >> >> >> Hello, >> >> Patch fix for the following tickets: >> >> https://fedorahosted.org/freeipa/ticket/5022 >> https://fedorahosted.org/freeipa/ticket/5320 >> >> Thanks, >> >> Gabe >> >> >> > ACK NACK, you can't install a server over an already installed client, thus the original check is correct. -- Jan Cholasta From mbasti at redhat.com Tue Dec 8 15:17:43 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 8 Dec 2015 16:17:43 +0100 Subject: [Freeipa-devel] [PATCH 0065] ipa-replica-install prints incorrect error message when replica is already installed In-Reply-To: <5666F3BB.8030006@redhat.com> References: <5666F2B8.1090803@redhat.com> <5666F3BB.8030006@redhat.com> Message-ID: <5666F497.3040500@redhat.com> On 08.12.2015 16:14, Jan Cholasta wrote: > On 8.12.2015 16:09, Martin Basti wrote: >> >> >> On 01.12.2015 14:57, Gabe Alford wrote: >>> Sorry guys, I forgot to add a meaningful subject to this message. >>> Ignore the previous thread start. >>> >>> ---------- Forwarded message ---------- >>> From: *Gabe Alford* >> > >>> Date: Mon, Nov 30, 2015 at 7:31 PM >>> Subject: [PATCH 0065] >>> To: freeipa-devel >> > >>> >>> >>> Hello, >>> >>> Patch fix for the following tickets: >>> >>> https://fedorahosted.org/freeipa/ticket/5022 >>> https://fedorahosted.org/freeipa/ticket/5320 >>> >>> Thanks, >>> >>> Gabe >>> >>> >>> >> ACK > > NACK, you can't install a server over an already installed client, > thus the original check is correct. > Ahh domain level 0, right, but this check can be added before the client check. With domain level 1, this check should stay there IMO. Martin From jcholast at redhat.com Tue Dec 8 15:20:25 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 8 Dec 2015 16:20:25 +0100 Subject: [Freeipa-devel] [PATCH 0065] ipa-replica-install prints incorrect error message when replica is already installed In-Reply-To: <5666F497.3040500@redhat.com> References: <5666F2B8.1090803@redhat.com> <5666F3BB.8030006@redhat.com> <5666F497.3040500@redhat.com> Message-ID: <5666F539.1000904@redhat.com> On 8.12.2015 16:17, Martin Basti wrote: > > > On 08.12.2015 16:14, Jan Cholasta wrote: >> On 8.12.2015 16:09, Martin Basti wrote: >>> >>> >>> On 01.12.2015 14:57, Gabe Alford wrote: >>>> Sorry guys, I forgot to add a meaningful subject to this message. >>>> Ignore the previous thread start. >>>> >>>> ---------- Forwarded message ---------- >>>> From: *Gabe Alford* >>> > >>>> Date: Mon, Nov 30, 2015 at 7:31 PM >>>> Subject: [PATCH 0065] >>>> To: freeipa-devel >>> > >>>> >>>> >>>> Hello, >>>> >>>> Patch fix for the following tickets: >>>> >>>> https://fedorahosted.org/freeipa/ticket/5022 >>>> https://fedorahosted.org/freeipa/ticket/5320 >>>> >>>> Thanks, >>>> >>>> Gabe >>>> >>>> >>>> >>> ACK >> >> NACK, you can't install a server over an already installed client, >> thus the original check is correct. >> > Ahh domain level 0, right, but this check can be added before the client > check. Yes. > With domain level 1, this check should stay there IMO. Yes. It should say "IPA server is already configured" rather than "IPA replica is already configured", though. -- Jan Cholasta From ofayans at redhat.com Tue Dec 8 15:20:44 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Tue, 8 Dec 2015 16:20:44 +0100 Subject: [Freeipa-devel] [PATCH 0071] replica: Fix ipa-replica-install with replica file (domain, level 0). In-Reply-To: <5666E31E.1030705@redhat.com> References: <5666E31E.1030705@redhat.com> Message-ID: <5666F54C.80801@redhat.com> ACK. The initial issue is fixed. On 12/08/2015 03:03 PM, David Kupka wrote: > https://fedorahosted.org/freeipa/ticket/5531 > > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From redhatrises at gmail.com Tue Dec 8 15:26:20 2015 From: redhatrises at gmail.com (Gabe Alford) Date: Tue, 8 Dec 2015 08:26:20 -0700 Subject: [Freeipa-devel] [PATCH 0065] ipa-replica-install prints incorrect error message when replica is already installed In-Reply-To: <5666F539.1000904@redhat.com> References: <5666F2B8.1090803@redhat.com> <5666F3BB.8030006@redhat.com> <5666F497.3040500@redhat.com> <5666F539.1000904@redhat.com> Message-ID: Just to confirm: if server is installed: Let's stop here and not do anything else if domain level 0: check if client installed and stop here Right? On Tue, Dec 8, 2015 at 8:20 AM, Jan Cholasta wrote: > On 8.12.2015 16:17, Martin Basti wrote: > >> >> >> On 08.12.2015 16:14, Jan Cholasta wrote: >> >>> On 8.12.2015 16:09, Martin Basti wrote: >>> >>>> >>>> >>>> On 01.12.2015 14:57, Gabe Alford wrote: >>>> >>>>> Sorry guys, I forgot to add a meaningful subject to this message. >>>>> Ignore the previous thread start. >>>>> >>>>> ---------- Forwarded message ---------- >>>>> From: *Gabe Alford* >>>> > >>>>> Date: Mon, Nov 30, 2015 at 7:31 PM >>>>> Subject: [PATCH 0065] >>>>> To: freeipa-devel >>>> > >>>>> >>>>> >>>>> Hello, >>>>> >>>>> Patch fix for the following tickets: >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/5022 >>>>> https://fedorahosted.org/freeipa/ticket/5320 >>>>> >>>>> Thanks, >>>>> >>>>> Gabe >>>>> >>>>> >>>>> >>>>> ACK >>>> >>> >>> NACK, you can't install a server over an already installed client, >>> thus the original check is correct. >>> >>> Ahh domain level 0, right, but this check can be added before the client >> check. >> > > Yes. > > With domain level 1, this check should stay there IMO. >> > > Yes. It should say "IPA server is already configured" rather than "IPA > replica is already configured", though. > > -- > Jan Cholasta > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbasti at redhat.com Tue Dec 8 15:27:44 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 8 Dec 2015 16:27:44 +0100 Subject: [Freeipa-devel] [PATCH 0065] ipa-replica-install prints incorrect error message when replica is already installed In-Reply-To: References: <5666F2B8.1090803@redhat.com> <5666F3BB.8030006@redhat.com> <5666F497.3040500@redhat.com> <5666F539.1000904@redhat.com> Message-ID: <5666F6F0.2050005@redhat.com> On 08.12.2015 16:26, Gabe Alford wrote: > Just to confirm: > > if server is installed: > Let's stop here and not do anything else > > if domain level 0: > check if client installed and stop here > > Right? yes > > > On Tue, Dec 8, 2015 at 8:20 AM, Jan Cholasta > wrote: > > On 8.12.2015 16:17, Martin Basti wrote: > > > > On 08.12.2015 16:14, Jan Cholasta wrote: > > On 8.12.2015 16:09, Martin Basti wrote: > > > > On 01.12.2015 14:57, Gabe Alford wrote: > > Sorry guys, I forgot to add a meaningful subject > to this message. > Ignore the previous thread start. > > ---------- Forwarded message ---------- > From: *Gabe Alford* > >> > Date: Mon, Nov 30, 2015 at 7:31 PM > Subject: [PATCH 0065] > To: freeipa-devel > >> > > > Hello, > > Patch fix for the following tickets: > > https://fedorahosted.org/freeipa/ticket/5022 > https://fedorahosted.org/freeipa/ticket/5320 > > Thanks, > > Gabe > > > > ACK > > > NACK, you can't install a server over an already installed > client, > thus the original check is correct. > > Ahh domain level 0, right, but this check can be added before > the client > check. > > > Yes. > > With domain level 1, this check should stay there IMO. > > > Yes. It should say "IPA server is already configured" rather than > "IPA replica is already configured", though. > > -- > Jan Cholasta > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tbabej at redhat.com Tue Dec 8 15:33:05 2015 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 8 Dec 2015 16:33:05 +0100 Subject: [Freeipa-devel] [PATCH 0071] replica: Fix ipa-replica-install with replica file (domain, level 0). In-Reply-To: <5666F54C.80801@redhat.com> References: <5666E31E.1030705@redhat.com> <5666F54C.80801@redhat.com> Message-ID: <5666F831.5040705@redhat.com> On 12/08/2015 04:20 PM, Oleg Fayans wrote: > ACK. The initial issue is fixed. > > On 12/08/2015 03:03 PM, David Kupka wrote: >> https://fedorahosted.org/freeipa/ticket/5531 >> >> > Can we get some more love for the patch and provide at least a sentence worth of commit message before pushing? It's not obvious from the title what the patch does, other than it fixes things. Tomas From tbabej at redhat.com Tue Dec 8 15:53:49 2015 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 8 Dec 2015 16:53:49 +0100 Subject: [Freeipa-devel] [PATCH 0394] topology: Make sure the old 'realm' topology suffix is not In-Reply-To: <5666DAE6.3030708@redhat.com> References: <5666DAE6.3030708@redhat.com> Message-ID: <5666FD0D.6070001@redhat.com> On 12/08/2015 02:28 PM, Tomas Babej wrote: > Hi, > > The old 'realm' topology suffix is no longer used, however, it was being > created on masters with version 4.2.3 and later. Make sure it's properly > removed. > > Note that this is not the case for the 'ipaca' suffix, which was later > removed to 'ca'. > > https://fedorahosted.org/freeipa/ticket/5526 > Actually, we found out with Martin that this patch deletes realm instead of domain suffix, against all initial impressions. Updated patch attached. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0394-2-topology-Make-sure-the-old-realm-topology-suffix-is-.patch Type: text/x-patch Size: 1352 bytes Desc: not available URL: From redhatrises at gmail.com Tue Dec 8 16:04:48 2015 From: redhatrises at gmail.com (Gabe Alford) Date: Tue, 8 Dec 2015 09:04:48 -0700 Subject: [Freeipa-devel] [PATCH 0065] ipa-replica-install prints incorrect error message when replica is already installed In-Reply-To: <5666F6F0.2050005@redhat.com> References: <5666F2B8.1090803@redhat.com> <5666F3BB.8030006@redhat.com> <5666F497.3040500@redhat.com> <5666F539.1000904@redhat.com> <5666F6F0.2050005@redhat.com> Message-ID: Updated patch attached. On Tue, Dec 8, 2015 at 8:27 AM, Martin Basti wrote: > > > On 08.12.2015 16:26, Gabe Alford wrote: > > Just to confirm: > > if server is installed: > Let's stop here and not do anything else > > if domain level 0: > check if client installed and stop here > > Right? > > yes > > > > > On Tue, Dec 8, 2015 at 8:20 AM, Jan Cholasta wrote: > >> On 8.12.2015 16:17, Martin Basti wrote: >> >>> >>> >>> On 08.12.2015 16:14, Jan Cholasta wrote: >>> >>>> On 8.12.2015 16:09, Martin Basti wrote: >>>> >>>>> >>>>> >>>>> On 01.12.2015 14:57, Gabe Alford wrote: >>>>> >>>>>> Sorry guys, I forgot to add a meaningful subject to this message. >>>>>> Ignore the previous thread start. >>>>>> >>>>>> ---------- Forwarded message ---------- >>>>>> From: *Gabe Alford* >>>>> > >>>>>> Date: Mon, Nov 30, 2015 at 7:31 PM >>>>>> Subject: [PATCH 0065] >>>>>> To: freeipa-devel >>>>> > >>>>>> >>>>>> >>>>>> Hello, >>>>>> >>>>>> Patch fix for the following tickets: >>>>>> >>>>>> https://fedorahosted.org/freeipa/ticket/5022 >>>>>> https://fedorahosted.org/freeipa/ticket/5320 >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Gabe >>>>>> >>>>>> >>>>>> >>>>>> ACK >>>>> >>>> >>>> NACK, you can't install a server over an already installed client, >>>> thus the original check is correct. >>>> >>>> Ahh domain level 0, right, but this check can be added before the client >>> check. >>> >> >> Yes. >> >> With domain level 1, this check should stay there IMO. >>> >> >> Yes. It should say "IPA server is already configured" rather than "IPA >> replica is already configured", though. >> >> -- >> Jan Cholasta >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rga-0064-2-ipa-replica-install-prints-incorrect-error-message-w.patch Type: text/x-patch Size: 2427 bytes Desc: not available URL: From mbabinsk at redhat.com Tue Dec 8 16:32:17 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Tue, 8 Dec 2015 17:32:17 +0100 Subject: [Freeipa-devel] [PATCH 0394] topology: Make sure the old 'realm' topology suffix is not In-Reply-To: <5666FD0D.6070001@redhat.com> References: <5666DAE6.3030708@redhat.com> <5666FD0D.6070001@redhat.com> Message-ID: <56670611.5090604@redhat.com> On 12/08/2015 04:53 PM, Tomas Babej wrote: > > > On 12/08/2015 02:28 PM, Tomas Babej wrote: >> Hi, >> >> The old 'realm' topology suffix is no longer used, however, it was being >> created on masters with version 4.2.3 and later. Make sure it's properly >> removed. >> >> Note that this is not the case for the 'ipaca' suffix, which was later >> removed to 'ca'. >> >> https://fedorahosted.org/freeipa/ticket/5526 >> > > Actually, we found out with Martin that this patch deletes realm instead > of domain suffix, against all initial impressions. > > Updated patch attached. > > Tomas > > > Works for me, ACK. I have also made some hardening in topology connectivity checks so that this kind of situations is handled and reported by them. I will send a patch in separate thread. -- Martin^3 Babinsky From mbabinsk at redhat.com Tue Dec 8 16:35:37 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Tue, 8 Dec 2015 17:35:37 +0100 Subject: [Freeipa-devel] [PATCH 0114] harden domain level 1 topology connectivity checks Message-ID: <566706D9.2080006@redhat.com> A sort of auxilliary patch which makes topology checks more resistant to https://fedorahosted.org/freeipa/ticket/5526 If required I will open a separate ticket for it though. -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0114-harden-domain-level-1-topology-connectivity-checks.patch Type: text/x-patch Size: 4517 bytes Desc: not available URL: From mbabinsk at redhat.com Tue Dec 8 17:02:26 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Tue, 8 Dec 2015 18:02:26 +0100 Subject: [Freeipa-devel] [PATCH 0115] fix error message assertion in negative forced client reenrollment tests Message-ID: <56670D22.1050109@redhat.com> This patch fixes the assertions in negative test cases of 'test_forced_client_reenrollment' CI test suite. On ipa-4-2 it fixes https://fedorahosted.org/freeipa/ticket/5511 and makes all 8 tests in this suite green, shiny and happy again. It also fixes negative test cases on master branch, but the positive cases are broken there due to https://fedorahosted.org/freeipa/ticket/5528 -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0115-fix-error-message-assertion-in-negative-forced-clien.patch Type: text/x-patch Size: 1219 bytes Desc: not available URL: From ofayans at redhat.com Tue Dec 8 22:48:17 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Tue, 8 Dec 2015 23:48:17 +0100 Subject: [Freeipa-devel] [PATCH] First part of the replica promotion tests + testplan In-Reply-To: <5666DC18.2040906@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> <565DB7D3.5040500@redhat.com> <565F0BDD.8080206@redhat.com> <565F17EE.3020701@redhat.com> <5666DC18.2040906@redhat.com> Message-ID: <56675E31.3050203@redhat.com> Substituted a hardcoded suffix name with a constant DOMAIN_SUFFIX_NAME On 12/08/2015 02:33 PM, Oleg Fayans wrote: > Hi all, > > > The patches are rebased against the current master. > > On 12/02/2015 05:10 PM, Martin Basti wrote: >> >> >> On 02.12.2015 16:18, Oleg Fayans wrote: >>> Hi Martin, >>> >>> On 12/01/2015 04:08 PM, Martin Basti wrote: >>>> >>>> >>>> On 27.11.2015 16:26, Oleg Fayans wrote: >>>>> And patch N 16 passes lint too: >>>>> >>>>> On 11/27/2015 04:03 PM, Oleg Fayans wrote: >>>>>> Hi, >>>>>> >>>>>> On 11/27/2015 03:26 PM, Martin Basti wrote: >>>>>>> >>>>>>> >>>>>>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>>>>>> Hi Martin, >>>>>>>> >>>>>>>> All your suggestions were taken into account. Both patches are >>>>>>>> updated. Thank you for your help! >>>>>>>> >>>>>>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>>>>>> Hi Martin, >>>>>>>>>> >>>>>>>>>> I agree to all your points but one. please, see my comment below >>>>>>>>>> >>>>>>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> 0) Note >>>>>>>>>>> Please be aware of https://fedorahosted.org/freeipa/ticket/5469 >>>>>>>>>>> during >>>>>>>>>>> KRA testing >>>>>>>>>>> >>>>>>>>>>> 1) >>>>>>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, this may >>>>>>>>>>> change >>>>>>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for domain >>>>>>>>>>> level 0 >>>>>>>>>>> and 1 >>>>>>>>>>> >>>>>>>>>>> 2) >>>>>>>>>>> Why uninstall KRA then server, is not enough just uninstall >>>>>>>>>>> server >>>>>>>>>>> which >>>>>>>>>>> covers KRA uninstall? >>>>>>>>>>> >>>>>>>>>>> + def teardown_method(self, method): >>>>>>>>>>> + for host in self.replicas: >>>>>>>>>>> + host.run_command(self.kra_uninstall, >>>>>>>>>>> raiseonerr=False) >>>>>>>>>>> + tasks.uninstall_master(host) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 3) >>>>>>>>>>> Can be this function more generic? It should allow specify host >>>>>>>>>>> where >>>>>>>>>>> KRA should be installed not just master >>>>>>>>>>> >>>>>>>>>>> + def test_kra_install_master(self): >>>>>>>>>>> + self.master.run_command(self.kra_install) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 4) >>>>>>>>>>> >>>>>>>>>>> TestLevel0(Dummy): >>>>>>>>>>> Can be the test name more specific, something like >>>>>>>>>>> TestReplicaPromotionLevel0 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 5) >>>>>>>>>>> please remove this, the patch is on review and it will be pushed >>>>>>>>>>> sooner >>>>>>>>>>> than tests >>>>>>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>>>>>> >>>>>>>>>>> and as I mentioned in ticket #5455, I cannot reproduce it with >>>>>>>>>>> ipa-kra-install, so please provide steps to reproduce if you >>>>>>>>>>> insist >>>>>>>>>>> that >>>>>>>>>>> this still does not work as expected with KRA. >>>>>>>>>>> >>>>>>>>>>> 6) This is completely wrong, it removes everything that we >>>>>>>>>>> tried to >>>>>>>>>>> achieve with previous patches with domain level in CI >>>>>>>>>> >>>>>>>>>> Actually, being able to configure domain level per class is WAY >>>>>>>>>> more >>>>>>>>>> convenient, than to always have to think which domain level is >>>>>>>>>> appropriate for which particular test during jenkins job >>>>>>>>>> configuration. In fact, I should have thought about it from the >>>>>>>>>> very >>>>>>>>>> beginning. For example, in test_replica_promotion.py we have on >>>>>>>>>> class, >>>>>>>>>> which intiates with domain level = 1, while others - with domain >>>>>>>>>> level >>>>>>>>>> 0. With config-based approach, we would have to implement a >>>>>>>>>> separate >>>>>>>>>> step that raises domain level. Overall, I am against the approach, >>>>>>>>>> when you have to remember to set certain domain level in config >>>>>>>>>> for >>>>>>>>>> any particular test. The tests themselves should be aware of the >>>>>>>>>> domain level they need. >>>>>>>>> I do not say that we should not have something that overrides >>>>>>>>> settings >>>>>>>>> in from config in a particular test case, I say your patch is >>>>>>>>> doing it >>>>>>>>> wrong. >>>>>>>>> >>>>>>>>> I agree it is useful to have param domain_level in install_master, >>>>>>>>> and >>>>>>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by default, >>>>>>>>> because with your current patch the domain_level in config is not >>>>>>>>> used >>>>>>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>>>>>> >>>>>>>>> For example I want to achieve this goal: >>>>>>>>> test_vault.py, this test suite can run on domain level1 and on >>>>>>>>> domain >>>>>>>>> level0, so with one test we can test 2 domain levels just with >>>>>>>>> putting >>>>>>>>> domain level into config file. >>>>>>>>> >>>>>>>>> I agree that with extraordinary test like replica promotion test >>>>>>>>> is, we >>>>>>>>> need something that allows override the config file. >>>>>>>>> >>>>>>>>> As I said bellow, domain_level default value should be None in >>>>>>>>> install_master and install_topo plugin. If domain level was >>>>>>>>> specified >>>>>>>>> use the specified one, if not (value is None) use the domain level >>>>>>>>> from >>>>>>>>> config file. >>>>>>>> >>>>>>>> Agreed :) >>>>>>>> >>>>>>>>> >>>>>>>>> Martin >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> [PATCH] Enabled setting domain_level per class derived from >>>>>>>>>>> TestIntegration >>>>>>>>>>> >>>>>>>>>>> When I configure domain level 0 in yaml config, how is this >>>>>>>>>>> supposed to >>>>>>>>>>> get into install methods when you removed that code? >>>>>>>>>>> >>>>>>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>>>>>> + "--domain-level=%i" % domain_level >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> You always use MAX_DOMAIN_LEVEL in this case or whatever is >>>>>>>>>>> specified in >>>>>>>>>>> domain_level option. >>>>>>>>>>> I suggest to use domain_level=None, and when it is None use >>>>>>>>>>> 'host.config.domain_level', if it is not None, use 'domain_level' >>>>>>>>>>> >>>>>>>>>>> With this we can specify domain level in config file for test >>>>>>>>>>> that >>>>>>>>>>> can >>>>>>>>>>> be used for both domain levels and you can manually specify >>>>>>>>>>> domain >>>>>>>>>>> level >>>>>>>>>>> for test that requires specific domain level. >>>>>>>>>>> >>>>>>>>>>> Also this should go away >>>>>>>>>>> >>>>>>>>>>> @classmethod >>>>>>>>>>> def install(cls, mh): >>>>>>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>>>>>> if cls.topology is None: >>>>>>>>>>> return >>>>>>>>>>> >>>>>>>>>>> I do not see reason why test should override configuration in >>>>>>>>>>> config in >>>>>>>>>>> this case. >>>>>>>>>>> >>>>>>>>>>> Martin >>>>>>>>>>> >>>>>>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Here is the updated version of the patch (more tests + fixed the >>>>>>>>>>>> issues of the first one) + patch 0017, that implements the >>>>>>>>>>>> necessary >>>>>>>>>>>> changes in the background code, i. e. patch 16 does not work >>>>>>>>>>>> without >>>>>>>>>>>> patch 17 >>>>>>>>>>>> >>>>>>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>>>>>> Hi guys, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Here are first two automated testcases from this (so far >>>>>>>>>>>>>> incomplete) >>>>>>>>>>>>>> testplan: >>>>>>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>>>>>> >>>>>>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> PATCH 16: NACK >>>>>>>>>>>>> >>>>>>>>>>>>> 1) >>>>>>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>>>>>> 'domain_level' to >>>>>>>>>>>>> install_topo()? >>>>>>>>>>>>> Also it is good practise to add new option as the last >>>>>>>>>>>>> parameter. >>>>>>>>>>>>> >>>>>>>>>>>>> 2) >>>>>>>>>>>>> cab you in both tests specify a domain level with constant >>>>>>>>>>>>> instead of >>>>>>>>>>>>> number literal? >>>>>>>>>>>>> >>>>>>>>>>>>> 3) >>>>>>>>>>>>> both test call install_topo with custom domain level, but it >>>>>>>>>>>>> cannot >>>>>>>>>>>>> work >>>>>>>>>>>>> because 1) (did you run the test?) >>>>>>>>>>>>> >>>>>>>>>>>>> 4) >>>>>>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>>>>>> Respectively why there is call of install_topo() that installs >>>>>>>>>>>>> replica. >>>>>>>>>>>>> As this test just tests that ipa-replica-prepare is not working >>>>>>>>>>>>> anymore, >>>>>>>>>>>>> is it worth to spend 20 minutes with installing replica and >>>>>>>>>>>>> then >>>>>>>>>>>>> just no >>>>>>>>>>>>> tot use it? IMO to install master in install step is enough. >>>>>>>>>>>>> >>>>>>>>>>>>> Martin^2 >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> ./make-lint >>>>>>> ************* Module ipatests.test_integration.base >>>>>>> ipatests/test_integration/base.py:66: [E1101(no-member), >>>>>>> IntegrationTest.install] Class 'IntegrationTest' has no >>>>>>> 'domain_level' >>>>>>> member) >>>>>>> ************* Module ipatests.test_integration.test_replica_promotion >>>>>>> ipatests/test_integration/test_replica_promotion.py:16: >>>>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no 'domain_level' >>>>>>> member) >>>>>>> ipatests/test_integration/test_replica_promotion.py:117: >>>>>>> [E1101(no-member), >>>>>>> TestCAInstall.test_ca_install_without_replica_file] >>>>>>> Module 'ipatests.test_integration.tasks' has no 'setup_replica' >>>>>>> member) >>>>>>> >>>>>>> >>>>>>> Is it so hard to run pylint before patch is posted for review? >>>>>> >>>>>> Sorry, my bad. Fixed. >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> 1) >>>> Why is this change in the patch? >>>> - # Clean up the test directory >>>> - host.run_command(['rm', '-rvf', host.config.test_dir]) >>> >>> Otherwise 2 out of 8 tests fail with IOError at line 78 of >>> ipatests/test_integration/tasks.py >>> >>> I do not understand yet how does this happen, but if you remove >>> ipatests folder once, it then fails to be created again. >>> >> So this should be in separated patch and investigated properly. > Agree. Removed >> >>>> - >>>> >>>> >>>> 2) >>>> is enough to have this check only in install_master, install_topo can >>>> pass None to install_master >>>> + if domain_level is None: >>>> + domain_level = master.config.domain_level >>> >>> Done >>> >>>> >>>> >>>> 3) >>>> IMO replica-manage del should cleanup hosts entry, so following code >>>> should not be needed. >>>> + if cleanhost: >>>> + kinit_admin(master) >>>> + master.run_command(["ipa", "host-del", "--updatedns", >>>> replica.hostname], >>>> + raiseonerr=False) >>> >>> Well, in fact it does not. At least the corresponding dns record stays >>> and causes the subsequent ipa-client-install to fail. Probably it's a >>> bug. On the other hand, if I promote an existing client to replica and >>> then delete this replica, then, I probably want the host record (that >>> was created during client-install) to stay in the system. So, does not >>> look like a bug to me. >> No you don't, because replica uninstallation also removes the client. >> >> I tried it with ipa42, ipa-replica-manage del removes host entry, and >> DNS A records, only SSHFP records stay there (I'm not sure if it is bug >> or feature) >> >> Also I received this message >> """ >> Failed to cleanup replica1.ipa.test DNS entries: no matching entry found >> You may need to manually remove them from the tree >> """ >> But, A record has been removed, so this is probably false positive and >> it needs to have a ticket. > > Agree, that was an issue with my setup. > Removed > >>> >>>> >>>> 4) >>>> This variable is not used >>>> + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] >>> >>> Removed >>> >>>> >>>> 5) >>>> Why do you need this >>>> + kra_install = ["ipa-kra-install", "-U", "-p", >>>> config.dirman_password] >>>> when you implemented tasks.install_kra that returns the exactly the same >>>> result? >>> >>> Right. Removed >>> >>>> >>>> 6) >>>> PEP8 >>>> ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 blank lines, >>>> found 1 >>>> ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 blank lines, >>>> found 1 >>>> ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 blank lines, >>>> found 1 >>>> ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 blank lines, >>>> found 1 >>>> ./ipatests/test_integration/tasks.py:950:80: E501 line too long (80 > 79 >>>> characters) >>>> >>>> ./ipatests/test_integration/test_replica_promotion.py:29:80: E501 line >>>> too long (85 > 79 characters) >>>> ./ipatests/test_integration/test_replica_promotion.py:64:80: E501 line >>>> too long (85 > 79 characters) >>>> ./ipatests/test_integration/test_replica_promotion.py:67:80: E501 line >>>> too long (88 > 79 characters) >>>> ./ipatests/test_integration/test_replica_promotion.py:93:80: E501 line >>>> too long (80 > 79 characters) >>>> ./ipatests/test_integration/test_replica_promotion.py:94:80: E501 line >>>> too long (83 > 79 characters) >>>> ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 line >>>> too long (81 > 79 characters) >>>> ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 line >>>> too long (80 > 79 characters) >>>> ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 line >>>> too long (82 > 79 characters) >>>> ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 line >>>> too long (80 > 79 characters) >>> >>> Most of these complaints are unrelated to the current patches. >>> It's better to create a separate patch addressing PEP8 errors. >>> >> I beg for your pardon, those PEP8 errors have been introduced by your >> patches. > > Fixed > >>>> >>>> 7) >>>> Why this must be stored in instance? IMO to have it stored as local >>>> variable is perfect >>>> TestKRAInstall, TestCAInstall >>>> self.replica1_filename = tasks.get_replica_filename(replica1) >>>> self.replica2_filename = tasks.get_replica_filename(replica2) >>> >>> Agree. Fixed >>> >> > > > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0016.6-First-part-of-replica-promotion-tests.patch Type: text/x-patch Size: 1505 bytes Desc: not available URL: From jcholast at redhat.com Wed Dec 9 06:08:38 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 9 Dec 2015 07:08:38 +0100 Subject: [Freeipa-devel] [PATCH 0065] ipa-replica-install prints incorrect error message when replica is already installed In-Reply-To: References: <5666F2B8.1090803@redhat.com> <5666F3BB.8030006@redhat.com> <5666F497.3040500@redhat.com> <5666F539.1000904@redhat.com> <5666F6F0.2050005@redhat.com> Message-ID: <5667C566.5000906@redhat.com> LGTM On 8.12.2015 17:04, Gabe Alford wrote: > Updated patch attached. > > On Tue, Dec 8, 2015 at 8:27 AM, Martin Basti > wrote: > > > > On 08.12.2015 16:26, Gabe Alford wrote: >> Just to confirm: >> >> if server is installed: >> Let's stop here and not do anything else >> >> if domain level 0: >> check if client installed and stop here >> >> Right? > yes > > >> >> >> On Tue, Dec 8, 2015 at 8:20 AM, Jan Cholasta > > wrote: >> >> On 8.12.2015 16:17, Martin Basti wrote: >> >> >> >> On 08.12.2015 16:14, Jan Cholasta wrote: >> >> On 8.12.2015 16:09, Martin Basti wrote: >> >> >> >> On 01.12.2015 14:57, Gabe Alford wrote: >> >> Sorry guys, I forgot to add a meaningful >> subject to this message. >> Ignore the previous thread start. >> >> ---------- Forwarded message ---------- >> From: *Gabe Alford* > >> > >> >> Date: Mon, Nov 30, 2015 at 7:31 PM >> Subject: [PATCH 0065] >> To: freeipa-devel > >> > >> >> >> >> Hello, >> >> Patch fix for the following tickets: >> >> https://fedorahosted.org/freeipa/ticket/5022 >> https://fedorahosted.org/freeipa/ticket/5320 >> >> Thanks, >> >> Gabe >> >> >> >> ACK >> >> >> NACK, you can't install a server over an already >> installed client, >> thus the original check is correct. >> >> Ahh domain level 0, right, but this check can be added >> before the client >> check. >> >> >> Yes. >> >> With domain level 1, this check should stay there IMO. >> >> >> Yes. It should say "IPA server is already configured" rather >> than "IPA replica is already configured", though. >> >> -- >> Jan Cholasta >> >> > > -- Jan Cholasta From mbabinsk at redhat.com Wed Dec 9 06:37:36 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Wed, 9 Dec 2015 07:37:36 +0100 Subject: [Freeipa-devel] [PATCH 0112] CI tests: ignore disconnected domain level 1 topology on IPA master teardown In-Reply-To: <56658147.1030508@redhat.com> References: <566559E4.9020506@redhat.com> <56656879.2050007@redhat.com> <56658147.1030508@redhat.com> Message-ID: <5667CC30.5020004@redhat.com> On 12/07/2015 01:53 PM, Martin Babinsky wrote: > On 12/07/2015 12:07 PM, Oleg Fayans wrote: >> Hi Martin, >> >> CONFIGURED_DOMAIN_LEVEL is declared, but not used. The rest looks fine >> to me >> >> On 12/07/2015 11:05 AM, Martin Babinsky wrote: >>> This patch should fix teardown methods in replication-related CI tests >>> ran at non-zero domain level. >>> >>> >>> >> > Ah that was a leftover from previous implementation. Here's updated patch. > > > Patch needed a rebase. Attaching new revision. -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0112.2-CI-tests-ignore-disconnected-domain-level-1-topology.patch Type: text/x-patch Size: 2216 bytes Desc: not available URL: From ofayans at redhat.com Wed Dec 9 07:26:19 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Wed, 9 Dec 2015 08:26:19 +0100 Subject: [Freeipa-devel] [PATCH 0112] CI tests: ignore disconnected domain level 1 topology on IPA master teardown In-Reply-To: <5667CC30.5020004@redhat.com> References: <566559E4.9020506@redhat.com> <56656879.2050007@redhat.com> <56658147.1030508@redhat.com> <5667CC30.5020004@redhat.com> Message-ID: <5667D79B.7090904@redhat.com> ACK On 12/09/2015 07:37 AM, Martin Babinsky wrote: > On 12/07/2015 01:53 PM, Martin Babinsky wrote: >> On 12/07/2015 12:07 PM, Oleg Fayans wrote: >>> Hi Martin, >>> >>> CONFIGURED_DOMAIN_LEVEL is declared, but not used. The rest looks fine >>> to me >>> >>> On 12/07/2015 11:05 AM, Martin Babinsky wrote: >>>> This patch should fix teardown methods in replication-related CI tests >>>> ran at non-zero domain level. >>>> >>>> >>>> >>> >> Ah that was a leftover from previous implementation. Here's updated >> patch. >> >> >> > Patch needed a rebase. Attaching new revision. > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From dkupka at redhat.com Wed Dec 9 07:31:44 2015 From: dkupka at redhat.com (David Kupka) Date: Wed, 9 Dec 2015 08:31:44 +0100 Subject: [Freeipa-devel] [PATCH 0071] replica: Fix ipa-replica-install with replica file (domain, level 0). In-Reply-To: <5666F831.5040705@redhat.com> References: <5666E31E.1030705@redhat.com> <5666F54C.80801@redhat.com> <5666F831.5040705@redhat.com> Message-ID: <5667D8E0.6070303@redhat.com> On 08/12/15 16:33, Tomas Babej wrote: > > > On 12/08/2015 04:20 PM, Oleg Fayans wrote: >> ACK. The initial issue is fixed. >> >> On 12/08/2015 03:03 PM, David Kupka wrote: >>> https://fedorahosted.org/freeipa/ticket/5531 >>> >>> >> > > Can we get some more love for the patch and provide at least a sentence > worth of commit message before pushing? > > It's not obvious from the title what the patch does, other than it fixes > things. > > Tomas > I believe it's pretty obvious from linked ticket and attached patch changing just 5 lines. But you're right verbosity in commit message could save time later. Patch with changed commit message attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0071.1-replica-Fix-ipa-replica-install-with-replica-file-dom.patch Type: text/x-patch Size: 2575 bytes Desc: not available URL: From mbasti at redhat.com Wed Dec 9 08:40:14 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 9 Dec 2015 09:40:14 +0100 Subject: [Freeipa-devel] [PATCH 0112] CI tests: ignore disconnected domain level 1 topology on IPA master teardown In-Reply-To: <5667D79B.7090904@redhat.com> References: <566559E4.9020506@redhat.com> <56656879.2050007@redhat.com> <56658147.1030508@redhat.com> <5667CC30.5020004@redhat.com> <5667D79B.7090904@redhat.com> Message-ID: <5667E8EE.5090906@redhat.com> On 09.12.2015 08:26, Oleg Fayans wrote: > ACK > > On 12/09/2015 07:37 AM, Martin Babinsky wrote: >> On 12/07/2015 01:53 PM, Martin Babinsky wrote: >>> On 12/07/2015 12:07 PM, Oleg Fayans wrote: >>>> Hi Martin, >>>> >>>> CONFIGURED_DOMAIN_LEVEL is declared, but not used. The rest looks fine >>>> to me >>>> >>>> On 12/07/2015 11:05 AM, Martin Babinsky wrote: >>>>> This patch should fix teardown methods in replication-related CI tests >>>>> ran at non-zero domain level. >>>>> >>>>> >>>>> >>> Ah that was a leftover from previous implementation. Here's updated >>> patch. >>> >>> >>> >> Patch needed a rebase. Attaching new revision. >> >> Pushed to master: 35fae355cc4441164f149288b3d126129a82aed7 From ldoudova at redhat.com Wed Dec 9 08:41:38 2015 From: ldoudova at redhat.com (Lenka Doudova) Date: Wed, 9 Dec 2015 09:41:38 +0100 Subject: [Freeipa-devel] [TESTS][PATCH 0006] Add comments to stageuser plugin tests In-Reply-To: <564F004A.2060806@redhat.com> References: <564C7A23.5010500@redhat.com> <564C9E77.4050406@redhat.com> <564D88AF.8010500@redhat.com> <564D97AC.6050901@redhat.com> <564F004A.2060806@redhat.com> Message-ID: <5667E942.7030305@redhat.com> Hi, attaching fixed patches for master and ipa-4-2 branch. Also fixes test accordingly to https://fedorahosted.org/freeipa/ticket/5387. Lenka On 11/20/2015 12:13 PM, Martin Babinsky wrote: > On 11/19/2015 10:34 AM, Petr Viktorin wrote: >> On 11/19/2015 09:30 AM, Lenka Doudova wrote: >>> On 11/18/2015 04:51 PM, Martin Babinsky wrote: >>>> On 11/18/2015 02:16 PM, Lenka Doudova wrote: >>>>> Hi, >>>>> >>>>> here's a patch that adds a few comments to stageuser tests in >>>>> order to >>>>> allow easier determining of a problem when tests fail. >>>>> >>>>> Lenka >>>>> >>>>> >>>> >>>> Hi Lenka, >>>> >>>> Firstly a technical detail: Python indexes lists from 0, so the >>>> comments in 'options_ok' do not correctly map to the test names >>>> anyway. >>>> >>>> I am also not sure if this patch is worth reviewing and pushing as it >>>> IMHO doesn't help in the identification of failed tests at all. >>>> >>>> This should be solved at more fundamental level. >>>> >>> Ouch, good point, I didn't realize. Sorry. >>> >>> Anyway, the issue is that even if tests are run in verbose mode, you >>> get >>> output like this: >>> >>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser27] >>> >>> PASSED >>> >>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser28] >>> >>> PASSED >>> >>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser29] >>> >>> PASSED >>> >>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser210] >>> >>> PASSED >>> >>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser211] >>> >>> PASSED >>> >>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser212] >>> >>> PASSED >>> >>> >>> If some test fails, you can't really tell which command was the one >>> responsible for the fail. This should be solved by >>> https://fedorahosted.org/freeipa/ticket/5449. Until that's done, >>> though, >>> the only way to find out which command failed is looking at the code >>> and >>> finding which parameters were put into the command, which was not much >>> possible without better commenting the test case (as I realized last >>> week when David Kupka asked me to help him find the reason for failed >>> tests). >>> Obviously I can rewrite the tests so there's 27 separate test cases, >>> one >>> for each parameter, instead of one method that 'unwraps' into 27 test >>> cases, which would entirely eliminate the confusion. So far I don't >>> know >>> of a way to put 27 similar test cases in one method which would allow >>> easy recognition of the test cases. >>> I'll wait with fixing the patch until further discussion. >> >> Hello, >> Pytest wants you to be a bit more explicit about how to name the >> parameters. (It "hides" dicts by default, because large dicts would make >> the output even more confusing than the numbers.) >> >> Please try the attached patch. >> Docs are at >> https://pytest.org/latest/fixture.html#parametrizing-a-fixture >> >> >> > This looks like a better approach IMHO, you can then see which > attribute/value was being checked. > > I would very much favor more descriptive test/fixture names in the > beginning. > -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ldoudova-0006.2-Adding-descriptive-IDs-to-stageuser-tests.patch Type: text/x-patch Size: 5887 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ldoudova-0006.2-ipa42-Adding-descriptive-IDs-to-stageuser-tests.patch Type: text/x-patch Size: 5026 bytes Desc: not available URL: From mbasti at redhat.com Wed Dec 9 08:43:02 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 9 Dec 2015 09:43:02 +0100 Subject: [Freeipa-devel] [PATCH 0394] topology: Make sure the old 'realm' topology suffix is not In-Reply-To: <56670611.5090604@redhat.com> References: <5666DAE6.3030708@redhat.com> <5666FD0D.6070001@redhat.com> <56670611.5090604@redhat.com> Message-ID: <5667E996.2070900@redhat.com> On 08.12.2015 17:32, Martin Babinsky wrote: > On 12/08/2015 04:53 PM, Tomas Babej wrote: >> >> >> On 12/08/2015 02:28 PM, Tomas Babej wrote: >>> Hi, >>> >>> The old 'realm' topology suffix is no longer used, however, it was >>> being >>> created on masters with version 4.2.3 and later. Make sure it's >>> properly >>> removed. >>> >>> Note that this is not the case for the 'ipaca' suffix, which was later >>> removed to 'ca'. >>> >>> https://fedorahosted.org/freeipa/ticket/5526 >>> >> >> Actually, we found out with Martin that this patch deletes realm instead >> of domain suffix, against all initial impressions. >> >> Updated patch attached. >> >> Tomas >> >> >> > > Works for me, ACK. > > I have also made some hardening in topology connectivity checks so > that this kind of situations is handled and reported by them. I will > send a patch in separate thread. > Pushed to master: a84b7d2117aafc5182640d0a22675b214c27dd7c From jcholast at redhat.com Wed Dec 9 09:02:36 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 9 Dec 2015 10:02:36 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API Message-ID: <5667EE2C.2050100@redhat.com> Hi, the attached patches fix . Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-523-build-put-oddjob-scripts-into-separate-directory.patch Type: text/x-patch Size: 2412 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-524-replica-install-add-remote-connection-check-over-API.patch Type: text/x-patch Size: 29449 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-525-replica-promotion-use-host-credentials-for-connectio.patch Type: text/x-patch Size: 2221 bytes Desc: not available URL: From mbasti at redhat.com Wed Dec 9 09:13:10 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 9 Dec 2015 10:13:10 +0100 Subject: [Freeipa-devel] [TESTS][PATCH 0006] Add comments to stageuser plugin tests In-Reply-To: <5667E942.7030305@redhat.com> References: <564C7A23.5010500@redhat.com> <564C9E77.4050406@redhat.com> <564D88AF.8010500@redhat.com> <564D97AC.6050901@redhat.com> <564F004A.2060806@redhat.com> <5667E942.7030305@redhat.com> Message-ID: <5667F0A6.1090104@redhat.com> On 09.12.2015 09:41, Lenka Doudova wrote: > Hi, > > attaching fixed patches for master and ipa-4-2 branch. > Also fixes test accordingly to > https://fedorahosted.org/freeipa/ticket/5387. > > Lenka > > On 11/20/2015 12:13 PM, Martin Babinsky wrote: >> On 11/19/2015 10:34 AM, Petr Viktorin wrote: >>> On 11/19/2015 09:30 AM, Lenka Doudova wrote: >>>> On 11/18/2015 04:51 PM, Martin Babinsky wrote: >>>>> On 11/18/2015 02:16 PM, Lenka Doudova wrote: >>>>>> Hi, >>>>>> >>>>>> here's a patch that adds a few comments to stageuser tests in >>>>>> order to >>>>>> allow easier determining of a problem when tests fail. >>>>>> >>>>>> Lenka >>>>>> >>>>>> >>>>> >>>>> Hi Lenka, >>>>> >>>>> Firstly a technical detail: Python indexes lists from 0, so the >>>>> comments in 'options_ok' do not correctly map to the test names >>>>> anyway. >>>>> >>>>> I am also not sure if this patch is worth reviewing and pushing as it >>>>> IMHO doesn't help in the identification of failed tests at all. >>>>> >>>>> This should be solved at more fundamental level. >>>>> >>>> Ouch, good point, I didn't realize. Sorry. >>>> >>>> Anyway, the issue is that even if tests are run in verbose mode, >>>> you get >>>> output like this: >>>> >>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser27] >>>> >>>> PASSED >>>> >>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser28] >>>> >>>> PASSED >>>> >>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser29] >>>> >>>> PASSED >>>> >>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser210] >>>> >>>> PASSED >>>> >>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser211] >>>> >>>> PASSED >>>> >>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser212] >>>> >>>> PASSED >>>> >>>> >>>> If some test fails, you can't really tell which command was the one >>>> responsible for the fail. This should be solved by >>>> https://fedorahosted.org/freeipa/ticket/5449. Until that's done, >>>> though, >>>> the only way to find out which command failed is looking at the >>>> code and >>>> finding which parameters were put into the command, which was not much >>>> possible without better commenting the test case (as I realized last >>>> week when David Kupka asked me to help him find the reason for failed >>>> tests). >>>> Obviously I can rewrite the tests so there's 27 separate test >>>> cases, one >>>> for each parameter, instead of one method that 'unwraps' into 27 test >>>> cases, which would entirely eliminate the confusion. So far I don't >>>> know >>>> of a way to put 27 similar test cases in one method which would allow >>>> easy recognition of the test cases. >>>> I'll wait with fixing the patch until further discussion. >>> >>> Hello, >>> Pytest wants you to be a bit more explicit about how to name the >>> parameters. (It "hides" dicts by default, because large dicts would >>> make >>> the output even more confusing than the numbers.) >>> >>> Please try the attached patch. >>> Docs are at >>> https://pytest.org/latest/fixture.html#parametrizing-a-fixture >>> >>> >>> >> This looks like a better approach IMHO, you can then see which >> attribute/value was being checked. >> >> I would very much favor more descriptive test/fixture names in the >> beginning. >> > > > Hello, we use usually bottom posting on freeipa-devel please try to keep reply in this way. Patch: I do not like the idea of separated lists, IMO it is hard to manage and is easy to create mistakes. How about this (untested, just from top of my head): http://fpaste.org/298994/65184014/ Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbasti at redhat.com Wed Dec 9 09:17:03 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 9 Dec 2015 10:17:03 +0100 Subject: [Freeipa-devel] [PATCH 522] replica promotion: allow OTP bulk client enrollment In-Reply-To: <5666CADC.3070407@redhat.com> References: <565F0D00.6050805@redhat.com> <56653386.5020703@redhat.com> <5665E807.3070600@redhat.com> <56668C43.3000908@redhat.com> <5666A35D.8040203@redhat.com> <5666C3C7.2070505@redhat.com> <5666C891.1020205@redhat.com> <5666CADC.3070407@redhat.com> Message-ID: <5667F18F.5010009@redhat.com> On 08.12.2015 13:19, Martin Basti wrote: > > > On 08.12.2015 13:09, Jan Cholasta wrote: >> On 8.12.2015 12:49, Martin Basti wrote: >>> >>> >>> On 08.12.2015 10:31, Martin Basti wrote: >>>> >>>> >>>> On 08.12.2015 08:52, Jan Cholasta wrote: >>>>> On 7.12.2015 21:11, Martin Basti wrote: >>>>>> >>>>>> >>>>>> On 07.12.2015 08:21, Jan Cholasta wrote: >>>>>>> On 2.12.2015 16:23, Jan Cholasta wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> the attached patch fixes >>>>>>>> . >>>>>>>> >>>>>>>> Note that you still have to provide admin password in >>>>>>>> ipa-replica-install, either using --admin-password or >>>>>>>> interactively, >>>>>>>> because: >>>>>>>> >>>>>>>> a) Admin password is required for replica promotion. This will be >>>>>>>> fixed >>>>>>>> with . >>>>>>>> >>>>>>>> Patches are on the list: >>>>>>>> . >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> Pushed. >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> b) Admin password is required for connection check. This will be >>>>>>>> fixed >>>>>>>> with . >>>>>>> >>>>>>> Martin Basti pointed out that admin password should not be asked >>>>>>> interactively during OTP replica promotion. Fixed. >>>>>>> >>>>>>> Updated and rebased patch attached. >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> 1) >>>>>> [root at vm-058-138 ~]# ipa-replica-install --server >>>>>> vm-058-137.abc.idm.lab.eng.brq.redhat.com --domain >>>>>> abc.idm.lab.eng.brq.redhat.com --password=bubak --setup-ca >>>>>> Configuring client side components >>>>>> Password for admin at ABC.IDM.LAB.ENG.BRQ.REDHAT.COM: >>>>>> >>>>>> IMO password should be asked first, before any installation >>>>>> begins (IMO >>>>>> this is for conncheck) >>>>> >>>>> The same thing happens without my patch. Could you file a ticket? >>>> https://fedorahosted.org/freeipa/ticket/5525 >>>> >>>>> >>>>>> >>>>>> 2) >>>>>> When host is not in ipaservers hostgroup. Also I would expect >>>>>> different >>>>>> error message >>>>>> ipa-replica-install --server >>>>>> vm-058-137.abc.idm.lab.eng.brq.redhat.com >>>>>> --domain abc.idm.lab.eng.brq.redhat.com --password=bubak --setup-ca >>>>>> --skip-conncheck >>>>>> >>>>>> .... >>>>>> step() >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 352, in >>>>>> step = lambda: next(self.__gen) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>> line 81, in run_generator_with_yield_from >>>>>> six.reraise(*exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>> line 59, in run_generator_with_yield_from >>>>>> value = gen.send(prev_value) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>>>> line 63, in _install >>>>>> for nothing in self._installer(self.parent): >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >>>>>> >>>>>> >>>>>> line 1507, in main >>>>>> promote_check(self) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >>>>>> >>>>>> >>>>>> line 374, in decorated >>>>>> func(installer) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", >>>>>> >>>>>> >>>>>> line 1002, in promote_check >>>>>> conn.connect(ccache=installer._ccache) >>>>>> File "/usr/lib/python2.7/site-packages/ipalib/backend.py", >>>>>> line 66, >>>>>> in connect >>>>>> conn = self.create_connection(*args, **kw) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", >>>>>> line 199, in create_connection >>>>>> principal = krb_utils.get_principal(ccache_name=ccache) >>>>>> File "/usr/lib/python2.7/site-packages/ipalib/krb_utils.py", line >>>>>> 184, in get_principal >>>>>> raise errors.CCacheError(message=unicode(e)) >>>>>> >>>>>> 2015-12-07T16:23:40Z DEBUG The ipa-replica-install command failed, >>>>>> exception: CCacheError: Major (851968): Unspecified GSS failure. >>>>>> Minor >>>>>> code may provide more information, Minor (2529639053): No Kerberos >>>>>> credentials available >>>>>> 2015-12-07T16:23:40Z ERROR Major (851968): Unspecified GSS failure. >>>>>> Minor code may provide more information, Minor (2529639053): No >>>>>> Kerberos >>>>>> credentials available >>>>> >>>>> Fixed. >>>>> >>>>>> >>>>>> >>>>>> 3) >>>>>> This case is not handle very well: >>>>>> a) install client with OTP password >>>>>> b) install replica with the same OTP password (when host is no in >>>>>> ipaservers group, if host is in ipaservers group it works) >>>>>> >>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR Major >>>>>> (851968): Unspecified GSS failure. Minor code may provide more >>>>>> information, Minor (2529639053): No Kerberos credentials available >>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The >>>>>> ipa-replica-install command failed. See >>>>>> /var/log/ipareplica-install.log >>>>>> for more information >>>>> >>>>> This is the same as 2). >>>>> >>>>>> >>>>>> 4) >>>>>> This is not user friendly >>>>>> I used wrong OTP password, can we somehow propagate the actual error >>>>>> from client install to stderr? >>>>>> >>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR >>>>>> Configuration of >>>>>> client side components failed! >>>>>> ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' >>>>>> '--unattended' '--domain' 'abc.idm.lab.eng.brq.redhat.com' >>>>>> '--server' >>>>>> 'vm-058-137.abc.idm.lab.eng.brq.redhat.com' '--password' 'buba'' >>>>>> returned non-zero exit status 1 >>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The >>>>>> ipa-replica-install command failed. See >>>>>> /var/log/ipareplica-install.log >>>>>> for more information >>>>> >>>>> The same thing happens without my patch for any other error. Could >>>>> you file a ticket? >>>> >>>> https://fedorahosted.org/freeipa/ticket/5527 >>>> >>>>> >>>>> Updated patch attached. >>>>> >>>> Working on review >>>> >>> >>> Is this expected that client will be installed even if there is not >>> enough privileges to install replica? >>> >>> # ipa-replica-install --server >>> vm-058-137.abc.idm.lab.eng.brq.redhat.com --domain >>> abc.idm.lab.eng.brq.redhat.com --password bubak --skip-conncheck >>> Configuring client side components >>> ipa.ipapython.install.cli.install_tool(Replica): ERROR >>> Insufficient privileges to promote the server. >>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The >>> ipa-replica-install command failed. See /var/log/ipareplica-install.log >>> for more information >> >> Yes. The check can't be done without the host keytab, which you get >> with ipa-client-install. If ipa-client-install wasn't monolithic, the >> check could be done earlier, but we are not there yet. >> >> The client should be probably uninstalled in case of failure, though. >> > ACK > > I will report this in a separate ticket > Pushed to master: faf608556427849b33f4525b9bac2e71020bb962 From mbasti at redhat.com Wed Dec 9 09:30:03 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 9 Dec 2015 10:30:03 +0100 Subject: [Freeipa-devel] [PATCH] First part of the replica promotion tests + testplan In-Reply-To: <56675E31.3050203@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> <565DB7D3.5040500@redhat.com> <565F0BDD.8080206@redhat.com> <565F17EE.3020701@redhat.com> <5666DC18.2040906@redhat.com> <56675E31.3050203@redhat.com> Message-ID: <5667F49B.5020709@redhat.com> On 08.12.2015 23:48, Oleg Fayans wrote: > Substituted a hardcoded suffix name with a constant DOMAIN_SUFFIX_NAME > > On 12/08/2015 02:33 PM, Oleg Fayans wrote: >> Hi all, >> >> >> The patches are rebased against the current master. >> >> On 12/02/2015 05:10 PM, Martin Basti wrote: >>> >>> On 02.12.2015 16:18, Oleg Fayans wrote: >>>> Hi Martin, >>>> >>>> On 12/01/2015 04:08 PM, Martin Basti wrote: >>>>> >>>>> On 27.11.2015 16:26, Oleg Fayans wrote: >>>>>> And patch N 16 passes lint too: >>>>>> >>>>>> On 11/27/2015 04:03 PM, Oleg Fayans wrote: >>>>>>> Hi, >>>>>>> >>>>>>> On 11/27/2015 03:26 PM, Martin Basti wrote: >>>>>>>> >>>>>>>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>>>>>>> Hi Martin, >>>>>>>>> >>>>>>>>> All your suggestions were taken into account. Both patches are >>>>>>>>> updated. Thank you for your help! >>>>>>>>> >>>>>>>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>>>>>>> >>>>>>>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>>>>>>> Hi Martin, >>>>>>>>>>> >>>>>>>>>>> I agree to all your points but one. please, see my comment below >>>>>>>>>>> >>>>>>>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> 0) Note >>>>>>>>>>>> Please be aware of https://fedorahosted.org/freeipa/ticket/5469 >>>>>>>>>>>> during >>>>>>>>>>>> KRA testing >>>>>>>>>>>> >>>>>>>>>>>> 1) >>>>>>>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, this may >>>>>>>>>>>> change >>>>>>>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for domain >>>>>>>>>>>> level 0 >>>>>>>>>>>> and 1 >>>>>>>>>>>> >>>>>>>>>>>> 2) >>>>>>>>>>>> Why uninstall KRA then server, is not enough just uninstall >>>>>>>>>>>> server >>>>>>>>>>>> which >>>>>>>>>>>> covers KRA uninstall? >>>>>>>>>>>> >>>>>>>>>>>> + def teardown_method(self, method): >>>>>>>>>>>> + for host in self.replicas: >>>>>>>>>>>> + host.run_command(self.kra_uninstall, >>>>>>>>>>>> raiseonerr=False) >>>>>>>>>>>> + tasks.uninstall_master(host) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 3) >>>>>>>>>>>> Can be this function more generic? It should allow specify host >>>>>>>>>>>> where >>>>>>>>>>>> KRA should be installed not just master >>>>>>>>>>>> >>>>>>>>>>>> + def test_kra_install_master(self): >>>>>>>>>>>> + self.master.run_command(self.kra_install) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 4) >>>>>>>>>>>> >>>>>>>>>>>> TestLevel0(Dummy): >>>>>>>>>>>> Can be the test name more specific, something like >>>>>>>>>>>> TestReplicaPromotionLevel0 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 5) >>>>>>>>>>>> please remove this, the patch is on review and it will be pushed >>>>>>>>>>>> sooner >>>>>>>>>>>> than tests >>>>>>>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>>>>>>> >>>>>>>>>>>> and as I mentioned in ticket #5455, I cannot reproduce it with >>>>>>>>>>>> ipa-kra-install, so please provide steps to reproduce if you >>>>>>>>>>>> insist >>>>>>>>>>>> that >>>>>>>>>>>> this still does not work as expected with KRA. >>>>>>>>>>>> >>>>>>>>>>>> 6) This is completely wrong, it removes everything that we >>>>>>>>>>>> tried to >>>>>>>>>>>> achieve with previous patches with domain level in CI >>>>>>>>>>> Actually, being able to configure domain level per class is WAY >>>>>>>>>>> more >>>>>>>>>>> convenient, than to always have to think which domain level is >>>>>>>>>>> appropriate for which particular test during jenkins job >>>>>>>>>>> configuration. In fact, I should have thought about it from the >>>>>>>>>>> very >>>>>>>>>>> beginning. For example, in test_replica_promotion.py we have on >>>>>>>>>>> class, >>>>>>>>>>> which intiates with domain level = 1, while others - with domain >>>>>>>>>>> level >>>>>>>>>>> 0. With config-based approach, we would have to implement a >>>>>>>>>>> separate >>>>>>>>>>> step that raises domain level. Overall, I am against the approach, >>>>>>>>>>> when you have to remember to set certain domain level in config >>>>>>>>>>> for >>>>>>>>>>> any particular test. The tests themselves should be aware of the >>>>>>>>>>> domain level they need. >>>>>>>>>> I do not say that we should not have something that overrides >>>>>>>>>> settings >>>>>>>>>> in from config in a particular test case, I say your patch is >>>>>>>>>> doing it >>>>>>>>>> wrong. >>>>>>>>>> >>>>>>>>>> I agree it is useful to have param domain_level in install_master, >>>>>>>>>> and >>>>>>>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by default, >>>>>>>>>> because with your current patch the domain_level in config is not >>>>>>>>>> used >>>>>>>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>>>>>>> >>>>>>>>>> For example I want to achieve this goal: >>>>>>>>>> test_vault.py, this test suite can run on domain level1 and on >>>>>>>>>> domain >>>>>>>>>> level0, so with one test we can test 2 domain levels just with >>>>>>>>>> putting >>>>>>>>>> domain level into config file. >>>>>>>>>> >>>>>>>>>> I agree that with extraordinary test like replica promotion test >>>>>>>>>> is, we >>>>>>>>>> need something that allows override the config file. >>>>>>>>>> >>>>>>>>>> As I said bellow, domain_level default value should be None in >>>>>>>>>> install_master and install_topo plugin. If domain level was >>>>>>>>>> specified >>>>>>>>>> use the specified one, if not (value is None) use the domain level >>>>>>>>>> from >>>>>>>>>> config file. >>>>>>>>> Agreed :) >>>>>>>>> >>>>>>>>>> Martin >>>>>>>>>>>> [PATCH] Enabled setting domain_level per class derived from >>>>>>>>>>>> TestIntegration >>>>>>>>>>>> >>>>>>>>>>>> When I configure domain level 0 in yaml config, how is this >>>>>>>>>>>> supposed to >>>>>>>>>>>> get into install methods when you removed that code? >>>>>>>>>>>> >>>>>>>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>>>>>>> + "--domain-level=%i" % domain_level >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> You always use MAX_DOMAIN_LEVEL in this case or whatever is >>>>>>>>>>>> specified in >>>>>>>>>>>> domain_level option. >>>>>>>>>>>> I suggest to use domain_level=None, and when it is None use >>>>>>>>>>>> 'host.config.domain_level', if it is not None, use 'domain_level' >>>>>>>>>>>> >>>>>>>>>>>> With this we can specify domain level in config file for test >>>>>>>>>>>> that >>>>>>>>>>>> can >>>>>>>>>>>> be used for both domain levels and you can manually specify >>>>>>>>>>>> domain >>>>>>>>>>>> level >>>>>>>>>>>> for test that requires specific domain level. >>>>>>>>>>>> >>>>>>>>>>>> Also this should go away >>>>>>>>>>>> >>>>>>>>>>>> @classmethod >>>>>>>>>>>> def install(cls, mh): >>>>>>>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>>>>>>> if cls.topology is None: >>>>>>>>>>>> return >>>>>>>>>>>> >>>>>>>>>>>> I do not see reason why test should override configuration in >>>>>>>>>>>> config in >>>>>>>>>>>> this case. >>>>>>>>>>>> >>>>>>>>>>>> Martin >>>>>>>>>>>> >>>>>>>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Here is the updated version of the patch (more tests + fixed the >>>>>>>>>>>>> issues of the first one) + patch 0017, that implements the >>>>>>>>>>>>> necessary >>>>>>>>>>>>> changes in the background code, i. e. patch 16 does not work >>>>>>>>>>>>> without >>>>>>>>>>>>> patch 17 >>>>>>>>>>>>> >>>>>>>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>>>>>>> Hi guys, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Here are first two automated testcases from this (so far >>>>>>>>>>>>>>> incomplete) >>>>>>>>>>>>>>> testplan: >>>>>>>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> PATCH 16: NACK >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1) >>>>>>>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>>>>>>> 'domain_level' to >>>>>>>>>>>>>> install_topo()? >>>>>>>>>>>>>> Also it is good practise to add new option as the last >>>>>>>>>>>>>> parameter. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2) >>>>>>>>>>>>>> cab you in both tests specify a domain level with constant >>>>>>>>>>>>>> instead of >>>>>>>>>>>>>> number literal? >>>>>>>>>>>>>> >>>>>>>>>>>>>> 3) >>>>>>>>>>>>>> both test call install_topo with custom domain level, but it >>>>>>>>>>>>>> cannot >>>>>>>>>>>>>> work >>>>>>>>>>>>>> because 1) (did you run the test?) >>>>>>>>>>>>>> >>>>>>>>>>>>>> 4) >>>>>>>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>>>>>>> Respectively why there is call of install_topo() that installs >>>>>>>>>>>>>> replica. >>>>>>>>>>>>>> As this test just tests that ipa-replica-prepare is not working >>>>>>>>>>>>>> anymore, >>>>>>>>>>>>>> is it worth to spend 20 minutes with installing replica and >>>>>>>>>>>>>> then >>>>>>>>>>>>>> just no >>>>>>>>>>>>>> tot use it? IMO to install master in install step is enough. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Martin^2 >>>>>>>>>>>>>> >>>>>>>> ./make-lint >>>>>>>> ************* Module ipatests.test_integration.base >>>>>>>> ipatests/test_integration/base.py:66: [E1101(no-member), >>>>>>>> IntegrationTest.install] Class 'IntegrationTest' has no >>>>>>>> 'domain_level' >>>>>>>> member) >>>>>>>> ************* Module ipatests.test_integration.test_replica_promotion >>>>>>>> ipatests/test_integration/test_replica_promotion.py:16: >>>>>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no 'domain_level' >>>>>>>> member) >>>>>>>> ipatests/test_integration/test_replica_promotion.py:117: >>>>>>>> [E1101(no-member), >>>>>>>> TestCAInstall.test_ca_install_without_replica_file] >>>>>>>> Module 'ipatests.test_integration.tasks' has no 'setup_replica' >>>>>>>> member) >>>>>>>> >>>>>>>> >>>>>>>> Is it so hard to run pylint before patch is posted for review? >>>>>>> Sorry, my bad. Fixed. >>>>>>> >>>>>>> >>>>>>> >>>>> 1) >>>>> Why is this change in the patch? >>>>> - # Clean up the test directory >>>>> - host.run_command(['rm', '-rvf', host.config.test_dir]) >>>> Otherwise 2 out of 8 tests fail with IOError at line 78 of >>>> ipatests/test_integration/tasks.py >>>> >>>> I do not understand yet how does this happen, but if you remove >>>> ipatests folder once, it then fails to be created again. >>>> >>> So this should be in separated patch and investigated properly. >> Agree. Removed >>>>> - >>>>> >>>>> >>>>> 2) >>>>> is enough to have this check only in install_master, install_topo can >>>>> pass None to install_master >>>>> + if domain_level is None: >>>>> + domain_level = master.config.domain_level >>>> Done >>>> >>>>> >>>>> 3) >>>>> IMO replica-manage del should cleanup hosts entry, so following code >>>>> should not be needed. >>>>> + if cleanhost: >>>>> + kinit_admin(master) >>>>> + master.run_command(["ipa", "host-del", "--updatedns", >>>>> replica.hostname], >>>>> + raiseonerr=False) >>>> Well, in fact it does not. At least the corresponding dns record stays >>>> and causes the subsequent ipa-client-install to fail. Probably it's a >>>> bug. On the other hand, if I promote an existing client to replica and >>>> then delete this replica, then, I probably want the host record (that >>>> was created during client-install) to stay in the system. So, does not >>>> look like a bug to me. >>> No you don't, because replica uninstallation also removes the client. >>> >>> I tried it with ipa42, ipa-replica-manage del removes host entry, and >>> DNS A records, only SSHFP records stay there (I'm not sure if it is bug >>> or feature) >>> >>> Also I received this message >>> """ >>> Failed to cleanup replica1.ipa.test DNS entries: no matching entry found >>> You may need to manually remove them from the tree >>> """ >>> But, A record has been removed, so this is probably false positive and >>> it needs to have a ticket. >> Agree, that was an issue with my setup. >> Removed >> >>>>> 4) >>>>> This variable is not used >>>>> + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] >>>> Removed >>>> >>>>> 5) >>>>> Why do you need this >>>>> + kra_install = ["ipa-kra-install", "-U", "-p", >>>>> config.dirman_password] >>>>> when you implemented tasks.install_kra that returns the exactly the same >>>>> result? >>>> Right. Removed >>>> >>>>> 6) >>>>> PEP8 >>>>> ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 blank lines, >>>>> found 1 >>>>> ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 blank lines, >>>>> found 1 >>>>> ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 blank lines, >>>>> found 1 >>>>> ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 blank lines, >>>>> found 1 >>>>> ./ipatests/test_integration/tasks.py:950:80: E501 line too long (80 > 79 >>>>> characters) >>>>> >>>>> ./ipatests/test_integration/test_replica_promotion.py:29:80: E501 line >>>>> too long (85 > 79 characters) >>>>> ./ipatests/test_integration/test_replica_promotion.py:64:80: E501 line >>>>> too long (85 > 79 characters) >>>>> ./ipatests/test_integration/test_replica_promotion.py:67:80: E501 line >>>>> too long (88 > 79 characters) >>>>> ./ipatests/test_integration/test_replica_promotion.py:93:80: E501 line >>>>> too long (80 > 79 characters) >>>>> ./ipatests/test_integration/test_replica_promotion.py:94:80: E501 line >>>>> too long (83 > 79 characters) >>>>> ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 line >>>>> too long (81 > 79 characters) >>>>> ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 line >>>>> too long (80 > 79 characters) >>>>> ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 line >>>>> too long (82 > 79 characters) >>>>> ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 line >>>>> too long (80 > 79 characters) >>>> Most of these complaints are unrelated to the current patches. >>>> It's better to create a separate patch addressing PEP8 errors. >>>> >>> I beg for your pardon, those PEP8 errors have been introduced by your >>> patches. >> Fixed >> >>>>> 7) >>>>> Why this must be stored in instance? IMO to have it stored as local >>>>> variable is perfect >>>>> TestKRAInstall, TestCAInstall >>>>> self.replica1_filename = tasks.get_replica_filename(replica1) >>>>> self.replica2_filename = tasks.get_replica_filename(replica2) >>>> Agree. Fixed >>>> >> >> This patch is missing something. From abokovoy at redhat.com Wed Dec 9 09:34:40 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 9 Dec 2015 11:34:40 +0200 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <5667EE2C.2050100@redhat.com> References: <5667EE2C.2050100@redhat.com> Message-ID: <20151209093440.GG4620@redhat.com> On Wed, 09 Dec 2015, Jan Cholasta wrote: >From 142fd5364cf9d31d7e2c35959516ac8d9054c9da Mon Sep 17 00:00:00 2001 >From: Jan Cholasta >Date: Wed, 9 Dec 2015 08:17:07 +0100 >Subject: [PATCH 1/3] build: put oddjob scripts into separate directory > >https://fedorahosted.org/freeipa/ticket/5497 >--- > freeipa.spec.in | 3 ++- > install/oddjob/Makefile.am | 2 +- > install/oddjob/etc/oddjobd.conf.d/oddjobd-ipa-trust.conf | 2 +- > 3 files changed, 4 insertions(+), 3 deletions(-) > >diff --git a/freeipa.spec.in b/freeipa.spec.in >index a60d9b6..95948e7 100644 >--- a/freeipa.spec.in >+++ b/freeipa.spec.in >@@ -740,6 +740,7 @@ fi > %{_libexecdir}/ipa/ipa-dnskeysync-replica > %{_libexecdir}/ipa/ipa-ods-exporter > %{_libexecdir}/ipa/ipa-httpd-kdcproxy >+%dir %{_libexecdir}/ipa/oddjob > %ghost %verify(not owner group) %dir %{_sharedstatedir}/kdcproxy > %dir %attr(0755,root,root) %{_sysconfdir}/ipa/kdcproxy > %config(noreplace) %{_sysconfdir}/sysconfig/ipa_memcached >@@ -914,7 +915,7 @@ fi > %ghost %{_libdir}/krb5/plugins/libkrb5/winbind_krb5_locator.so > %{_sysconfdir}/dbus-1/system.d/oddjob-ipa-trust.conf > %{_sysconfdir}/oddjobd.conf.d/oddjobd-ipa-trust.conf >-%%attr(755,root,root) %{_libexecdir}/ipa/com.redhat.idm.trust-fetch-domains >+%attr(755,root,root) %{_libexecdir}/ipa/oddjob/com.redhat.idm.trust-fetch-domains As you modified oddjobd config file, you need to restart oddjobd on upgrade to let it re-read the config. > > %endif # ONLY_CLIENT > >diff --git a/install/oddjob/Makefile.am b/install/oddjob/Makefile.am >index 9dde10c..5cdaf2b 100644 >--- a/install/oddjob/Makefile.am >+++ b/install/oddjob/Makefile.am >@@ -1,6 +1,6 @@ > NULL = > >-oddjobdir = $(libexecdir)/ipa >+oddjobdir = $(libexecdir)/ipa/oddjob > oddjobconfdir = $(sysconfdir)/oddjobd.conf.d > dbusconfdir = $(sysconfdir)/dbus-1/system.d > >diff --git a/install/oddjob/etc/oddjobd.conf.d/oddjobd-ipa-trust.conf b/install/oddjob/etc/oddjobd.conf.d/oddjobd-ipa-trust.conf >index 17817de..bc2e8d1 100644 >--- a/install/oddjob/etc/oddjobd.conf.d/oddjobd-ipa-trust.conf >+++ b/install/oddjob/etc/oddjobd.conf.d/oddjobd-ipa-trust.conf >@@ -10,7 +10,7 @@ > > > >- + arguments="1" > argument_passing_method="cmdline" > prepend_user_name="no"/> >-- >2.4.3 > -- / Alexander Bokovoy From jcholast at redhat.com Wed Dec 9 09:38:45 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 9 Dec 2015 10:38:45 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <20151209093440.GG4620@redhat.com> References: <5667EE2C.2050100@redhat.com> <20151209093440.GG4620@redhat.com> Message-ID: <5667F6A5.7090903@redhat.com> On 9.12.2015 10:34, Alexander Bokovoy wrote: > On Wed, 09 Dec 2015, Jan Cholasta wrote: >> From 142fd5364cf9d31d7e2c35959516ac8d9054c9da Mon Sep 17 00:00:00 2001 >> From: Jan Cholasta >> Date: Wed, 9 Dec 2015 08:17:07 +0100 >> Subject: [PATCH 1/3] build: put oddjob scripts into separate directory >> >> https://fedorahosted.org/freeipa/ticket/5497 >> --- >> freeipa.spec.in | 3 ++- >> install/oddjob/Makefile.am | 2 +- >> install/oddjob/etc/oddjobd.conf.d/oddjobd-ipa-trust.conf | 2 +- >> 3 files changed, 4 insertions(+), 3 deletions(-) >> >> diff --git a/freeipa.spec.in b/freeipa.spec.in >> index a60d9b6..95948e7 100644 >> --- a/freeipa.spec.in >> +++ b/freeipa.spec.in >> @@ -740,6 +740,7 @@ fi >> %{_libexecdir}/ipa/ipa-dnskeysync-replica >> %{_libexecdir}/ipa/ipa-ods-exporter >> %{_libexecdir}/ipa/ipa-httpd-kdcproxy >> +%dir %{_libexecdir}/ipa/oddjob >> %ghost %verify(not owner group) %dir %{_sharedstatedir}/kdcproxy >> %dir %attr(0755,root,root) %{_sysconfdir}/ipa/kdcproxy >> %config(noreplace) %{_sysconfdir}/sysconfig/ipa_memcached >> @@ -914,7 +915,7 @@ fi >> %ghost %{_libdir}/krb5/plugins/libkrb5/winbind_krb5_locator.so >> %{_sysconfdir}/dbus-1/system.d/oddjob-ipa-trust.conf >> %{_sysconfdir}/oddjobd.conf.d/oddjobd-ipa-trust.conf >> -%%attr(755,root,root) >> %{_libexecdir}/ipa/com.redhat.idm.trust-fetch-domains >> +%attr(755,root,root) >> %{_libexecdir}/ipa/oddjob/com.redhat.idm.trust-fetch-domains > As you modified oddjobd config file, you need to restart oddjobd on > upgrade to let it re-read the config. Right, I have accidentally put that to %pre rather than %post. -- Jan Cholasta From mbasti at redhat.com Wed Dec 9 09:37:43 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 9 Dec 2015 10:37:43 +0100 Subject: [Freeipa-devel] [PATCH 0065] ipa-replica-install prints incorrect error message when replica is already installed In-Reply-To: <5667C566.5000906@redhat.com> References: <5666F2B8.1090803@redhat.com> <5666F3BB.8030006@redhat.com> <5666F497.3040500@redhat.com> <5666F539.1000904@redhat.com> <5666F6F0.2050005@redhat.com> <5667C566.5000906@redhat.com> Message-ID: <5667F667.9040603@redhat.com> NACK Patch contains syntax error, missing brace ipaserver/install/server/replicainstall.py:850: [E0001(syntax-error), ] invalid syntax) Martin On 09.12.2015 07:08, Jan Cholasta wrote: > LGTM > > On 8.12.2015 17:04, Gabe Alford wrote: >> Updated patch attached. >> >> On Tue, Dec 8, 2015 at 8:27 AM, Martin Basti > > wrote: >> >> >> >> On 08.12.2015 16:26, Gabe Alford wrote: >>> Just to confirm: >>> >>> if server is installed: >>> Let's stop here and not do anything else >>> >>> if domain level 0: >>> check if client installed and stop here >>> >>> Right? >> yes >> >> >>> >>> >>> On Tue, Dec 8, 2015 at 8:20 AM, Jan Cholasta >> > wrote: >>> >>> On 8.12.2015 16:17, Martin Basti wrote: >>> >>> >>> >>> On 08.12.2015 16:14, Jan Cholasta wrote: >>> >>> On 8.12.2015 16:09, Martin Basti wrote: >>> >>> >>> >>> On 01.12.2015 14:57, Gabe Alford wrote: >>> >>> Sorry guys, I forgot to add a meaningful >>> subject to this message. >>> Ignore the previous thread start. >>> >>> ---------- Forwarded message ---------- >>> From: *Gabe Alford* >> >>> >> >> >>> Date: Mon, Nov 30, 2015 at 7:31 PM >>> Subject: [PATCH 0065] >>> To: freeipa-devel >> >>> >> >> >>> >>> >>> Hello, >>> >>> Patch fix for the following tickets: >>> >>> https://fedorahosted.org/freeipa/ticket/5022 >>> https://fedorahosted.org/freeipa/ticket/5320 >>> >>> Thanks, >>> >>> Gabe >>> >>> >>> >>> ACK >>> >>> >>> NACK, you can't install a server over an already >>> installed client, >>> thus the original check is correct. >>> >>> Ahh domain level 0, right, but this check can be added >>> before the client >>> check. >>> >>> >>> Yes. >>> >>> With domain level 1, this check should stay there IMO. >>> >>> >>> Yes. It should say "IPA server is already configured" rather >>> than "IPA replica is already configured", though. >>> >>> -- >>> Jan Cholasta >>> >>> >> >> > > From ofayans at redhat.com Wed Dec 9 10:14:19 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Wed, 9 Dec 2015 11:14:19 +0100 Subject: [Freeipa-devel] [PATCH] First part of the replica promotion tests + testplan In-Reply-To: <5667F49B.5020709@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> <565DB7D3.5040500@redhat.com> <565F0BDD.8080206@redhat.com> <565F17EE.3020701@redhat.com> <5666DC18.2040906@redhat.com> <56675E31.3050203@redhat.com> <5667F49B.5020709@redhat.com> Message-ID: <5667FEFB.6090904@redhat.com> Hi Martin On 12/09/2015 10:30 AM, Martin Basti wrote: > > > On 08.12.2015 23:48, Oleg Fayans wrote: >> Substituted a hardcoded suffix name with a constant DOMAIN_SUFFIX_NAME >> >> On 12/08/2015 02:33 PM, Oleg Fayans wrote: >>> Hi all, >>> >>> >>> The patches are rebased against the current master. >>> >>> On 12/02/2015 05:10 PM, Martin Basti wrote: >>>> >>>> On 02.12.2015 16:18, Oleg Fayans wrote: >>>>> Hi Martin, >>>>> >>>>> On 12/01/2015 04:08 PM, Martin Basti wrote: >>>>>> >>>>>> On 27.11.2015 16:26, Oleg Fayans wrote: >>>>>>> And patch N 16 passes lint too: >>>>>>> >>>>>>> On 11/27/2015 04:03 PM, Oleg Fayans wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> On 11/27/2015 03:26 PM, Martin Basti wrote: >>>>>>>>> >>>>>>>>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>>>>>>>> Hi Martin, >>>>>>>>>> >>>>>>>>>> All your suggestions were taken into account. Both patches are >>>>>>>>>> updated. Thank you for your help! >>>>>>>>>> >>>>>>>>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>>>>>>>> >>>>>>>>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>>>>>>>> Hi Martin, >>>>>>>>>>>> >>>>>>>>>>>> I agree to all your points but one. please, see my comment >>>>>>>>>>>> below >>>>>>>>>>>> >>>>>>>>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> 0) Note >>>>>>>>>>>>> Please be aware of >>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5469 >>>>>>>>>>>>> during >>>>>>>>>>>>> KRA testing >>>>>>>>>>>>> >>>>>>>>>>>>> 1) >>>>>>>>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, this may >>>>>>>>>>>>> change >>>>>>>>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for domain >>>>>>>>>>>>> level 0 >>>>>>>>>>>>> and 1 >>>>>>>>>>>>> >>>>>>>>>>>>> 2) >>>>>>>>>>>>> Why uninstall KRA then server, is not enough just uninstall >>>>>>>>>>>>> server >>>>>>>>>>>>> which >>>>>>>>>>>>> covers KRA uninstall? >>>>>>>>>>>>> >>>>>>>>>>>>> + def teardown_method(self, method): >>>>>>>>>>>>> + for host in self.replicas: >>>>>>>>>>>>> + host.run_command(self.kra_uninstall, >>>>>>>>>>>>> raiseonerr=False) >>>>>>>>>>>>> + tasks.uninstall_master(host) >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> 3) >>>>>>>>>>>>> Can be this function more generic? It should allow specify >>>>>>>>>>>>> host >>>>>>>>>>>>> where >>>>>>>>>>>>> KRA should be installed not just master >>>>>>>>>>>>> >>>>>>>>>>>>> + def test_kra_install_master(self): >>>>>>>>>>>>> + self.master.run_command(self.kra_install) >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> 4) >>>>>>>>>>>>> >>>>>>>>>>>>> TestLevel0(Dummy): >>>>>>>>>>>>> Can be the test name more specific, something like >>>>>>>>>>>>> TestReplicaPromotionLevel0 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> 5) >>>>>>>>>>>>> please remove this, the patch is on review and it will be >>>>>>>>>>>>> pushed >>>>>>>>>>>>> sooner >>>>>>>>>>>>> than tests >>>>>>>>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>>>>>>>> >>>>>>>>>>>>> and as I mentioned in ticket #5455, I cannot reproduce it with >>>>>>>>>>>>> ipa-kra-install, so please provide steps to reproduce if you >>>>>>>>>>>>> insist >>>>>>>>>>>>> that >>>>>>>>>>>>> this still does not work as expected with KRA. >>>>>>>>>>>>> >>>>>>>>>>>>> 6) This is completely wrong, it removes everything that we >>>>>>>>>>>>> tried to >>>>>>>>>>>>> achieve with previous patches with domain level in CI >>>>>>>>>>>> Actually, being able to configure domain level per class is WAY >>>>>>>>>>>> more >>>>>>>>>>>> convenient, than to always have to think which domain level is >>>>>>>>>>>> appropriate for which particular test during jenkins job >>>>>>>>>>>> configuration. In fact, I should have thought about it from the >>>>>>>>>>>> very >>>>>>>>>>>> beginning. For example, in test_replica_promotion.py we have on >>>>>>>>>>>> class, >>>>>>>>>>>> which intiates with domain level = 1, while others - with >>>>>>>>>>>> domain >>>>>>>>>>>> level >>>>>>>>>>>> 0. With config-based approach, we would have to implement a >>>>>>>>>>>> separate >>>>>>>>>>>> step that raises domain level. Overall, I am against the >>>>>>>>>>>> approach, >>>>>>>>>>>> when you have to remember to set certain domain level in config >>>>>>>>>>>> for >>>>>>>>>>>> any particular test. The tests themselves should be aware of >>>>>>>>>>>> the >>>>>>>>>>>> domain level they need. >>>>>>>>>>> I do not say that we should not have something that overrides >>>>>>>>>>> settings >>>>>>>>>>> in from config in a particular test case, I say your patch is >>>>>>>>>>> doing it >>>>>>>>>>> wrong. >>>>>>>>>>> >>>>>>>>>>> I agree it is useful to have param domain_level in >>>>>>>>>>> install_master, >>>>>>>>>>> and >>>>>>>>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by >>>>>>>>>>> default, >>>>>>>>>>> because with your current patch the domain_level in config is >>>>>>>>>>> not >>>>>>>>>>> used >>>>>>>>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>>>>>>>> >>>>>>>>>>> For example I want to achieve this goal: >>>>>>>>>>> test_vault.py, this test suite can run on domain level1 and on >>>>>>>>>>> domain >>>>>>>>>>> level0, so with one test we can test 2 domain levels just with >>>>>>>>>>> putting >>>>>>>>>>> domain level into config file. >>>>>>>>>>> >>>>>>>>>>> I agree that with extraordinary test like replica promotion test >>>>>>>>>>> is, we >>>>>>>>>>> need something that allows override the config file. >>>>>>>>>>> >>>>>>>>>>> As I said bellow, domain_level default value should be None in >>>>>>>>>>> install_master and install_topo plugin. If domain level was >>>>>>>>>>> specified >>>>>>>>>>> use the specified one, if not (value is None) use the domain >>>>>>>>>>> level >>>>>>>>>>> from >>>>>>>>>>> config file. >>>>>>>>>> Agreed :) >>>>>>>>>> >>>>>>>>>>> Martin >>>>>>>>>>>>> [PATCH] Enabled setting domain_level per class derived from >>>>>>>>>>>>> TestIntegration >>>>>>>>>>>>> >>>>>>>>>>>>> When I configure domain level 0 in yaml config, how is this >>>>>>>>>>>>> supposed to >>>>>>>>>>>>> get into install methods when you removed that code? >>>>>>>>>>>>> >>>>>>>>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>>>>>>>> + "--domain-level=%i" % domain_level >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> You always use MAX_DOMAIN_LEVEL in this case or whatever is >>>>>>>>>>>>> specified in >>>>>>>>>>>>> domain_level option. >>>>>>>>>>>>> I suggest to use domain_level=None, and when it is None use >>>>>>>>>>>>> 'host.config.domain_level', if it is not None, use >>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>> >>>>>>>>>>>>> With this we can specify domain level in config file for test >>>>>>>>>>>>> that >>>>>>>>>>>>> can >>>>>>>>>>>>> be used for both domain levels and you can manually specify >>>>>>>>>>>>> domain >>>>>>>>>>>>> level >>>>>>>>>>>>> for test that requires specific domain level. >>>>>>>>>>>>> >>>>>>>>>>>>> Also this should go away >>>>>>>>>>>>> >>>>>>>>>>>>> @classmethod >>>>>>>>>>>>> def install(cls, mh): >>>>>>>>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>>>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>>>>>>>> if cls.topology is None: >>>>>>>>>>>>> return >>>>>>>>>>>>> >>>>>>>>>>>>> I do not see reason why test should override configuration in >>>>>>>>>>>>> config in >>>>>>>>>>>>> this case. >>>>>>>>>>>>> >>>>>>>>>>>>> Martin >>>>>>>>>>>>> >>>>>>>>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Here is the updated version of the patch (more tests + >>>>>>>>>>>>>> fixed the >>>>>>>>>>>>>> issues of the first one) + patch 0017, that implements the >>>>>>>>>>>>>> necessary >>>>>>>>>>>>>> changes in the background code, i. e. patch 16 does not work >>>>>>>>>>>>>> without >>>>>>>>>>>>>> patch 17 >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>>>>>>>> Hi guys, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Here are first two automated testcases from this (so far >>>>>>>>>>>>>>>> incomplete) >>>>>>>>>>>>>>>> testplan: >>>>>>>>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> PATCH 16: NACK >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>>>>>>>> 'domain_level' to >>>>>>>>>>>>>>> install_topo()? >>>>>>>>>>>>>>> Also it is good practise to add new option as the last >>>>>>>>>>>>>>> parameter. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>> cab you in both tests specify a domain level with constant >>>>>>>>>>>>>>> instead of >>>>>>>>>>>>>>> number literal? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>> both test call install_topo with custom domain level, but it >>>>>>>>>>>>>>> cannot >>>>>>>>>>>>>>> work >>>>>>>>>>>>>>> because 1) (did you run the test?) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>>>>>>>> Respectively why there is call of install_topo() that >>>>>>>>>>>>>>> installs >>>>>>>>>>>>>>> replica. >>>>>>>>>>>>>>> As this test just tests that ipa-replica-prepare is not >>>>>>>>>>>>>>> working >>>>>>>>>>>>>>> anymore, >>>>>>>>>>>>>>> is it worth to spend 20 minutes with installing replica and >>>>>>>>>>>>>>> then >>>>>>>>>>>>>>> just no >>>>>>>>>>>>>>> tot use it? IMO to install master in install step is enough. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Martin^2 >>>>>>>>>>>>>>> >>>>>>>>> ./make-lint >>>>>>>>> ************* Module ipatests.test_integration.base >>>>>>>>> ipatests/test_integration/base.py:66: [E1101(no-member), >>>>>>>>> IntegrationTest.install] Class 'IntegrationTest' has no >>>>>>>>> 'domain_level' >>>>>>>>> member) >>>>>>>>> ************* Module >>>>>>>>> ipatests.test_integration.test_replica_promotion >>>>>>>>> ipatests/test_integration/test_replica_promotion.py:16: >>>>>>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no >>>>>>>>> 'domain_level' >>>>>>>>> member) >>>>>>>>> ipatests/test_integration/test_replica_promotion.py:117: >>>>>>>>> [E1101(no-member), >>>>>>>>> TestCAInstall.test_ca_install_without_replica_file] >>>>>>>>> Module 'ipatests.test_integration.tasks' has no 'setup_replica' >>>>>>>>> member) >>>>>>>>> >>>>>>>>> >>>>>>>>> Is it so hard to run pylint before patch is posted for review? >>>>>>>> Sorry, my bad. Fixed. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> 1) >>>>>> Why is this change in the patch? >>>>>> - # Clean up the test directory >>>>>> - host.run_command(['rm', '-rvf', host.config.test_dir]) >>>>> Otherwise 2 out of 8 tests fail with IOError at line 78 of >>>>> ipatests/test_integration/tasks.py >>>>> >>>>> I do not understand yet how does this happen, but if you remove >>>>> ipatests folder once, it then fails to be created again. >>>>> >>>> So this should be in separated patch and investigated properly. >>> Agree. Removed >>>>>> - >>>>>> >>>>>> >>>>>> 2) >>>>>> is enough to have this check only in install_master, install_topo can >>>>>> pass None to install_master >>>>>> + if domain_level is None: >>>>>> + domain_level = master.config.domain_level >>>>> Done >>>>> >>>>>> >>>>>> 3) >>>>>> IMO replica-manage del should cleanup hosts entry, so following code >>>>>> should not be needed. >>>>>> + if cleanhost: >>>>>> + kinit_admin(master) >>>>>> + master.run_command(["ipa", "host-del", "--updatedns", >>>>>> replica.hostname], >>>>>> + raiseonerr=False) >>>>> Well, in fact it does not. At least the corresponding dns record stays >>>>> and causes the subsequent ipa-client-install to fail. Probably it's a >>>>> bug. On the other hand, if I promote an existing client to replica and >>>>> then delete this replica, then, I probably want the host record (that >>>>> was created during client-install) to stay in the system. So, does not >>>>> look like a bug to me. >>>> No you don't, because replica uninstallation also removes the client. >>>> >>>> I tried it with ipa42, ipa-replica-manage del removes host entry, and >>>> DNS A records, only SSHFP records stay there (I'm not sure if it is bug >>>> or feature) >>>> >>>> Also I received this message >>>> """ >>>> Failed to cleanup replica1.ipa.test DNS entries: no matching entry >>>> found >>>> You may need to manually remove them from the tree >>>> """ >>>> But, A record has been removed, so this is probably false positive and >>>> it needs to have a ticket. >>> Agree, that was an issue with my setup. >>> Removed >>> >>>>>> 4) >>>>>> This variable is not used >>>>>> + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] >>>>> Removed >>>>> >>>>>> 5) >>>>>> Why do you need this >>>>>> + kra_install = ["ipa-kra-install", "-U", "-p", >>>>>> config.dirman_password] >>>>>> when you implemented tasks.install_kra that returns the exactly >>>>>> the same >>>>>> result? >>>>> Right. Removed >>>>> >>>>>> 6) >>>>>> PEP8 >>>>>> ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 blank >>>>>> lines, >>>>>> found 1 >>>>>> ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 blank >>>>>> lines, >>>>>> found 1 >>>>>> ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 blank >>>>>> lines, >>>>>> found 1 >>>>>> ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 blank >>>>>> lines, >>>>>> found 1 >>>>>> ./ipatests/test_integration/tasks.py:950:80: E501 line too long >>>>>> (80 > 79 >>>>>> characters) >>>>>> >>>>>> ./ipatests/test_integration/test_replica_promotion.py:29:80: E501 >>>>>> line >>>>>> too long (85 > 79 characters) >>>>>> ./ipatests/test_integration/test_replica_promotion.py:64:80: E501 >>>>>> line >>>>>> too long (85 > 79 characters) >>>>>> ./ipatests/test_integration/test_replica_promotion.py:67:80: E501 >>>>>> line >>>>>> too long (88 > 79 characters) >>>>>> ./ipatests/test_integration/test_replica_promotion.py:93:80: E501 >>>>>> line >>>>>> too long (80 > 79 characters) >>>>>> ./ipatests/test_integration/test_replica_promotion.py:94:80: E501 >>>>>> line >>>>>> too long (83 > 79 characters) >>>>>> ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 >>>>>> line >>>>>> too long (81 > 79 characters) >>>>>> ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 >>>>>> line >>>>>> too long (80 > 79 characters) >>>>>> ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 >>>>>> line >>>>>> too long (82 > 79 characters) >>>>>> ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 >>>>>> line >>>>>> too long (80 > 79 characters) >>>>> Most of these complaints are unrelated to the current patches. >>>>> It's better to create a separate patch addressing PEP8 errors. >>>>> >>>> I beg for your pardon, those PEP8 errors have been introduced by your >>>> patches. >>> Fixed >>> >>>>>> 7) >>>>>> Why this must be stored in instance? IMO to have it stored as local >>>>>> variable is perfect >>>>>> TestKRAInstall, TestCAInstall >>>>>> self.replica1_filename = >>>>>> tasks.get_replica_filename(replica1) >>>>>> self.replica2_filename = >>>>>> tasks.get_replica_filename(replica2) >>>>> Agree. Fixed >>>>> >>> >>> > This patch is missing something. I am sorry, I forgot to revert my previous change. The correct patch is attached -- Oleg Fayans Quality Engineer FreeIPA team RedHat. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0016.7-First-part-of-replica-promotion-tests.patch Type: text/x-patch Size: 10166 bytes Desc: not available URL: From ldoudova at redhat.com Wed Dec 9 10:29:34 2015 From: ldoudova at redhat.com (Lenka Doudova) Date: Wed, 9 Dec 2015 11:29:34 +0100 Subject: [Freeipa-devel] [TESTS][PATCH 0006] Add comments to stageuser plugin tests In-Reply-To: <5667F0A6.1090104@redhat.com> References: <564C7A23.5010500@redhat.com> <564C9E77.4050406@redhat.com> <564D88AF.8010500@redhat.com> <564D97AC.6050901@redhat.com> <564F004A.2060806@redhat.com> <5667E942.7030305@redhat.com> <5667F0A6.1090104@redhat.com> Message-ID: <5668028E.60902@redhat.com> On 12/09/2015 10:13 AM, Martin Basti wrote: > > > On 09.12.2015 09:41, Lenka Doudova wrote: >> Hi, >> >> attaching fixed patches for master and ipa-4-2 branch. >> Also fixes test accordingly to >> https://fedorahosted.org/freeipa/ticket/5387. >> >> Lenka >> >> On 11/20/2015 12:13 PM, Martin Babinsky wrote: >>> On 11/19/2015 10:34 AM, Petr Viktorin wrote: >>>> On 11/19/2015 09:30 AM, Lenka Doudova wrote: >>>>> On 11/18/2015 04:51 PM, Martin Babinsky wrote: >>>>>> On 11/18/2015 02:16 PM, Lenka Doudova wrote: >>>>>>> Hi, >>>>>>> >>>>>>> here's a patch that adds a few comments to stageuser tests in >>>>>>> order to >>>>>>> allow easier determining of a problem when tests fail. >>>>>>> >>>>>>> Lenka >>>>>>> >>>>>>> >>>>>> >>>>>> Hi Lenka, >>>>>> >>>>>> Firstly a technical detail: Python indexes lists from 0, so the >>>>>> comments in 'options_ok' do not correctly map to the test names >>>>>> anyway. >>>>>> >>>>>> I am also not sure if this patch is worth reviewing and pushing >>>>>> as it >>>>>> IMHO doesn't help in the identification of failed tests at all. >>>>>> >>>>>> This should be solved at more fundamental level. >>>>>> >>>>> Ouch, good point, I didn't realize. Sorry. >>>>> >>>>> Anyway, the issue is that even if tests are run in verbose mode, >>>>> you get >>>>> output like this: >>>>> >>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser27] >>>>> >>>>> PASSED >>>>> >>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser28] >>>>> >>>>> PASSED >>>>> >>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser29] >>>>> >>>>> PASSED >>>>> >>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser210] >>>>> >>>>> PASSED >>>>> >>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser211] >>>>> >>>>> PASSED >>>>> >>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser212] >>>>> >>>>> PASSED >>>>> >>>>> >>>>> If some test fails, you can't really tell which command was the one >>>>> responsible for the fail. This should be solved by >>>>> https://fedorahosted.org/freeipa/ticket/5449. Until that's done, >>>>> though, >>>>> the only way to find out which command failed is looking at the >>>>> code and >>>>> finding which parameters were put into the command, which was not >>>>> much >>>>> possible without better commenting the test case (as I realized last >>>>> week when David Kupka asked me to help him find the reason for failed >>>>> tests). >>>>> Obviously I can rewrite the tests so there's 27 separate test >>>>> cases, one >>>>> for each parameter, instead of one method that 'unwraps' into 27 test >>>>> cases, which would entirely eliminate the confusion. So far I >>>>> don't know >>>>> of a way to put 27 similar test cases in one method which would allow >>>>> easy recognition of the test cases. >>>>> I'll wait with fixing the patch until further discussion. >>>> >>>> Hello, >>>> Pytest wants you to be a bit more explicit about how to name the >>>> parameters. (It "hides" dicts by default, because large dicts would >>>> make >>>> the output even more confusing than the numbers.) >>>> >>>> Please try the attached patch. >>>> Docs are at >>>> https://pytest.org/latest/fixture.html#parametrizing-a-fixture >>>> >>>> >>>> >>> This looks like a better approach IMHO, you can then see which >>> attribute/value was being checked. >>> >>> I would very much favor more descriptive test/fixture names in the >>> beginning. >>> >> >> >> > Hello, > > we use usually bottom posting on freeipa-devel please try to keep > reply in this way. > > Patch: > > I do not like the idea of separated lists, IMO it is hard to manage > and is easy to create mistakes. > > How about this (untested, just from top of my head): > http://fpaste.org/298994/65184014/ > > Martin Great idea, thanks. Fixed patches attached. Lenka -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ldoudova-0006.3-Adding-descriptive-IDs-to-stageuser-tests.patch Type: text/x-patch Size: 8188 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ldoudova-0006.3-ipa42-Adding-descriptive-IDs-to-stageuser-tests.patch Type: text/x-patch Size: 7329 bytes Desc: not available URL: From pviktori at redhat.com Wed Dec 9 11:04:57 2015 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 9 Dec 2015 12:04:57 +0100 Subject: [Freeipa-devel] [PATCH] 0748 Handle encoding for ipautil.run In-Reply-To: <56617DBB.9000109@redhat.com> References: <5638B8F9.6010703@redhat.com> <564D0FD9.4070505@redhat.com> <5652B5AC.2040508@redhat.com> <5652E174.50301@redhat.com> <56548E73.1090105@redhat.com> <565587AE.2070303@redhat.com> <5655CA08.8090701@redhat.com> <565C01FD.7050906@redhat.com> <565D83CC.8020802@redhat.com> <565EA4FA.3020307@redhat.com> <565EF7B9.6020507@redhat.com> <566011A5.4040200@redhat.com> <566054D7.9020008@redhat.com> <566145FE.3090005@redhat.com> <566162B2.9010809@redhat.com> <566175D0.1000105@redhat.com> <56617C48.4030005@redhat.com> <56617DBB.9000109@redhat.com> Message-ID: <56680AD9.7060703@redhat.com> On 12/04/2015 12:49 PM, Jan Cholasta wrote: > On 4.12.2015 12:43, Petr Viktorin wrote: >> On 12/04/2015 12:15 PM, Jan Cholasta wrote: >>> On 4.12.2015 10:53, Petr Viktorin wrote: >>>> On 12/04/2015 08:51 AM, Jan Cholasta wrote: >>>>> On 3.12.2015 15:42, Petr Viktorin wrote: >>>>>> On 12/03/2015 10:55 AM, Jan Cholasta wrote: >> [...] >>>>>>> If we do encode/decode in run(), I think there should be a way to >>>>>>> override the default encoding. My idea was to either accept/return >>>>>>> only >>>>>>> bytes and let the caller handle encoding themselves, or to handle >>>>>>> encoding in run(), but be able to override the defaults. >>>>>>> >>>>>>> Given we handle encoding in run(), I would imagine it like this: >>>>>>> >>>>>>> def run(args, stdin=None, raiseonerr=True, nolog=(), >>>>>>> env=None, >>>>>>> capture_stdout=False, skip_output=False, cwd=None, >>>>>>> runas=None, timeout=None, suplementary_groups=[], >>>>>>> capture_stderr=False, encode_stdout=False, >>>>>>> encode_stderr=False, encoding=None): >>>>>>> >>>>>>> i.e. nothing is captured by default, when stdout or stderr are >>>>>>> captured >>>>>>> they are returned as bytes by default, when stdout or stderr are >>>>>>> returned as text they are decoded using the locale encoding by >>>>>>> default, >>>>>>> or the encoding can be explicitly specified. >>>>>>> >>>>>>> Do you think this is feasible? >>>>>> >>>>>> Feasible, yes. >>>>>> In the majority of cases where the output is needed, we want text >>>>>> encoded with locale.getpreferredencoding(), so I'd rather make >>>>>> that the >>>>>> default rather than bytes. >>>>>> >>>>>> Or we could make "encode_stdout" imply "capture_stdout", so you >>>>>> wouldn't >>>>>> have to always specify both. (It would be better named >>>>>> "encoded_stdout" >>>>>> in that case.) >>>>> >>>>> I think it should be OK to decode by default. (IMO it would be best to >>>>> reverse the logic and name it "raw_stdout", with False default. Do we >>>>> need "raw_stderr" at all? I don't think we do.) >>>> >>>> Actually, now that I think about it: if the result was a namedtuple >>>> subclass, we can always set extra raw_stdout/raw_stderr attributes on >>>> it. (Unless skip_output=True, of course.) >>>> >>>> result = run(['generate_binary_data']) >>>> use(result.raw_stdout) >>>> >>>> # and for backcompat, >>>> rc, _out, _err = result >>> >>> Good idea. >>> >>> Perhaps we should use the same names as CalledProcessError for the >>> attributes, i.e. "returncode", "output", and "error_output", >>> "raw_output", "raw_error_output" for the new attributes? > > (Actually, since "returncode" is not "return_code", it should probably > be "returncode", "output", "erroroutput", "raw_output", "raw_erroroutput".) > >> >> OK. It's also good to use different names than Popen's stdout/stderr, >> which are file-like objects rather than strings. But then, >> capture_stdout/capture_stderr should be >> capture_output/capture_error_output. Here is the new patch. I also added output_log and error_log to the result to abstract the following common pattern: result = run([...]) if result.returncode != 0: stderr = result.error_output if six.PY3: error = stderr.encode(locale.getpreferredencoding(), 'replace') self.log.critical('...: %s', stderr) -- Petr Viktorin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0748.4-Refactor-ipautil.run.patch Type: text/x-patch Size: 71404 bytes Desc: not available URL: From mbasti at redhat.com Wed Dec 9 11:10:38 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 9 Dec 2015 12:10:38 +0100 Subject: [Freeipa-devel] [PATCH] First part of the replica promotion tests + testplan In-Reply-To: <5667FEFB.6090904@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> <565DB7D3.5040500@redhat.com> <565F0BDD.8080206@redhat.com> <565F17EE.3020701@redhat.com> <5666DC18.2040906@redhat.com> <56675E31.3050203@redhat.com> <5667F49B.5020709@redhat.com> <5667FEFB.6090904@redhat.com> Message-ID: <56680C2E.7060805@redhat.com> On 09.12.2015 11:14, Oleg Fayans wrote: > Hi Martin > > On 12/09/2015 10:30 AM, Martin Basti wrote: >> >> On 08.12.2015 23:48, Oleg Fayans wrote: >>> Substituted a hardcoded suffix name with a constant DOMAIN_SUFFIX_NAME >>> >>> On 12/08/2015 02:33 PM, Oleg Fayans wrote: >>>> Hi all, >>>> >>>> >>>> The patches are rebased against the current master. >>>> >>>> On 12/02/2015 05:10 PM, Martin Basti wrote: >>>>> On 02.12.2015 16:18, Oleg Fayans wrote: >>>>>> Hi Martin, >>>>>> >>>>>> On 12/01/2015 04:08 PM, Martin Basti wrote: >>>>>>> On 27.11.2015 16:26, Oleg Fayans wrote: >>>>>>>> And patch N 16 passes lint too: >>>>>>>> >>>>>>>> On 11/27/2015 04:03 PM, Oleg Fayans wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> On 11/27/2015 03:26 PM, Martin Basti wrote: >>>>>>>>>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>>>>>>>>> Hi Martin, >>>>>>>>>>> >>>>>>>>>>> All your suggestions were taken into account. Both patches are >>>>>>>>>>> updated. Thank you for your help! >>>>>>>>>>> >>>>>>>>>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>>>>>>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>> >>>>>>>>>>>>> I agree to all your points but one. please, see my comment >>>>>>>>>>>>> below >>>>>>>>>>>>> >>>>>>>>>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> 0) Note >>>>>>>>>>>>>> Please be aware of >>>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5469 >>>>>>>>>>>>>> during >>>>>>>>>>>>>> KRA testing >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1) >>>>>>>>>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, this may >>>>>>>>>>>>>> change >>>>>>>>>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for domain >>>>>>>>>>>>>> level 0 >>>>>>>>>>>>>> and 1 >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2) >>>>>>>>>>>>>> Why uninstall KRA then server, is not enough just uninstall >>>>>>>>>>>>>> server >>>>>>>>>>>>>> which >>>>>>>>>>>>>> covers KRA uninstall? >>>>>>>>>>>>>> >>>>>>>>>>>>>> + def teardown_method(self, method): >>>>>>>>>>>>>> + for host in self.replicas: >>>>>>>>>>>>>> + host.run_command(self.kra_uninstall, >>>>>>>>>>>>>> raiseonerr=False) >>>>>>>>>>>>>> + tasks.uninstall_master(host) >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> 3) >>>>>>>>>>>>>> Can be this function more generic? It should allow specify >>>>>>>>>>>>>> host >>>>>>>>>>>>>> where >>>>>>>>>>>>>> KRA should be installed not just master >>>>>>>>>>>>>> >>>>>>>>>>>>>> + def test_kra_install_master(self): >>>>>>>>>>>>>> + self.master.run_command(self.kra_install) >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> 4) >>>>>>>>>>>>>> >>>>>>>>>>>>>> TestLevel0(Dummy): >>>>>>>>>>>>>> Can be the test name more specific, something like >>>>>>>>>>>>>> TestReplicaPromotionLevel0 >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> 5) >>>>>>>>>>>>>> please remove this, the patch is on review and it will be >>>>>>>>>>>>>> pushed >>>>>>>>>>>>>> sooner >>>>>>>>>>>>>> than tests >>>>>>>>>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>>>>>>>>> >>>>>>>>>>>>>> and as I mentioned in ticket #5455, I cannot reproduce it with >>>>>>>>>>>>>> ipa-kra-install, so please provide steps to reproduce if you >>>>>>>>>>>>>> insist >>>>>>>>>>>>>> that >>>>>>>>>>>>>> this still does not work as expected with KRA. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 6) This is completely wrong, it removes everything that we >>>>>>>>>>>>>> tried to >>>>>>>>>>>>>> achieve with previous patches with domain level in CI >>>>>>>>>>>>> Actually, being able to configure domain level per class is WAY >>>>>>>>>>>>> more >>>>>>>>>>>>> convenient, than to always have to think which domain level is >>>>>>>>>>>>> appropriate for which particular test during jenkins job >>>>>>>>>>>>> configuration. In fact, I should have thought about it from the >>>>>>>>>>>>> very >>>>>>>>>>>>> beginning. For example, in test_replica_promotion.py we have on >>>>>>>>>>>>> class, >>>>>>>>>>>>> which intiates with domain level = 1, while others - with >>>>>>>>>>>>> domain >>>>>>>>>>>>> level >>>>>>>>>>>>> 0. With config-based approach, we would have to implement a >>>>>>>>>>>>> separate >>>>>>>>>>>>> step that raises domain level. Overall, I am against the >>>>>>>>>>>>> approach, >>>>>>>>>>>>> when you have to remember to set certain domain level in config >>>>>>>>>>>>> for >>>>>>>>>>>>> any particular test. The tests themselves should be aware of >>>>>>>>>>>>> the >>>>>>>>>>>>> domain level they need. >>>>>>>>>>>> I do not say that we should not have something that overrides >>>>>>>>>>>> settings >>>>>>>>>>>> in from config in a particular test case, I say your patch is >>>>>>>>>>>> doing it >>>>>>>>>>>> wrong. >>>>>>>>>>>> >>>>>>>>>>>> I agree it is useful to have param domain_level in >>>>>>>>>>>> install_master, >>>>>>>>>>>> and >>>>>>>>>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by >>>>>>>>>>>> default, >>>>>>>>>>>> because with your current patch the domain_level in config is >>>>>>>>>>>> not >>>>>>>>>>>> used >>>>>>>>>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>>>>>>>>> >>>>>>>>>>>> For example I want to achieve this goal: >>>>>>>>>>>> test_vault.py, this test suite can run on domain level1 and on >>>>>>>>>>>> domain >>>>>>>>>>>> level0, so with one test we can test 2 domain levels just with >>>>>>>>>>>> putting >>>>>>>>>>>> domain level into config file. >>>>>>>>>>>> >>>>>>>>>>>> I agree that with extraordinary test like replica promotion test >>>>>>>>>>>> is, we >>>>>>>>>>>> need something that allows override the config file. >>>>>>>>>>>> >>>>>>>>>>>> As I said bellow, domain_level default value should be None in >>>>>>>>>>>> install_master and install_topo plugin. If domain level was >>>>>>>>>>>> specified >>>>>>>>>>>> use the specified one, if not (value is None) use the domain >>>>>>>>>>>> level >>>>>>>>>>>> from >>>>>>>>>>>> config file. >>>>>>>>>>> Agreed :) >>>>>>>>>>> >>>>>>>>>>>> Martin >>>>>>>>>>>>>> [PATCH] Enabled setting domain_level per class derived from >>>>>>>>>>>>>> TestIntegration >>>>>>>>>>>>>> >>>>>>>>>>>>>> When I configure domain level 0 in yaml config, how is this >>>>>>>>>>>>>> supposed to >>>>>>>>>>>>>> get into install methods when you removed that code? >>>>>>>>>>>>>> >>>>>>>>>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>>>>>>>>> + "--domain-level=%i" % domain_level >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> You always use MAX_DOMAIN_LEVEL in this case or whatever is >>>>>>>>>>>>>> specified in >>>>>>>>>>>>>> domain_level option. >>>>>>>>>>>>>> I suggest to use domain_level=None, and when it is None use >>>>>>>>>>>>>> 'host.config.domain_level', if it is not None, use >>>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>>> >>>>>>>>>>>>>> With this we can specify domain level in config file for test >>>>>>>>>>>>>> that >>>>>>>>>>>>>> can >>>>>>>>>>>>>> be used for both domain levels and you can manually specify >>>>>>>>>>>>>> domain >>>>>>>>>>>>>> level >>>>>>>>>>>>>> for test that requires specific domain level. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Also this should go away >>>>>>>>>>>>>> >>>>>>>>>>>>>> @classmethod >>>>>>>>>>>>>> def install(cls, mh): >>>>>>>>>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>>>>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>>>>>>>>> if cls.topology is None: >>>>>>>>>>>>>> return >>>>>>>>>>>>>> >>>>>>>>>>>>>> I do not see reason why test should override configuration in >>>>>>>>>>>>>> config in >>>>>>>>>>>>>> this case. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Martin >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Here is the updated version of the patch (more tests + >>>>>>>>>>>>>>> fixed the >>>>>>>>>>>>>>> issues of the first one) + patch 0017, that implements the >>>>>>>>>>>>>>> necessary >>>>>>>>>>>>>>> changes in the background code, i. e. patch 16 does not work >>>>>>>>>>>>>>> without >>>>>>>>>>>>>>> patch 17 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>>>>>>>>> Hi guys, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Here are first two automated testcases from this (so far >>>>>>>>>>>>>>>>> incomplete) >>>>>>>>>>>>>>>>> testplan: >>>>>>>>>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> PATCH 16: NACK >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>>>>>>>>> 'domain_level' to >>>>>>>>>>>>>>>> install_topo()? >>>>>>>>>>>>>>>> Also it is good practise to add new option as the last >>>>>>>>>>>>>>>> parameter. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>>> cab you in both tests specify a domain level with constant >>>>>>>>>>>>>>>> instead of >>>>>>>>>>>>>>>> number literal? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>>> both test call install_topo with custom domain level, but it >>>>>>>>>>>>>>>> cannot >>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>> because 1) (did you run the test?) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>>>>>>>>> Respectively why there is call of install_topo() that >>>>>>>>>>>>>>>> installs >>>>>>>>>>>>>>>> replica. >>>>>>>>>>>>>>>> As this test just tests that ipa-replica-prepare is not >>>>>>>>>>>>>>>> working >>>>>>>>>>>>>>>> anymore, >>>>>>>>>>>>>>>> is it worth to spend 20 minutes with installing replica and >>>>>>>>>>>>>>>> then >>>>>>>>>>>>>>>> just no >>>>>>>>>>>>>>>> tot use it? IMO to install master in install step is enough. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Martin^2 >>>>>>>>>>>>>>>> >>>>>>>>>> ./make-lint >>>>>>>>>> ************* Module ipatests.test_integration.base >>>>>>>>>> ipatests/test_integration/base.py:66: [E1101(no-member), >>>>>>>>>> IntegrationTest.install] Class 'IntegrationTest' has no >>>>>>>>>> 'domain_level' >>>>>>>>>> member) >>>>>>>>>> ************* Module >>>>>>>>>> ipatests.test_integration.test_replica_promotion >>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:16: >>>>>>>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no >>>>>>>>>> 'domain_level' >>>>>>>>>> member) >>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:117: >>>>>>>>>> [E1101(no-member), >>>>>>>>>> TestCAInstall.test_ca_install_without_replica_file] >>>>>>>>>> Module 'ipatests.test_integration.tasks' has no 'setup_replica' >>>>>>>>>> member) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Is it so hard to run pylint before patch is posted for review? >>>>>>>>> Sorry, my bad. Fixed. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> 1) >>>>>>> Why is this change in the patch? >>>>>>> - # Clean up the test directory >>>>>>> - host.run_command(['rm', '-rvf', host.config.test_dir]) >>>>>> Otherwise 2 out of 8 tests fail with IOError at line 78 of >>>>>> ipatests/test_integration/tasks.py >>>>>> >>>>>> I do not understand yet how does this happen, but if you remove >>>>>> ipatests folder once, it then fails to be created again. >>>>>> >>>>> So this should be in separated patch and investigated properly. >>>> Agree. Removed >>>>>>> - >>>>>>> >>>>>>> >>>>>>> 2) >>>>>>> is enough to have this check only in install_master, install_topo can >>>>>>> pass None to install_master >>>>>>> + if domain_level is None: >>>>>>> + domain_level = master.config.domain_level >>>>>> Done >>>>>> >>>>>>> 3) >>>>>>> IMO replica-manage del should cleanup hosts entry, so following code >>>>>>> should not be needed. >>>>>>> + if cleanhost: >>>>>>> + kinit_admin(master) >>>>>>> + master.run_command(["ipa", "host-del", "--updatedns", >>>>>>> replica.hostname], >>>>>>> + raiseonerr=False) >>>>>> Well, in fact it does not. At least the corresponding dns record stays >>>>>> and causes the subsequent ipa-client-install to fail. Probably it's a >>>>>> bug. On the other hand, if I promote an existing client to replica and >>>>>> then delete this replica, then, I probably want the host record (that >>>>>> was created during client-install) to stay in the system. So, does not >>>>>> look like a bug to me. >>>>> No you don't, because replica uninstallation also removes the client. >>>>> >>>>> I tried it with ipa42, ipa-replica-manage del removes host entry, and >>>>> DNS A records, only SSHFP records stay there (I'm not sure if it is bug >>>>> or feature) >>>>> >>>>> Also I received this message >>>>> """ >>>>> Failed to cleanup replica1.ipa.test DNS entries: no matching entry >>>>> found >>>>> You may need to manually remove them from the tree >>>>> """ >>>>> But, A record has been removed, so this is probably false positive and >>>>> it needs to have a ticket. >>>> Agree, that was an issue with my setup. >>>> Removed >>>> >>>>>>> 4) >>>>>>> This variable is not used >>>>>>> + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] >>>>>> Removed >>>>>> >>>>>>> 5) >>>>>>> Why do you need this >>>>>>> + kra_install = ["ipa-kra-install", "-U", "-p", >>>>>>> config.dirman_password] >>>>>>> when you implemented tasks.install_kra that returns the exactly >>>>>>> the same >>>>>>> result? >>>>>> Right. Removed >>>>>> >>>>>>> 6) >>>>>>> PEP8 >>>>>>> ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 blank >>>>>>> lines, >>>>>>> found 1 >>>>>>> ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 blank >>>>>>> lines, >>>>>>> found 1 >>>>>>> ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 blank >>>>>>> lines, >>>>>>> found 1 >>>>>>> ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 blank >>>>>>> lines, >>>>>>> found 1 >>>>>>> ./ipatests/test_integration/tasks.py:950:80: E501 line too long >>>>>>> (80 > 79 >>>>>>> characters) >>>>>>> >>>>>>> ./ipatests/test_integration/test_replica_promotion.py:29:80: E501 >>>>>>> line >>>>>>> too long (85 > 79 characters) >>>>>>> ./ipatests/test_integration/test_replica_promotion.py:64:80: E501 >>>>>>> line >>>>>>> too long (85 > 79 characters) >>>>>>> ./ipatests/test_integration/test_replica_promotion.py:67:80: E501 >>>>>>> line >>>>>>> too long (88 > 79 characters) >>>>>>> ./ipatests/test_integration/test_replica_promotion.py:93:80: E501 >>>>>>> line >>>>>>> too long (80 > 79 characters) >>>>>>> ./ipatests/test_integration/test_replica_promotion.py:94:80: E501 >>>>>>> line >>>>>>> too long (83 > 79 characters) >>>>>>> ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 >>>>>>> line >>>>>>> too long (81 > 79 characters) >>>>>>> ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 >>>>>>> line >>>>>>> too long (80 > 79 characters) >>>>>>> ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 >>>>>>> line >>>>>>> too long (82 > 79 characters) >>>>>>> ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 >>>>>>> line >>>>>>> too long (80 > 79 characters) >>>>>> Most of these complaints are unrelated to the current patches. >>>>>> It's better to create a separate patch addressing PEP8 errors. >>>>>> >>>>> I beg for your pardon, those PEP8 errors have been introduced by your >>>>> patches. >>>> Fixed >>>> >>>>>>> 7) >>>>>>> Why this must be stored in instance? IMO to have it stored as local >>>>>>> variable is perfect >>>>>>> TestKRAInstall, TestCAInstall >>>>>>> self.replica1_filename = >>>>>>> tasks.get_replica_filename(replica1) >>>>>>> self.replica2_filename = >>>>>>> tasks.get_replica_filename(replica2) >>>>>> Agree. Fixed >>>>>> >>>> >> This patch is missing something. > I am sorry, I forgot to revert my previous change. The correct patch is > attached > ************* Module ipatests.test_integration.test_replica_promotion ipatests/test_integration/test_replica_promotion.py:15: [E1123(unexpected-keyword-arg), Dummy.install] Unexpected keyword argument 'domain_level' in function call) ipatests/test_integration/test_replica_promotion.py:15: [E1101(no-member), Dummy.install] Class 'Dummy' has no 'domain_level' member) ipatests/test_integration/test_replica_promotion.py:19: [E1101(no-member), Dummy.teardown_method] Module 'ipatests.test_integration.tasks' has no 'uninstall_replica' member) ipatests/test_integration/test_replica_promotion.py:67: [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] Module 'ipatests.test_integration.tasks' has no 'ipa_backup' member) ipatests/test_integration/test_replica_promotion.py:72: [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] Module 'ipatests.test_integration.tasks' has no 'ipa_restore' member) ipatests/test_integration/test_replica_promotion.py:120: [E1123(unexpected-keyword-arg), TestCAInstall.install] Unexpected keyword argument 'domain_level' in function call) From mbasti at redhat.com Wed Dec 9 11:17:49 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 9 Dec 2015 12:17:49 +0100 Subject: [Freeipa-devel] [PATCH 016 - 017] First part of the replica promotion tests + testplan In-Reply-To: <56680C2E.7060805@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> <565DB7D3.5040500@redhat.com> <565F0BDD.8080206@redhat.com> <565F17EE.3020701@redhat.com> <5666DC18.2040906@redhat.com> <56675E31.3050203@redhat.com> <5667F49B.5020709@redhat.com> <5667FEFB.6090904@redhat.com> <56680C2E.7060805@redhat.com> Message-ID: <56680DDD.2000106@redhat.com> On 09.12.2015 12:10, Martin Basti wrote: > > > On 09.12.2015 11:14, Oleg Fayans wrote: >> Hi Martin >> >> On 12/09/2015 10:30 AM, Martin Basti wrote: >>> >>> On 08.12.2015 23:48, Oleg Fayans wrote: >>>> Substituted a hardcoded suffix name with a constant DOMAIN_SUFFIX_NAME >>>> >>>> On 12/08/2015 02:33 PM, Oleg Fayans wrote: >>>>> Hi all, >>>>> >>>>> >>>>> The patches are rebased against the current master. >>>>> >>>>> On 12/02/2015 05:10 PM, Martin Basti wrote: >>>>>> On 02.12.2015 16:18, Oleg Fayans wrote: >>>>>>> Hi Martin, >>>>>>> >>>>>>> On 12/01/2015 04:08 PM, Martin Basti wrote: >>>>>>>> On 27.11.2015 16:26, Oleg Fayans wrote: >>>>>>>>> And patch N 16 passes lint too: >>>>>>>>> >>>>>>>>> On 11/27/2015 04:03 PM, Oleg Fayans wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> On 11/27/2015 03:26 PM, Martin Basti wrote: >>>>>>>>>>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>>>>>>>>>> Hi Martin, >>>>>>>>>>>> >>>>>>>>>>>> All your suggestions were taken into account. Both patches are >>>>>>>>>>>> updated. Thank you for your help! >>>>>>>>>>>> >>>>>>>>>>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>>>>>>>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I agree to all your points but one. please, see my comment >>>>>>>>>>>>>> below >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 0) Note >>>>>>>>>>>>>>> Please be aware of >>>>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5469 >>>>>>>>>>>>>>> during >>>>>>>>>>>>>>> KRA testing >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, >>>>>>>>>>>>>>> this may >>>>>>>>>>>>>>> change >>>>>>>>>>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for domain >>>>>>>>>>>>>>> level 0 >>>>>>>>>>>>>>> and 1 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>> Why uninstall KRA then server, is not enough just uninstall >>>>>>>>>>>>>>> server >>>>>>>>>>>>>>> which >>>>>>>>>>>>>>> covers KRA uninstall? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> + def teardown_method(self, method): >>>>>>>>>>>>>>> + for host in self.replicas: >>>>>>>>>>>>>>> + host.run_command(self.kra_uninstall, >>>>>>>>>>>>>>> raiseonerr=False) >>>>>>>>>>>>>>> + tasks.uninstall_master(host) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>> Can be this function more generic? It should allow specify >>>>>>>>>>>>>>> host >>>>>>>>>>>>>>> where >>>>>>>>>>>>>>> KRA should be installed not just master >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> + def test_kra_install_master(self): >>>>>>>>>>>>>>> + self.master.run_command(self.kra_install) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> TestLevel0(Dummy): >>>>>>>>>>>>>>> Can be the test name more specific, something like >>>>>>>>>>>>>>> TestReplicaPromotionLevel0 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 5) >>>>>>>>>>>>>>> please remove this, the patch is on review and it will be >>>>>>>>>>>>>>> pushed >>>>>>>>>>>>>>> sooner >>>>>>>>>>>>>>> than tests >>>>>>>>>>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> and as I mentioned in ticket #5455, I cannot reproduce >>>>>>>>>>>>>>> it with >>>>>>>>>>>>>>> ipa-kra-install, so please provide steps to reproduce if >>>>>>>>>>>>>>> you >>>>>>>>>>>>>>> insist >>>>>>>>>>>>>>> that >>>>>>>>>>>>>>> this still does not work as expected with KRA. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 6) This is completely wrong, it removes everything that we >>>>>>>>>>>>>>> tried to >>>>>>>>>>>>>>> achieve with previous patches with domain level in CI >>>>>>>>>>>>>> Actually, being able to configure domain level per class >>>>>>>>>>>>>> is WAY >>>>>>>>>>>>>> more >>>>>>>>>>>>>> convenient, than to always have to think which domain >>>>>>>>>>>>>> level is >>>>>>>>>>>>>> appropriate for which particular test during jenkins job >>>>>>>>>>>>>> configuration. In fact, I should have thought about it >>>>>>>>>>>>>> from the >>>>>>>>>>>>>> very >>>>>>>>>>>>>> beginning. For example, in test_replica_promotion.py we >>>>>>>>>>>>>> have on >>>>>>>>>>>>>> class, >>>>>>>>>>>>>> which intiates with domain level = 1, while others - with >>>>>>>>>>>>>> domain >>>>>>>>>>>>>> level >>>>>>>>>>>>>> 0. With config-based approach, we would have to implement a >>>>>>>>>>>>>> separate >>>>>>>>>>>>>> step that raises domain level. Overall, I am against the >>>>>>>>>>>>>> approach, >>>>>>>>>>>>>> when you have to remember to set certain domain level in >>>>>>>>>>>>>> config >>>>>>>>>>>>>> for >>>>>>>>>>>>>> any particular test. The tests themselves should be aware of >>>>>>>>>>>>>> the >>>>>>>>>>>>>> domain level they need. >>>>>>>>>>>>> I do not say that we should not have something that overrides >>>>>>>>>>>>> settings >>>>>>>>>>>>> in from config in a particular test case, I say your patch is >>>>>>>>>>>>> doing it >>>>>>>>>>>>> wrong. >>>>>>>>>>>>> >>>>>>>>>>>>> I agree it is useful to have param domain_level in >>>>>>>>>>>>> install_master, >>>>>>>>>>>>> and >>>>>>>>>>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by >>>>>>>>>>>>> default, >>>>>>>>>>>>> because with your current patch the domain_level in config is >>>>>>>>>>>>> not >>>>>>>>>>>>> used >>>>>>>>>>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>>>>>>>>>> >>>>>>>>>>>>> For example I want to achieve this goal: >>>>>>>>>>>>> test_vault.py, this test suite can run on domain level1 >>>>>>>>>>>>> and on >>>>>>>>>>>>> domain >>>>>>>>>>>>> level0, so with one test we can test 2 domain levels just >>>>>>>>>>>>> with >>>>>>>>>>>>> putting >>>>>>>>>>>>> domain level into config file. >>>>>>>>>>>>> >>>>>>>>>>>>> I agree that with extraordinary test like replica >>>>>>>>>>>>> promotion test >>>>>>>>>>>>> is, we >>>>>>>>>>>>> need something that allows override the config file. >>>>>>>>>>>>> >>>>>>>>>>>>> As I said bellow, domain_level default value should be >>>>>>>>>>>>> None in >>>>>>>>>>>>> install_master and install_topo plugin. If domain level was >>>>>>>>>>>>> specified >>>>>>>>>>>>> use the specified one, if not (value is None) use the domain >>>>>>>>>>>>> level >>>>>>>>>>>>> from >>>>>>>>>>>>> config file. >>>>>>>>>>>> Agreed :) >>>>>>>>>>>> >>>>>>>>>>>>> Martin >>>>>>>>>>>>>>> [PATCH] Enabled setting domain_level per class derived from >>>>>>>>>>>>>>> TestIntegration >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> When I configure domain level 0 in yaml config, how is this >>>>>>>>>>>>>>> supposed to >>>>>>>>>>>>>>> get into install methods when you removed that code? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>>>>>>>>>> + "--domain-level=%i" % domain_level >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> You always use MAX_DOMAIN_LEVEL in this case or whatever is >>>>>>>>>>>>>>> specified in >>>>>>>>>>>>>>> domain_level option. >>>>>>>>>>>>>>> I suggest to use domain_level=None, and when it is None use >>>>>>>>>>>>>>> 'host.config.domain_level', if it is not None, use >>>>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> With this we can specify domain level in config file for >>>>>>>>>>>>>>> test >>>>>>>>>>>>>>> that >>>>>>>>>>>>>>> can >>>>>>>>>>>>>>> be used for both domain levels and you can manually specify >>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>> level >>>>>>>>>>>>>>> for test that requires specific domain level. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Also this should go away >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> @classmethod >>>>>>>>>>>>>>> def install(cls, mh): >>>>>>>>>>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>>>>>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>>>>>>>>>> if cls.topology is None: >>>>>>>>>>>>>>> return >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I do not see reason why test should override >>>>>>>>>>>>>>> configuration in >>>>>>>>>>>>>>> config in >>>>>>>>>>>>>>> this case. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Martin >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Here is the updated version of the patch (more tests + >>>>>>>>>>>>>>>> fixed the >>>>>>>>>>>>>>>> issues of the first one) + patch 0017, that implements the >>>>>>>>>>>>>>>> necessary >>>>>>>>>>>>>>>> changes in the background code, i. e. patch 16 does not >>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>> without >>>>>>>>>>>>>>>> patch 17 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>>>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>>>>>>>>>> Hi guys, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Here are first two automated testcases from this (so far >>>>>>>>>>>>>>>>>> incomplete) >>>>>>>>>>>>>>>>>> testplan: >>>>>>>>>>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> PATCH 16: NACK >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>>>>>>>>>> 'domain_level' to >>>>>>>>>>>>>>>>> install_topo()? >>>>>>>>>>>>>>>>> Also it is good practise to add new option as the last >>>>>>>>>>>>>>>>> parameter. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>>>> cab you in both tests specify a domain level with >>>>>>>>>>>>>>>>> constant >>>>>>>>>>>>>>>>> instead of >>>>>>>>>>>>>>>>> number literal? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>>>> both test call install_topo with custom domain level, >>>>>>>>>>>>>>>>> but it >>>>>>>>>>>>>>>>> cannot >>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>> because 1) (did you run the test?) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>>>>>>>>>> Respectively why there is call of install_topo() that >>>>>>>>>>>>>>>>> installs >>>>>>>>>>>>>>>>> replica. >>>>>>>>>>>>>>>>> As this test just tests that ipa-replica-prepare is not >>>>>>>>>>>>>>>>> working >>>>>>>>>>>>>>>>> anymore, >>>>>>>>>>>>>>>>> is it worth to spend 20 minutes with installing >>>>>>>>>>>>>>>>> replica and >>>>>>>>>>>>>>>>> then >>>>>>>>>>>>>>>>> just no >>>>>>>>>>>>>>>>> tot use it? IMO to install master in install step is >>>>>>>>>>>>>>>>> enough. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Martin^2 >>>>>>>>>>>>>>>>> >>>>>>>>>>> ./make-lint >>>>>>>>>>> ************* Module ipatests.test_integration.base >>>>>>>>>>> ipatests/test_integration/base.py:66: [E1101(no-member), >>>>>>>>>>> IntegrationTest.install] Class 'IntegrationTest' has no >>>>>>>>>>> 'domain_level' >>>>>>>>>>> member) >>>>>>>>>>> ************* Module >>>>>>>>>>> ipatests.test_integration.test_replica_promotion >>>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:16: >>>>>>>>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no >>>>>>>>>>> 'domain_level' >>>>>>>>>>> member) >>>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:117: >>>>>>>>>>> [E1101(no-member), >>>>>>>>>>> TestCAInstall.test_ca_install_without_replica_file] >>>>>>>>>>> Module 'ipatests.test_integration.tasks' has no 'setup_replica' >>>>>>>>>>> member) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Is it so hard to run pylint before patch is posted for review? >>>>>>>>>> Sorry, my bad. Fixed. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> 1) >>>>>>>> Why is this change in the patch? >>>>>>>> - # Clean up the test directory >>>>>>>> - host.run_command(['rm', '-rvf', host.config.test_dir]) >>>>>>> Otherwise 2 out of 8 tests fail with IOError at line 78 of >>>>>>> ipatests/test_integration/tasks.py >>>>>>> >>>>>>> I do not understand yet how does this happen, but if you remove >>>>>>> ipatests folder once, it then fails to be created again. >>>>>>> >>>>>> So this should be in separated patch and investigated properly. >>>>> Agree. Removed >>>>>>>> - >>>>>>>> >>>>>>>> >>>>>>>> 2) >>>>>>>> is enough to have this check only in install_master, >>>>>>>> install_topo can >>>>>>>> pass None to install_master >>>>>>>> + if domain_level is None: >>>>>>>> + domain_level = master.config.domain_level >>>>>>> Done >>>>>>> >>>>>>>> 3) >>>>>>>> IMO replica-manage del should cleanup hosts entry, so following >>>>>>>> code >>>>>>>> should not be needed. >>>>>>>> + if cleanhost: >>>>>>>> + kinit_admin(master) >>>>>>>> + master.run_command(["ipa", "host-del", "--updatedns", >>>>>>>> replica.hostname], >>>>>>>> + raiseonerr=False) >>>>>>> Well, in fact it does not. At least the corresponding dns record >>>>>>> stays >>>>>>> and causes the subsequent ipa-client-install to fail. Probably >>>>>>> it's a >>>>>>> bug. On the other hand, if I promote an existing client to >>>>>>> replica and >>>>>>> then delete this replica, then, I probably want the host record >>>>>>> (that >>>>>>> was created during client-install) to stay in the system. So, >>>>>>> does not >>>>>>> look like a bug to me. >>>>>> No you don't, because replica uninstallation also removes the >>>>>> client. >>>>>> >>>>>> I tried it with ipa42, ipa-replica-manage del removes host entry, >>>>>> and >>>>>> DNS A records, only SSHFP records stay there (I'm not sure if it >>>>>> is bug >>>>>> or feature) >>>>>> >>>>>> Also I received this message >>>>>> """ >>>>>> Failed to cleanup replica1.ipa.test DNS entries: no matching entry >>>>>> found >>>>>> You may need to manually remove them from the tree >>>>>> """ >>>>>> But, A record has been removed, so this is probably false >>>>>> positive and >>>>>> it needs to have a ticket. >>>>> Agree, that was an issue with my setup. >>>>> Removed >>>>> >>>>>>>> 4) >>>>>>>> This variable is not used >>>>>>>> + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] >>>>>>> Removed >>>>>>> >>>>>>>> 5) >>>>>>>> Why do you need this >>>>>>>> + kra_install = ["ipa-kra-install", "-U", "-p", >>>>>>>> config.dirman_password] >>>>>>>> when you implemented tasks.install_kra that returns the exactly >>>>>>>> the same >>>>>>>> result? >>>>>>> Right. Removed >>>>>>> >>>>>>>> 6) >>>>>>>> PEP8 >>>>>>>> ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 blank >>>>>>>> lines, >>>>>>>> found 1 >>>>>>>> ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 blank >>>>>>>> lines, >>>>>>>> found 1 >>>>>>>> ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 blank >>>>>>>> lines, >>>>>>>> found 1 >>>>>>>> ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 blank >>>>>>>> lines, >>>>>>>> found 1 >>>>>>>> ./ipatests/test_integration/tasks.py:950:80: E501 line too long >>>>>>>> (80 > 79 >>>>>>>> characters) >>>>>>>> >>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:29:80: E501 >>>>>>>> line >>>>>>>> too long (85 > 79 characters) >>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:64:80: E501 >>>>>>>> line >>>>>>>> too long (85 > 79 characters) >>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:67:80: E501 >>>>>>>> line >>>>>>>> too long (88 > 79 characters) >>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:93:80: E501 >>>>>>>> line >>>>>>>> too long (80 > 79 characters) >>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:94:80: E501 >>>>>>>> line >>>>>>>> too long (83 > 79 characters) >>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 >>>>>>>> line >>>>>>>> too long (81 > 79 characters) >>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 >>>>>>>> line >>>>>>>> too long (80 > 79 characters) >>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 >>>>>>>> line >>>>>>>> too long (82 > 79 characters) >>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 >>>>>>>> line >>>>>>>> too long (80 > 79 characters) >>>>>>> Most of these complaints are unrelated to the current patches. >>>>>>> It's better to create a separate patch addressing PEP8 errors. >>>>>>> >>>>>> I beg for your pardon, those PEP8 errors have been introduced by >>>>>> your >>>>>> patches. >>>>> Fixed >>>>> >>>>>>>> 7) >>>>>>>> Why this must be stored in instance? IMO to have it stored as >>>>>>>> local >>>>>>>> variable is perfect >>>>>>>> TestKRAInstall, TestCAInstall >>>>>>>> self.replica1_filename = >>>>>>>> tasks.get_replica_filename(replica1) >>>>>>>> self.replica2_filename = >>>>>>>> tasks.get_replica_filename(replica2) >>>>>>> Agree. Fixed >>>>>>> >>>>> >>> This patch is missing something. >> I am sorry, I forgot to revert my previous change. The correct patch is >> attached >> > ************* Module ipatests.test_integration.test_replica_promotion > ipatests/test_integration/test_replica_promotion.py:15: > [E1123(unexpected-keyword-arg), Dummy.install] Unexpected keyword > argument 'domain_level' in function call) > ipatests/test_integration/test_replica_promotion.py:15: > [E1101(no-member), Dummy.install] Class 'Dummy' has no 'domain_level' > member) > ipatests/test_integration/test_replica_promotion.py:19: > [E1101(no-member), Dummy.teardown_method] Module > 'ipatests.test_integration.tasks' has no 'uninstall_replica' member) > ipatests/test_integration/test_replica_promotion.py:67: > [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] > Module 'ipatests.test_integration.tasks' has no 'ipa_backup' member) > ipatests/test_integration/test_replica_promotion.py:72: > [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] > Module 'ipatests.test_integration.tasks' has no 'ipa_restore' member) > ipatests/test_integration/test_replica_promotion.py:120: > [E1123(unexpected-keyword-arg), TestCAInstall.install] Unexpected > keyword argument 'domain_level' in function call) > Sorry I forgot to apply patch 17, my bad, I'm continuing with review From tbabej at redhat.com Wed Dec 9 11:35:14 2015 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 9 Dec 2015 12:35:14 +0100 Subject: [Freeipa-devel] [PATCH 0071] replica: Fix ipa-replica-install with replica file (domain, level 0). In-Reply-To: <5667D8E0.6070303@redhat.com> References: <5666E31E.1030705@redhat.com> <5666F54C.80801@redhat.com> <5666F831.5040705@redhat.com> <5667D8E0.6070303@redhat.com> Message-ID: <566811F2.7010603@redhat.com> On 12/09/2015 08:31 AM, David Kupka wrote: > On 08/12/15 16:33, Tomas Babej wrote: >> >> >> On 12/08/2015 04:20 PM, Oleg Fayans wrote: >>> ACK. The initial issue is fixed. >>> >>> On 12/08/2015 03:03 PM, David Kupka wrote: >>>> https://fedorahosted.org/freeipa/ticket/5531 >>>> >>>> >>> >> >> Can we get some more love for the patch and provide at least a sentence >> worth of commit message before pushing? >> >> It's not obvious from the title what the patch does, other than it fixes >> things. >> >> Tomas >> > I believe it's pretty obvious from linked ticket and attached patch > changing just 5 lines. > But you're right verbosity in commit message could save time later. > Patch with changed commit message attached. > Yes, I'd rather avoid the need to go to the ticket when perusing the git log. ACK, thanks. Tomas From mbabinsk at redhat.com Wed Dec 9 11:40:11 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Wed, 9 Dec 2015 12:40:11 +0100 Subject: [Freeipa-devel] [TESTS][PATCH 0006] Add comments to stageuser plugin tests In-Reply-To: <5668028E.60902@redhat.com> References: <564C7A23.5010500@redhat.com> <564C9E77.4050406@redhat.com> <564D88AF.8010500@redhat.com> <564D97AC.6050901@redhat.com> <564F004A.2060806@redhat.com> <5667E942.7030305@redhat.com> <5667F0A6.1090104@redhat.com> <5668028E.60902@redhat.com> Message-ID: <5668131B.1070207@redhat.com> On 12/09/2015 11:29 AM, Lenka Doudova wrote: > > > On 12/09/2015 10:13 AM, Martin Basti wrote: >> >> >> On 09.12.2015 09:41, Lenka Doudova wrote: >>> Hi, >>> >>> attaching fixed patches for master and ipa-4-2 branch. >>> Also fixes test accordingly to >>> https://fedorahosted.org/freeipa/ticket/5387. >>> >>> Lenka >>> >>> On 11/20/2015 12:13 PM, Martin Babinsky wrote: >>>> On 11/19/2015 10:34 AM, Petr Viktorin wrote: >>>>> On 11/19/2015 09:30 AM, Lenka Doudova wrote: >>>>>> On 11/18/2015 04:51 PM, Martin Babinsky wrote: >>>>>>> On 11/18/2015 02:16 PM, Lenka Doudova wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> here's a patch that adds a few comments to stageuser tests in >>>>>>>> order to >>>>>>>> allow easier determining of a problem when tests fail. >>>>>>>> >>>>>>>> Lenka >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> Hi Lenka, >>>>>>> >>>>>>> Firstly a technical detail: Python indexes lists from 0, so the >>>>>>> comments in 'options_ok' do not correctly map to the test names >>>>>>> anyway. >>>>>>> >>>>>>> I am also not sure if this patch is worth reviewing and pushing >>>>>>> as it >>>>>>> IMHO doesn't help in the identification of failed tests at all. >>>>>>> >>>>>>> This should be solved at more fundamental level. >>>>>>> >>>>>> Ouch, good point, I didn't realize. Sorry. >>>>>> >>>>>> Anyway, the issue is that even if tests are run in verbose mode, >>>>>> you get >>>>>> output like this: >>>>>> >>>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser27] >>>>>> >>>>>> PASSED >>>>>> >>>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser28] >>>>>> >>>>>> PASSED >>>>>> >>>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser29] >>>>>> >>>>>> PASSED >>>>>> >>>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser210] >>>>>> >>>>>> PASSED >>>>>> >>>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser211] >>>>>> >>>>>> PASSED >>>>>> >>>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser212] >>>>>> >>>>>> PASSED >>>>>> >>>>>> >>>>>> If some test fails, you can't really tell which command was the one >>>>>> responsible for the fail. This should be solved by >>>>>> https://fedorahosted.org/freeipa/ticket/5449. Until that's done, >>>>>> though, >>>>>> the only way to find out which command failed is looking at the >>>>>> code and >>>>>> finding which parameters were put into the command, which was not >>>>>> much >>>>>> possible without better commenting the test case (as I realized last >>>>>> week when David Kupka asked me to help him find the reason for failed >>>>>> tests). >>>>>> Obviously I can rewrite the tests so there's 27 separate test >>>>>> cases, one >>>>>> for each parameter, instead of one method that 'unwraps' into 27 test >>>>>> cases, which would entirely eliminate the confusion. So far I >>>>>> don't know >>>>>> of a way to put 27 similar test cases in one method which would allow >>>>>> easy recognition of the test cases. >>>>>> I'll wait with fixing the patch until further discussion. >>>>> >>>>> Hello, >>>>> Pytest wants you to be a bit more explicit about how to name the >>>>> parameters. (It "hides" dicts by default, because large dicts would >>>>> make >>>>> the output even more confusing than the numbers.) >>>>> >>>>> Please try the attached patch. >>>>> Docs are at >>>>> https://pytest.org/latest/fixture.html#parametrizing-a-fixture >>>>> >>>>> >>>>> >>>> This looks like a better approach IMHO, you can then see which >>>> attribute/value was being checked. >>>> >>>> I would very much favor more descriptive test/fixture names in the >>>> beginning. >>>> >>> >>> >>> >> Hello, >> >> we use usually bottom posting on freeipa-devel please try to keep >> reply in this way. >> >> Patch: >> >> I do not like the idea of separated lists, IMO it is hard to manage >> and is easy to create mistakes. >> >> How about this (untested, just from top of my head): >> http://fpaste.org/298994/65184014/ >> >> Martin > > Great idea, thanks. Fixed patches attached. > > Lenka > > Tests pass and code looks good, ACK. -- Martin^3 Babinsky From dkupka at redhat.com Wed Dec 9 12:37:39 2015 From: dkupka at redhat.com (David Kupka) Date: Wed, 9 Dec 2015 13:37:39 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <5666E822.90103@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> <5665843C.2090507@redhat.com> <56658C89.8050103@redhat.com> <56668D30.8060308@redhat.com> <5666BCBC.9040005@redhat.com> <5666E822.90103@redhat.com> Message-ID: <56682093.3010505@redhat.com> On 08/12/15 15:24, Petr Spacek wrote: > On 8.12.2015 12:19, David Kupka wrote: >> On 08/12/15 08:56, Petr Spacek wrote: >>> On 7.12.2015 14:41, David Kupka wrote: >>>> +def is_host_resolvable(fqdn): >>>> + if not isinstance(fqdn, DNSName): >>>> + fqdn = DNSName(fqdn) >>>> + for rdtype in (rdatatype.A, rdatatype.AAAA): >>>> + try: >>>> + resolver.query(fqdn.make_absolute(), rdtype) >>>> + except DNSException: >>>> + continue >>>> + else: >>>> + return True >>>> + >>>> + return False >>>> >>> >>> NACK, you are re-introducing duplicate function which was removed in >>> 498471e4aed1367b72cd74d15811d0584a6ee268. >>> >>> Please amend the patch ASAP to use new verify_host_resolvable() function so I >>> can test it and get it into 4.3. >>> >> Updated patches attached. > > Hmm, my review script screams: > > Detected base branch: origin/master > Checks will be made against following base commit: 848912a add missing > /ipaplatform/constants.py to .gitignore > Writing API to API.txt > ./ipalib/plugins/dns.py:2127:9: E124 closing bracket does not match visual > indentation > ./ipalib/plugins/dns.py:2711:13: E128 continuation line under-indented for > visual indent > ./ipalib/plugins/dns.py:2713:9: E124 closing bracket does not match visual > indentation > ./ipalib/plugins/dns.py:2716:13: E128 continuation line under-indented for > visual indent > ./ipapython/ipautil.py:928:1: E302 expected 2 blank lines, found 1 > ./ipapython/ipautil.py:948:17: E128 continuation line under-indented for > visual indent > ./ipapython/ipautil.py:956:1: E302 expected 2 blank lines, found 1 > ./ipapython/ipautil.py:997:80: E501 line too long (83 > 79 characters) > ./ipapython/ipautil.py:998:80: E501 line too long (83 > 79 characters) > ./ipaserver/install/bindinstance.py:49:9: E128 continuation line > under-indented for visual indent > ./ipaserver/install/bindinstance.py:292:80: E501 line too long (80 > 79 > characters) > ./ipaserver/install/bindinstance.py:438:19: E128 continuation line > under-indented for visual indent > ./ipaserver/install/server/common.py:271:63: E261 at least two spaces before > inline comment > ./ipaserver/install/server/common.py:271:80: E501 line too long (90 > 79 > characters) > ERROR: PEP8 --diff failed, continuing ... > ERROR: API.txt was changed without a change in VERSION, continuing ... > Summary of detected errors: > ERROR: PEP8 --diff failed > ERROR: API.txt was changed without a change in VERSION > > Please fix it :-) > > Make make this more pleasant for you, you can use (and review) my attached > patch. It adds 'review' target to Makefile, it will do the same checks as I do. > Unfortunately your tool does not work for me (output w/ -o xtrace attached). Anyway I have incremented API version and fixed most of the pep8 errors except for E124 twice in ipalib/plugins/dns.py as it seems to be convention in the file and E501 twice in ipapython/ipautil.py because IMO breaking string constant into multiple lines does not help readability. Updated patches also attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0058.9-dns-do-not-add-forward-zone-if-it-is-already-resolva.patch Type: text/x-patch Size: 12272 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0064.6-dns-Check-if-domain-already-exists.patch Type: text/x-patch Size: 13413 bytes Desc: not available URL: -------------- next part -------------- $ make review if [ ! -e RELEASE ]; then echo 0 > RELEASE; fi sed -e s/__VERSION__/4.2.90.201512090009GITa6a6542/ -e s/__RELEASE__/0/ \ freeipa.spec.in > freeipa.spec sed -e s/__VERSION__/4.2.90.201512090009GITa6a6542/ version.m4.in \ > version.m4 sed -e s/__VERSION__/4.2.90.201512090009GITa6a6542/ ipapython/setup.py.in \ > ipapython/setup.py sed -e s/__VERSION__/4.2.90.201512090009GITa6a6542/ ipaplatform/setup.py.in \ > ipaplatform/setup.py sed -e s/__VERSION__/4.2.90.201512090009GITa6a6542/ ipapython/version.py.in \ > ipapython/version.py sed -e s/__VERSION__/4.2.90.201512090009GITa6a6542/ ipatests/setup.py.in \ > ipatests/setup.py sed -e s/__NUM_VERSION__/40290/ install/ui/src/libs/loader.js.in \ > install/ui/src/libs/loader.js perl -pi -e "s:__API_VERSION__:2.162:" install/ui/src/libs/loader.js perl -pi -e "s:__NUM_VERSION__:40290:" ipapython/version.py perl -pi -e "s:__VENDOR_VERSION__:4.2.90.201512090009GITa6a6542:" ipapython/version.py perl -pi -e "s:__API_VERSION__:2.162:" ipapython/version.py touch -r ipapython/version.py.in ipapython/version.py sed -e s/__VERSION__/4.2.90.201512090009GITa6a6542/ daemons/ipa-version.h.in \ > daemons/ipa-version.h perl -pi -e "s:__NUM_VERSION__:40290:" daemons/ipa-version.h perl -pi -e "s:__DATA_VERSION__:20100614120000:" daemons/ipa-version.h sed -e s/__VERSION__/4.2.90.201512090009GITa6a6542/ -e s/__RELEASE__/0/ \ ipa-client/ipa-client.spec.in > ipa-client/ipa-client.spec sed -e s/__VERSION__/4.2.90.201512090009GITa6a6542/ ipa-client/version.m4.in \ > ipa-client/version.m4 if [ "fedora" != "" ]; then \ sed -e s/__PLATFORM__/fedora/ \ ipaplatform/__init__.py.in > ipaplatform/__init__.py; \ rm -f ipaplatform/paths.py ipaplatform/services.py ipaplatform/tasks.py ipaplatform/constants.py; \ ln -s fedora/paths.py ipaplatform/paths.py; \ ln -s fedora/services.py ipaplatform/services.py; \ ln -s fedora/tasks.py ipaplatform/tasks.py; \ ln -s fedora/constants.py ipaplatform/constants.py; \ fi if [ "" != "yes" ]; then \ ./makeapi --validate; \ ./makeaci --validate; \ fi ./review.sh ++ mktemp --suff=.log + LOGFILE=/tmp/tmp.vI6nId4bkj.log + exec + exec ++ tee -i /tmp/tmp.vI6nId4bkj.log + echo -n 'make lint is running ... ' make lint is running ... + make --silent lint Building IPA 4.2.90.201512090009GITa6a6542 ~/freeipa ~/freeipa/install/po install/po/tmp.pot.update: warning: Charset "CHARSET" is not a portable encoding name. Message conversion to user's charset might not work. ~/freeipa/install/po tmp.pot updated tmp.pot: 3112 entries, 3155 msgid, 0 msgstr, 0 warnings 0 errors + PATCHCNT=0 + BASEBRANCH= ++ git branch --remote --contains + CURRBRANCH= + '[' '' == '' ']' ++ grep -v '^. $' ++ git branch --remote --contains HEAD~0 ++ : + BASEBRANCH= ++ expr 0 + 1 + PATCHCNT=1 + '[' '' == '' ']' ++ grep -v '^. $' ++ git branch --remote --contains HEAD~1 ++ : + BASEBRANCH= ++ expr 1 + 1 + PATCHCNT=2 + '[' '' == '' ']' ++ grep -v '^. $' ++ git branch --remote --contains HEAD~2 ++ : + BASEBRANCH= ++ expr 2 + 1 + PATCHCNT=3 + '[' '' == '' ']' ++ grep -v '^. $' ++ git branch --remote --contains HEAD~3 ++ : + BASEBRANCH= ++ expr 3 + 1 + PATCHCNT=4 + '[' '' == '' ']' ++ grep -v '^. $' ++ git branch --remote --contains HEAD~4 + BASEBRANCH=' origin/HEAD -> origin/master origin/master' ++ expr 4 + 1 + PATCHCNT=5 + '[' ' origin/HEAD -> origin/master origin/master' == '' ']' + echo 'Detected base branch: origin/HEAD -> origin/master origin/master' Detected base branch: origin/HEAD -> origin/master origin/master + echo -n 'Checks will be made against following base commit: ' Checks will be made against following base commit: + git log -1 --oneline origin/HEAD '->' origin/master origin/master fatal: unrecognized argument: -> Makefile:84: recipe for target 'review' failed make: *** [review] Error 128 From redhatrises at gmail.com Wed Dec 9 13:44:19 2015 From: redhatrises at gmail.com (Gabe Alford) Date: Wed, 9 Dec 2015 06:44:19 -0700 Subject: [Freeipa-devel] [PATCH 0065] ipa-replica-install prints incorrect error message when replica is already installed In-Reply-To: <5667F667.9040603@redhat.com> References: <5666F2B8.1090803@redhat.com> <5666F3BB.8030006@redhat.com> <5666F497.3040500@redhat.com> <5666F539.1000904@redhat.com> <5666F6F0.2050005@redhat.com> <5667C566.5000906@redhat.com> <5667F667.9040603@redhat.com> Message-ID: Fixed. Updated patch attached. On Wed, Dec 9, 2015 at 2:37 AM, Martin Basti wrote: > NACK > > Patch contains syntax error, missing brace > > ipaserver/install/server/replicainstall.py:850: [E0001(syntax-error), ] > invalid syntax) > > Martin > > > On 09.12.2015 07:08, Jan Cholasta wrote: > >> LGTM >> >> On 8.12.2015 17:04, Gabe Alford wrote: >> >>> Updated patch attached. >>> >>> On Tue, Dec 8, 2015 at 8:27 AM, Martin Basti >> > wrote: >>> >>> >>> >>> On 08.12.2015 16:26, Gabe Alford wrote: >>> >>>> Just to confirm: >>>> >>>> if server is installed: >>>> Let's stop here and not do anything else >>>> >>>> if domain level 0: >>>> check if client installed and stop here >>>> >>>> Right? >>>> >>> yes >>> >>> >>> >>>> >>>> On Tue, Dec 8, 2015 at 8:20 AM, Jan Cholasta >>> > wrote: >>>> >>>> On 8.12.2015 16:17, Martin Basti wrote: >>>> >>>> >>>> >>>> On 08.12.2015 16:14, Jan Cholasta wrote: >>>> >>>> On 8.12.2015 16:09, Martin Basti wrote: >>>> >>>> >>>> >>>> On 01.12.2015 14:57, Gabe Alford wrote: >>>> >>>> Sorry guys, I forgot to add a meaningful >>>> subject to this message. >>>> Ignore the previous thread start. >>>> >>>> ---------- Forwarded message ---------- >>>> From: *Gabe Alford* >>> >>>> >>> >> >>>> Date: Mon, Nov 30, 2015 at 7:31 PM >>>> Subject: [PATCH 0065] >>>> To: freeipa-devel >>> >>>> >>> >> >>>> >>>> >>>> Hello, >>>> >>>> Patch fix for the following tickets: >>>> >>>> https://fedorahosted.org/freeipa/ticket/5022 >>>> https://fedorahosted.org/freeipa/ticket/5320 >>>> >>>> Thanks, >>>> >>>> Gabe >>>> >>>> >>>> >>>> ACK >>>> >>>> >>>> NACK, you can't install a server over an already >>>> installed client, >>>> thus the original check is correct. >>>> >>>> Ahh domain level 0, right, but this check can be added >>>> before the client >>>> check. >>>> >>>> >>>> Yes. >>>> >>>> With domain level 1, this check should stay there IMO. >>>> >>>> >>>> Yes. It should say "IPA server is already configured" rather >>>> than "IPA replica is already configured", though. >>>> >>>> -- >>>> Jan Cholasta >>>> >>>> >>>> >>> >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rga-0064-3-ipa-replica-install-prints-incorrect-error-message-w.patch Type: text/x-patch Size: 2428 bytes Desc: not available URL: From jcholast at redhat.com Wed Dec 9 13:52:56 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 9 Dec 2015 14:52:56 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <5667EE2C.2050100@redhat.com> References: <5667EE2C.2050100@redhat.com> Message-ID: <56683238.3050700@redhat.com> On 9.12.2015 10:02, Jan Cholasta wrote: > Hi, > > the attached patches fix . Note that this needs selinux-policy fix to work, so put SELinux into permissive mode for testing: . -- Jan Cholasta From pspacek at redhat.com Wed Dec 9 14:30:27 2015 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 9 Dec 2015 15:30:27 +0100 Subject: [Freeipa-devel] [PATCH] Add 'review' target for make Message-ID: <56683B03.40703@redhat.com> Hello, this patch automates some of sanity checks proposed by Petr Vobornik. 'make review' should be used in root of clean Git tree which has patches under review applied. Magic in review.sh attempts to detect nearest remote branch which can be used as diff base for review. Please see review.sh for further details. -- Petr^2 Spacek From jcholast at redhat.com Wed Dec 9 14:48:11 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 9 Dec 2015 15:48:11 +0100 Subject: [Freeipa-devel] [PATCH 0098-0099] domain level 1 topology checks during IPA server uninstall In-Reply-To: <565EEFEC.1080703@redhat.com> References: <564E04BE.1020304@redhat.com> <56586187.5090402@redhat.com> <565C3399.5030109@redhat.com> <565C6E8C.2090900@redhat.com> <565C844B.7050901@redhat.com> <565C8A57.2000305@redhat.com> <565CA4B8.1010701@redhat.com> <565DA35D.6010804@redhat.com> <565EBDA3.40600@redhat.com> <565EED67.3040405@redhat.com> <565EEDDC.8060304@redhat.com> <565EEFEC.1080703@redhat.com> Message-ID: <56683F2B.8040000@redhat.com> On 2.12.2015 14:19, Martin Basti wrote: > > > On 02.12.2015 14:10, Martin Basti wrote: >> >> >> On 02.12.2015 14:08, Martin Babinsky wrote: >>> On 12/02/2015 10:45 AM, Martin Babinsky wrote: >>>> On 12/01/2015 02:40 PM, Martin Babinsky wrote: >>>>> On 11/30/2015 08:34 PM, Martin Basti wrote: >>>>>> >>>>>> >>>>>> On 30.11.2015 18:41, Martin Babinsky wrote: >>>>>>> On 11/30/2015 06:15 PM, Martin Basti wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 30.11.2015 16:43, Martin Babinsky wrote: >>>>>>>>> On 11/30/2015 12:31 PM, Jan Cholasta wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> On 27.11.2015 14:58, Martin Babinsky wrote: >>>>>>>>>>> On 11/19/2015 06:19 PM, Martin Babinsky wrote: >>>>>>>>>>>> These two patches fix the following tickets: >>>>>>>>>>>> >>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5377 >>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5409 >>>>>>>>>>>> >>>>>>>>>>>> I have added a new option '--ignore-disconnected-topology' >>>>>>>>>>>> which >>>>>>>>>>>> forces >>>>>>>>>>>> IPA master uninstall despite reported errors in topology. >>>>>>>>>>>> I'm not >>>>>>>>>>>> quite >>>>>>>>>>>> sure if we want to flood ipa-server-install with >>>>>>>>>>>> uninstall-specific >>>>>>>>>>>> options, maybe it is better to skip the check in unattended >>>>>>>>>>>> mode >>>>>>>>>>>> and >>>>>>>>>>>> just print a warning about disconnected topology and what to do >>>>>>>>>>>> about >>>>>>>>>>>> it. >>>>>>>>>>>> >>>>>>>>>>>> I would like to hear your opinions about this. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> Attaching rebased and updated patches. >>>>>>>>>> >>>>>>>>>> Patch 0098: LGTM >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Patch 0099: >>>>>>>>>> >>>>>>>>>> a) This check should be done in Server.__init__() rather than >>>>>>>>>> install_check(): >>>>>>>>>> >>>>>>>>>> + if options.ignore_disconnected_topology: >>>>>>>>>> + print("'--ignore-disconnected-topology' is used only >>>>>>>>>> during " >>>>>>>>>> + "uninstallation") >>>>>>>>>> + sys.exit(1) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> b) >>>>>>>>>> s/--ignore-disconnected-topology/--ignore-topology-disconnect/, >>>>>>>>>> for >>>>>>>>>> consistency with other options, e.g. --no-ui-redirect. >>>>>>>>>> >>>>>>>>>> Maybe even shorten it to --ignore-topology? But we probably don't >>>>>>>>>> want >>>>>>>>>> people to use this option much, so it might be better to keep it >>>>>>>>>> long? >>>>>>>>>> >>>>>>>>> I would rather leave it with the long option name, it is more >>>>>>>>> apparent >>>>>>>>> what this switch should be around. >>>>>>>>>> >>>>>>>>>> c) I'm fine with uninstall options, you can remove the TODO: >>>>>>>>>> >>>>>>>>>> + # TODO: ask jcholast about uninstallation options >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Honza >>>>>>>>>> >>>>>>>>> >>>>>>>>> Attaching updated patches. >>>>>>>>> >>>>>>>> NACK >>>>>>>> >>>>>>>> ipa-server-install --uninstall >>>>>>>> >>>>>>>> 2015-11-30T17:14:30Z DEBUG Destroyed connection >>>>>>>> context.ldap2_140081152041808 >>>>>>>> 2015-11-30T17:14:30Z DEBUG Traceback (most recent call last): >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>>>>>> line 91, in _handle_exception >>>>>>>> super(Continuous, self)._handle_exception(exc_info) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 387, in _handle_exception >>>>>>>> six.reraise(*exc_info) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 439, in _handle_exception >>>>>>>> super(ComponentBase, self)._handle_exception(exc_info) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 387, in _handle_exception >>>>>>>> six.reraise(*exc_info) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 355, in __runner >>>>>>>> step() >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 352, in >>>>>>>> step = lambda: next(self.__gen) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>>> line 81, in run_generator_with_yield_from >>>>>>>> six.reraise(*exc_info) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>>> line 59, in run_generator_with_yield_from >>>>>>>> value = gen.send(prev_value) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>>>>>> line 71, in _uninstall >>>>>>>> for nothing in self._uninstaller(self.parent): >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>>> >>>>>>>> >>>>>>>> line 1409, in main >>>>>>>> uninstall_check(self) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>>> >>>>>>>> >>>>>>>> line 265, in decorated >>>>>>>> func(installer) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>>> >>>>>>>> >>>>>>>> line 1140, in uninstall_check >>>>>>>> api, masters, options.ignore_disconnected_topology) >>>>>>>> AttributeError: 'uninstaller(Server)' object has no attribute >>>>>>>> 'ignore_disconnected_topology' >>>>>>>> >>>>>>>> 2015-11-30T17:14:30Z ERROR 'uninstaller(Server)' object has no >>>>>>>> attribute >>>>>>>> 'ignore_disconnected_topology' >>>>>>>> 2015-11-30T17:14:30Z INFO The ipa-server-install command was >>>>>>>> successful >>>>>>>> >>>>>>> >>>>>>> Sorry I have failed horribly during option rename. Attaching patch >>>>>>> that should actually work. >>>>>>> >>>>>> functional ACK >>>>> Attaching rebased patches reflecting the recent changes in the >>>>> handling >>>>> of managed topology suffixes handling. >>>>> >>>>> >>>>> >>>>> >>>> >>>> Jan had some more suggestions to the patches. Attaching updated >>>> version. >>>> >>>> >>>> >>> Attaching updated patch 99 with fixed error message. >>> >> Pushed to master: b8c619a7139bd7b65caa03b68431e22791ff19bf >> > ACK :-) I was doing some unrelated testing with domain level 0 and forgot to remove --ignore-topology disconnect from my command line before uninstalling server, which gave me an error that the option cannot be used in domain level 0 and I had to re-run ipa-server-install --uninstall with the option removed. IMO it would be better for UX if the option was ignored in domain level 0, since topology disconnects are *always* ignored in domain level 0. (Right?) The attached patch fixes that. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-526-server-uninstall-ignore-ignore-topology-disconnect-i.patch Type: text/x-patch Size: 1214 bytes Desc: not available URL: From mbabinsk at redhat.com Wed Dec 9 15:22:14 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Wed, 9 Dec 2015 16:22:14 +0100 Subject: [Freeipa-devel] [PATCH 0098-0099] domain level 1 topology checks during IPA server uninstall In-Reply-To: <56683F2B.8040000@redhat.com> References: <564E04BE.1020304@redhat.com> <56586187.5090402@redhat.com> <565C3399.5030109@redhat.com> <565C6E8C.2090900@redhat.com> <565C844B.7050901@redhat.com> <565C8A57.2000305@redhat.com> <565CA4B8.1010701@redhat.com> <565DA35D.6010804@redhat.com> <565EBDA3.40600@redhat.com> <565EED67.3040405@redhat.com> <565EEDDC.8060304@redhat.com> <565EEFEC.1080703@redhat.com> <56683F2B.8040000@redhat.com> Message-ID: <56684726.7020108@redhat.com> On 12/09/2015 03:48 PM, Jan Cholasta wrote: > On 2.12.2015 14:19, Martin Basti wrote: >> >> >> On 02.12.2015 14:10, Martin Basti wrote: >>> >>> >>> On 02.12.2015 14:08, Martin Babinsky wrote: >>>> On 12/02/2015 10:45 AM, Martin Babinsky wrote: >>>>> On 12/01/2015 02:40 PM, Martin Babinsky wrote: >>>>>> On 11/30/2015 08:34 PM, Martin Basti wrote: >>>>>>> >>>>>>> >>>>>>> On 30.11.2015 18:41, Martin Babinsky wrote: >>>>>>>> On 11/30/2015 06:15 PM, Martin Basti wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> On 30.11.2015 16:43, Martin Babinsky wrote: >>>>>>>>>> On 11/30/2015 12:31 PM, Jan Cholasta wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> On 27.11.2015 14:58, Martin Babinsky wrote: >>>>>>>>>>>> On 11/19/2015 06:19 PM, Martin Babinsky wrote: >>>>>>>>>>>>> These two patches fix the following tickets: >>>>>>>>>>>>> >>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5377 >>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5409 >>>>>>>>>>>>> >>>>>>>>>>>>> I have added a new option '--ignore-disconnected-topology' >>>>>>>>>>>>> which >>>>>>>>>>>>> forces >>>>>>>>>>>>> IPA master uninstall despite reported errors in topology. >>>>>>>>>>>>> I'm not >>>>>>>>>>>>> quite >>>>>>>>>>>>> sure if we want to flood ipa-server-install with >>>>>>>>>>>>> uninstall-specific >>>>>>>>>>>>> options, maybe it is better to skip the check in unattended >>>>>>>>>>>>> mode >>>>>>>>>>>>> and >>>>>>>>>>>>> just print a warning about disconnected topology and what >>>>>>>>>>>>> to do >>>>>>>>>>>>> about >>>>>>>>>>>>> it. >>>>>>>>>>>>> >>>>>>>>>>>>> I would like to hear your opinions about this. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> Attaching rebased and updated patches. >>>>>>>>>>> >>>>>>>>>>> Patch 0098: LGTM >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Patch 0099: >>>>>>>>>>> >>>>>>>>>>> a) This check should be done in Server.__init__() rather than >>>>>>>>>>> install_check(): >>>>>>>>>>> >>>>>>>>>>> + if options.ignore_disconnected_topology: >>>>>>>>>>> + print("'--ignore-disconnected-topology' is used only >>>>>>>>>>> during " >>>>>>>>>>> + "uninstallation") >>>>>>>>>>> + sys.exit(1) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> b) >>>>>>>>>>> s/--ignore-disconnected-topology/--ignore-topology-disconnect/, >>>>>>>>>>> for >>>>>>>>>>> consistency with other options, e.g. --no-ui-redirect. >>>>>>>>>>> >>>>>>>>>>> Maybe even shorten it to --ignore-topology? But we probably >>>>>>>>>>> don't >>>>>>>>>>> want >>>>>>>>>>> people to use this option much, so it might be better to keep it >>>>>>>>>>> long? >>>>>>>>>>> >>>>>>>>>> I would rather leave it with the long option name, it is more >>>>>>>>>> apparent >>>>>>>>>> what this switch should be around. >>>>>>>>>>> >>>>>>>>>>> c) I'm fine with uninstall options, you can remove the TODO: >>>>>>>>>>> >>>>>>>>>>> + # TODO: ask jcholast about uninstallation options >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Honza >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Attaching updated patches. >>>>>>>>>> >>>>>>>>> NACK >>>>>>>>> >>>>>>>>> ipa-server-install --uninstall >>>>>>>>> >>>>>>>>> 2015-11-30T17:14:30Z DEBUG Destroyed connection >>>>>>>>> context.ldap2_140081152041808 >>>>>>>>> 2015-11-30T17:14:30Z DEBUG Traceback (most recent call last): >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>>>>>>> line 91, in _handle_exception >>>>>>>>> super(Continuous, self)._handle_exception(exc_info) >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>>> line 387, in _handle_exception >>>>>>>>> six.reraise(*exc_info) >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>>> line 439, in _handle_exception >>>>>>>>> super(ComponentBase, self)._handle_exception(exc_info) >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>>> line 387, in _handle_exception >>>>>>>>> six.reraise(*exc_info) >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>>> line 355, in __runner >>>>>>>>> step() >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>>> line 352, in >>>>>>>>> step = lambda: next(self.__gen) >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>>>> line 81, in run_generator_with_yield_from >>>>>>>>> six.reraise(*exc_info) >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>>>> line 59, in run_generator_with_yield_from >>>>>>>>> value = gen.send(prev_value) >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>>>>>>> line 71, in _uninstall >>>>>>>>> for nothing in self._uninstaller(self.parent): >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> line 1409, in main >>>>>>>>> uninstall_check(self) >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> line 265, in decorated >>>>>>>>> func(installer) >>>>>>>>> File >>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> line 1140, in uninstall_check >>>>>>>>> api, masters, options.ignore_disconnected_topology) >>>>>>>>> AttributeError: 'uninstaller(Server)' object has no attribute >>>>>>>>> 'ignore_disconnected_topology' >>>>>>>>> >>>>>>>>> 2015-11-30T17:14:30Z ERROR 'uninstaller(Server)' object has no >>>>>>>>> attribute >>>>>>>>> 'ignore_disconnected_topology' >>>>>>>>> 2015-11-30T17:14:30Z INFO The ipa-server-install command was >>>>>>>>> successful >>>>>>>>> >>>>>>>> >>>>>>>> Sorry I have failed horribly during option rename. Attaching patch >>>>>>>> that should actually work. >>>>>>>> >>>>>>> functional ACK >>>>>> Attaching rebased patches reflecting the recent changes in the >>>>>> handling >>>>>> of managed topology suffixes handling. >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> Jan had some more suggestions to the patches. Attaching updated >>>>> version. >>>>> >>>>> >>>>> >>>> Attaching updated patch 99 with fixed error message. >>>> >>> Pushed to master: b8c619a7139bd7b65caa03b68431e22791ff19bf >>> >> ACK :-) > > I was doing some unrelated testing with domain level 0 and forgot to > remove --ignore-topology disconnect from my command line before > uninstalling server, which gave me an error that the option cannot be > used in domain level 0 and I had to re-run ipa-server-install > --uninstall with the option removed. > > IMO it would be better for UX if the option was ignored in domain level > 0, since topology disconnects are *always* ignored in domain level 0. > (Right?) > > The attached patch fixes that. > ACK. -- Martin^3 Babinsky From jcholast at redhat.com Wed Dec 9 15:38:43 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 9 Dec 2015 16:38:43 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <56683238.3050700@redhat.com> References: <5667EE2C.2050100@redhat.com> <56683238.3050700@redhat.com> Message-ID: <56684B03.2020905@redhat.com> On 9.12.2015 14:52, Jan Cholasta wrote: > On 9.12.2015 10:02, Jan Cholasta wrote: >> Hi, >> >> the attached patches fix . > > Note that this needs selinux-policy fix to work, so put SELinux into > permissive mode for testing: > . Updated patches attached. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-523.1-build-put-oddjob-scripts-into-separate-directory.patch Type: text/x-patch Size: 2412 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-524.1-replica-install-add-remote-connection-check-over-API.patch Type: text/x-patch Size: 29473 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-525.1-replica-promotion-use-host-credentials-for-connectio.patch Type: text/x-patch Size: 3557 bytes Desc: not available URL: From jcholast at redhat.com Wed Dec 9 15:39:31 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 9 Dec 2015 16:39:31 +0100 Subject: [Freeipa-devel] [PATCHES 509-514] replica promotion: use host credentials when setting up replication In-Reply-To: <566531EF.7000603@redhat.com> References: <564DE017.4020408@redhat.com> <1447951434.5132.27.camel@redhat.com> <564F4105.1040204@redhat.com> <1448038723.5132.59.camel@redhat.com> <5652C63E.5090206@redhat.com> <1448289289.7892.13.camel@redhat.com> <565324C4.1090007@redhat.com> <1448290024.7892.15.camel@redhat.com> <565D823E.5090600@redhat.com> <56604D94.7000705@redhat.com> <566167DE.9090204@redhat.com> <56619F5C.5000900@redhat.com> <5661A04E.9080409@redhat.com> <5661A593.5030003@redhat.com> <1449244714.3445.67.camel@redhat.com> <56649B78.1080805@redhat.com> <566531EF.7000603@redhat.com> Message-ID: <56684B33.3090103@redhat.com> On 7.12.2015 08:14, Jan Cholasta wrote: > On 6.12.2015 21:32, Martin Basti wrote: >> >> >> On 04.12.2015 16:58, Simo Sorce wrote: >>> On Fri, 2015-12-04 at 15:39 +0100, Jan Cholasta wrote: >>>> On 4.12.2015 15:16, Jan Cholasta wrote: >>>>> On 4.12.2015 15:12, Jan Cholasta wrote: >>>>>> On 4.12.2015 11:15, Petr Vobornik wrote: >>>>>>> On 12/03/2015 03:11 PM, Martin Basti wrote: >>>>>>>> >>>>>>>> On 01.12.2015 12:19, Jan Cholasta wrote: >>>>>>>>> On 23.11.2015 15:47, Simo Sorce wrote: >>>>>>>>>> On Mon, 2015-11-23 at 15:37 +0100, Jan Cholasta wrote: >>>>>>>>>>> Ad alternative is to add the host to ipaservers before the >>>>>>>>>>> checks >>>>>>>>>>> are >>>>>>>>>>> done and remove it again if any of them fail. >>>>>>>>>> Too error prone, I am ok with the current way in your patches >>>>>>>>>> until/unless I can think of a fail safe way. :-) >>>>>>>>> Updated patches attached. Note that 520 should be applied >>>>>>>>> between 509 >>>>>>>>> and 510. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> Functional ACK >>>>>>>> >>>>>>> Simo, do you want to review the ACIs or other things it the >>>>>>> patches? Or >>>>>>> can the patches be pushed? >>>>>> There were no changes in the ACIs since last time. >>>>> Actually, memberPrincipal was removed from the "IPA server hosts can >>>>> manage own Custodia secrets" ACI, as per Simo's request. >>>>> >>>>>> Rebased patches attached. >>>> Note that 520 should still be applied between 509 and 510. >>>> >>> LGTM >>> >> ACK > > Thanks. > > Pushed to master: 01ddf51df76f3298499973355c5461727e46ab5b Martin Babinsky found out that ipaservers is not created early enough when installing a replica of a 4.2 or older server which causes a crash. The attached patch fixes that. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-527-replica-install-add-ipaservers-before-the-server-s-h.patch Type: text/x-patch Size: 2022 bytes Desc: not available URL: From pspacek at redhat.com Wed Dec 9 17:55:10 2015 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 9 Dec 2015 18:55:10 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <56682093.3010505@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> <5665843C.2090507@redhat.com> <56658C89.8050103@redhat.com> <56668D30.8060308@redhat.com> <5666BCBC.9040005@redhat.com> <5666E822.90103@redhat.com> <56682093.3010505@redhat.com> Message-ID: <56686AFE.4000401@redhat.com> On 9.12.2015 13:37, David Kupka wrote: > On 08/12/15 15:24, Petr Spacek wrote: >> On 8.12.2015 12:19, David Kupka wrote: >>> On 08/12/15 08:56, Petr Spacek wrote: >>>> On 7.12.2015 14:41, David Kupka wrote: >>>>> +def is_host_resolvable(fqdn): >>>>> + if not isinstance(fqdn, DNSName): >>>>> + fqdn = DNSName(fqdn) >>>>> + for rdtype in (rdatatype.A, rdatatype.AAAA): >>>>> + try: >>>>> + resolver.query(fqdn.make_absolute(), rdtype) >>>>> + except DNSException: >>>>> + continue >>>>> + else: >>>>> + return True >>>>> + >>>>> + return False >>>>> >>>> >>>> NACK, you are re-introducing duplicate function which was removed in >>>> 498471e4aed1367b72cd74d15811d0584a6ee268. >>>> >>>> Please amend the patch ASAP to use new verify_host_resolvable() function so I >>>> can test it and get it into 4.3. >>>> >>> Updated patches attached. >> >> Hmm, my review script screams: >> >> Detected base branch: origin/master >> Checks will be made against following base commit: 848912a add missing >> /ipaplatform/constants.py to .gitignore >> Writing API to API.txt >> ./ipalib/plugins/dns.py:2127:9: E124 closing bracket does not match visual >> indentation >> ./ipalib/plugins/dns.py:2711:13: E128 continuation line under-indented for >> visual indent >> ./ipalib/plugins/dns.py:2713:9: E124 closing bracket does not match visual >> indentation >> ./ipalib/plugins/dns.py:2716:13: E128 continuation line under-indented for >> visual indent >> ./ipapython/ipautil.py:928:1: E302 expected 2 blank lines, found 1 >> ./ipapython/ipautil.py:948:17: E128 continuation line under-indented for >> visual indent >> ./ipapython/ipautil.py:956:1: E302 expected 2 blank lines, found 1 >> ./ipapython/ipautil.py:997:80: E501 line too long (83 > 79 characters) >> ./ipapython/ipautil.py:998:80: E501 line too long (83 > 79 characters) >> ./ipaserver/install/bindinstance.py:49:9: E128 continuation line >> under-indented for visual indent >> ./ipaserver/install/bindinstance.py:292:80: E501 line too long (80 > 79 >> characters) >> ./ipaserver/install/bindinstance.py:438:19: E128 continuation line >> under-indented for visual indent >> ./ipaserver/install/server/common.py:271:63: E261 at least two spaces before >> inline comment >> ./ipaserver/install/server/common.py:271:80: E501 line too long (90 > 79 >> characters) >> ERROR: PEP8 --diff failed, continuing ... >> ERROR: API.txt was changed without a change in VERSION, continuing ... >> Summary of detected errors: >> ERROR: PEP8 --diff failed >> ERROR: API.txt was changed without a change in VERSION >> >> Please fix it :-) >> >> Make make this more pleasant for you, you can use (and review) my attached >> patch. It adds 'review' target to Makefile, it will do the same checks as I do. >> > > Unfortunately your tool does not work for me (output w/ -o xtrace attached). > Anyway I have incremented API version and fixed most of the pep8 errors except > for E124 twice in ipalib/plugins/dns.py as it seems to be convention in the > file and E501 twice in ipapython/ipautil.py because IMO breaking string > constant into multiple lines does not help readability. > > Updated patches also attached. We are almost there, but NACK for now. 1) Following attempt to install IPA explodes. Please note that dom-245.idm.lab.eng.brq.redhat.com DNS domain is delegated to the IPA server before installation is started, so it gives 'timeout' or possibly SERVFAIL. # ipa-server-install --ds-password=root4lab --admin-password=root4lab --setup-dns --forwarder=10.38.5.26 --no-reverse --allow-zone-overlap --domain=dom-245.idm.lab.eng.brq.redhat.com --realm=DOM-245.IDM.LAB.ENG.BRQ.REDHAT.COM [...] Configuring DNS (named) [1/11]: generating rndc key file [2/11]: adding DNS container [3/11]: setting up our zone [error] InvocationError: DNS check for domain dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation timed out after 30.000453949 seconds. Please make sure that the domain is properly delegated to this IPA server. ipa.ipapython.install.cli.install_tool(Server): ERROR DNS check for domain dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation timed out after 30.000453949 seconds. Please make sure that the domain is properly delegated to this IPA server. ipa.ipapython.install.cli.install_tool(Server): ERROR The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information 2015-12-09T17:15:37Z DEBUG File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in execute return_value = self.run() File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line 318, in run cfgr.run() File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 310, in run self.execute() File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 332, in execute for nothing in self._executor(): File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, in __runner self._handle_exception(exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception six.reraise(*exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, in __runner step() File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, in step = lambda: next(self.__gen) File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from six.reraise(*exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from value = gen.send(prev_value) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 571, in _configure next(executor) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, in __runner self._handle_exception(exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 449, in _handle_exception self.__parent._handle_exception(exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception six.reraise(*exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 446, in _handle_exception super(ComponentBase, self)._handle_exception(exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception six.reraise(*exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, in __runner step() File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, in step = lambda: next(self.__gen) File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from six.reraise(*exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from value = gen.send(prev_value) File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line 63, in _install for nothing in self._installer(self.parent): File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", line 1471, in main install(self) File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", line 265, in decorated func(installer) File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", line 958, in install dns.install(False, False, options) File "/usr/lib/python2.7/site-packages/ipaserver/install/dns.py", line 322, in install bind.create_instance() File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", line 680, in create_instance self.start_creation() File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 447, in start_creation run_step(full_msg, method) File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 437, in run_step method() File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", line 805, in __setup_zone ns_hostname=self.api.env.host, force=True, api=self.api) File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", line 331, in add_zone force=force) File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 447, in __call__ ret = self.run(*args, **options) File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 764, in run return self.execute(*args, **options) File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line 2781, in execute result = super(dnszone_add, self).execute(*keys, **options) File "/usr/lib/python2.7/site-packages/ipalib/plugins/baseldap.py", line 1233, in execute *keys, **options) File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line 2747, in pre_callback ldap, dn, entry_attrs, attrs_list, *keys, **options) File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line 2153, in pre_callback raise errors.InvocationError(e.message) 2) Please print 'Checking DNS domain , please wait ...' when validating domain parameter in installer. The timeout is 30 seconds and users may be nervous when the installed blocks for 30 seconds without a visible reason. Precedent for this is in check_forwarders() in ipaserver/install/bindinstance.py . Encapsulating check_zone_overlap() in auxiliary method may be an option. 3) Timeout is a fatal error instead of warning. We have been discussing this and it should really be just a warning. 4) Nitpicks are attached in .patch file. 5) ipa dnszone-add checks work as expected, good job! Thank you for patience! -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: nits.patch Type: text/x-patch Size: 1554 bytes Desc: not available URL: From fskola at redhat.com Wed Dec 9 18:01:02 2015 From: fskola at redhat.com (Filip =?UTF-8?B?xaBrb2xh?=) Date: Wed, 9 Dec 2015 19:01:02 +0100 Subject: [Freeipa-devel] [PATCH 0002] Refactor test_group_plugin In-Reply-To: <5665B88E.4000507@redhat.com> References: <20151120135636.71171d5c@vor2.netbox.priv> <20151123145934.36901470@dhcp-25-21.brq.redhat.com> <20151123164249.4a576706@dhcp-25-21.brq.redhat.com> <565C76D6.3010406@redhat.com> <20151203201556.4ff69dba@vor2.netbox.priv> <5665B88E.4000507@redhat.com> Message-ID: <20151209190102.1bb8fc07@vor2.netbox.priv> On Mon, 7 Dec 2015 17:49:18 +0100 Milan Kub?k wrote: > On 12/03/2015 08:15 PM, Filip ?kola wrote: > > On Mon, 30 Nov 2015 17:18:30 +0100 > > Milan Kub?k wrote: > > > >> On 11/23/2015 04:42 PM, Filip ?kola wrote: > >>> Sending updated patch. > >>> > >>> F. > >>> > >>> On Mon, 23 Nov 2015 14:59:34 +0100 > >>> Filip ?kola wrote: > >>> > >>>> Found couple of issues (broke some dependencies). > >>>> > >>>> NACK > >>>> > >>>> F. > >>>> > >>>> On Fri, 20 Nov 2015 13:56:36 +0100 > >>>> Filip ?kola wrote: > >>>> > >>>>> Another one. > >>>>> > >>>>> F. > >>> > >> Hi, the tests look good. Few remarks, though. > >> > >> 1. Please, use the shortes copyright notice in new modules. > >> > >> # > >> # Copyright (C) 2015 FreeIPA Contributors see COPYING for > >> license # > >> > >> 2. The tests `test_group_remove_group_from_protected_group` and > >> `test_group_full_set_of_objectclass_not_available_post_detach` > >> were not ported. Please, include them in the patch. > >> > >> Also, for less hassle, please rebase your patches on top of > >> freeipa-mkubik-0025-3-Separated-Tracker-implementations-into-standalone-pa.patch > >> Which changes the location of tracker implementations and prevents > >> circular imports. > >> > >> Thanks. > >> > > > > > > Hi, > > > > these cases are there, in corresponding classes. They are marked > > with the original comments. (However I can move them to separate > > class if desirable.) > > > > The copyright notice is changed. Also included a few changes in the > > test with user without private group. > > > > Filip > NACK > > linter: > ************* Module tracker.group_plugin > ipatests/test_xmlrpc/tracker/group_plugin.py:257: > [E0102(function-redefined), GroupTracker.check_remove_member] method > already defined line 253) > > Probably a leftover after the rebase made on top of my patch. Please > fix it. You can check youch changes by make-lint script before > sending them. > > Thanks > Hi, I learned to use make-lint! Thanks, F. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-fskola-0002-4-Refactor-test_group_plugin.patch Type: text/x-patch Size: 74435 bytes Desc: not available URL: From mbasti at redhat.com Wed Dec 9 18:01:22 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 9 Dec 2015 19:01:22 +0100 Subject: [Freeipa-devel] [PATCH 0394] topology: Make sure the old 'realm' topology suffix is not In-Reply-To: <5667E996.2070900@redhat.com> References: <5666DAE6.3030708@redhat.com> <5666FD0D.6070001@redhat.com> <56670611.5090604@redhat.com> <5667E996.2070900@redhat.com> Message-ID: <56686C72.4060900@redhat.com> On 09.12.2015 09:43, Martin Basti wrote: > > > On 08.12.2015 17:32, Martin Babinsky wrote: >> On 12/08/2015 04:53 PM, Tomas Babej wrote: >>> >>> >>> On 12/08/2015 02:28 PM, Tomas Babej wrote: >>>> Hi, >>>> >>>> The old 'realm' topology suffix is no longer used, however, it was >>>> being >>>> created on masters with version 4.2.3 and later. Make sure it's >>>> properly >>>> removed. >>>> >>>> Note that this is not the case for the 'ipaca' suffix, which was later >>>> removed to 'ca'. >>>> >>>> https://fedorahosted.org/freeipa/ticket/5526 >>>> >>> >>> Actually, we found out with Martin that this patch deletes realm >>> instead >>> of domain suffix, against all initial impressions. >>> >>> Updated patch attached. >>> >>> Tomas >>> >>> >>> >> >> Works for me, ACK. >> >> I have also made some hardening in topology connectivity checks so >> that this kind of situations is handled and reported by them. I will >> send a patch in separate thread. >> > Pushed to master: a84b7d2117aafc5182640d0a22675b214c27dd7c > I accidentally pushed first revision of the patch, fix pushed to master: dcb5c2a5200a797b0eec9bb809c570f9ed80f7bb From mbasti at redhat.com Wed Dec 9 18:22:44 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 9 Dec 2015 19:22:44 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison Message-ID: <56687174.8000305@redhat.com> https://fedorahosted.org/freeipa/ticket/5535 Patch attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0373-Fix-version-comparison.patch Type: text/x-patch Size: 2417 bytes Desc: not available URL: From simo at redhat.com Wed Dec 9 19:18:23 2015 From: simo at redhat.com (Simo Sorce) Date: Wed, 9 Dec 2015 14:18:23 -0500 (EST) Subject: [Freeipa-devel] [PATCH 560] Allow to set allowed krb authz data type per user In-Reply-To: <20151201080732.GI9605@redhat.com> References: <1448406570.29102.26.camel@redhat.com> <56555E8C.5020608@redhat.com> <1448473396.17462.4.camel@redhat.com> <20151201080732.GI9605@redhat.com> Message-ID: <1934255375.26292682.1449688703387.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Alexander Bokovoy" > To: "Simo Sorce" > Cc: "Jan Cholasta" , "freeipa-devel" > Sent: Tuesday, December 1, 2015 3:07:32 AM > Subject: Re: [Freeipa-devel] [PATCH 560] Allow to set allowed krb authz data type per user > > On Wed, 25 Nov 2015, Simo Sorce wrote: > >On Wed, 2015-11-25 at 08:09 +0100, Jan Cholasta wrote: > >> On 25.11.2015 00:09, Simo Sorce wrote: > >> > This patch is untested and mostly an RFC. > >> > > >> > I think it is all we need to allow to specify authz data types per user > >> > and by setting the attribute to NONE preventing a user from getting > >> > MS-PAC data in their ticket. > >> > > >> > Alexander you changed quite a bit the code around here so I'd like to > >> > know if you think the change I made in the KDC will cause any issue with > >> > the special PACs we generate for master's principals. As far as I can > >> > tell it shouldn't. > >> > > >> > Any opinion is welcome. > >> > >> Before your change, the server entry was checked for AS requests, now > >> only the client entry is checked for AS requests. I'm not very familiar > >> with ipa-kdb, but shouldn't the server entry still be checked as a > >> fallback when there is no authorization data in the client entry? > > > >This is partly why I CCed Alexander, the way the get function works is > >that it will get policy on the entry itself and if nothing is there it > >will try with the global policy, so in both cases the global policy is > >sourced as fallback. > > > >For AS requests though you are generally asking for a TGT so the > >"server" is the krbtgt entry that has no policy. It is through though > >that a client *can* ask for a ticket directly via an AS request, that is > >uncommon and it is unclear to me what we should do in that case if > >client and server have incompatible options. > > > >Well this is why it is a RFC after all :) > Can we source global policy for the direct AS request as well? I think I would do this in a separate patch. > >> The attribute is exposed in the service plugin, shouldn't it be exposed > >> in the user plugin as well? > > > >I didn't do it on purpose yet but eventually we may want to expose it, > >indeed. The reason I didn't is that we may want to use something like > >CoS to populate the attribute based on group membership and I am not > >sure we want to expose it per user, up top debate. > I don't want to expose it in the config too. Agreed. attached find an updated patch as I found a crash bug with the older one in some situations. Simo. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-560-2-Allow-to-specify-Kerberos-authz-data-type-per-user.patch Type: text/x-patch Size: 4494 bytes Desc: not available URL: From pvoborni at redhat.com Wed Dec 9 19:51:18 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 9 Dec 2015 20:51:18 +0100 Subject: [Freeipa-devel] [PATCHES 516-517] spec file: put Python modules into standalone packages In-Reply-To: <5665A403.9090604@redhat.com> References: <5665A403.9090604@redhat.com> Message-ID: <56688636.40803@redhat.com> On 12/07/2015 04:21 PM, Jan Cholasta wrote: > Hi, > > the attached patches partially fix > . This is done to allow > the addition of Python 3 packages, see > . > See commit messages for more information. > > In order to test: > 1. make rpms > 2. > > > 3. Test with both dnf and yum-deprecated. > > Beware that when you run "yum-deprecated clean all", it does not remove > cache for the on-disk repository created in step 2, you have to remove > the /var/cache/yum/$basearch/$releasever/$reponame directory manually. > > Honza > Shouldn't freeipa-server-dns and freeipa-server-trust-add depend on freeipa-server? They do not in this patch. IMO they should. following updates work (all on f23, update from 4.2.3): dnf update dnf update freeipa-* yum-depracated update freeipa-* for both client or server with all packages. but when I tried to install only client "dnf install freeipa-client" and then following failed: dnf update freeipa-client The difference was: Installing: freeipa-client-common noarch freeipa-common noarch python2-ipaclient noarch python2-ipalib x86_64 Upgrading: freeipa-client Works: Installing: freeipa-client-common noarch freeipa-common noarch freeipa-python-compat noarch replacing freeipa-python.x86_64 4.2.3-1.1.fc23 python2-ipaclient noarch python2-ipalib x86_64 Upgrading: freeipa-client not sure if it is a problem, otherwise the patch looks OK. -- Petr Vobornik From simo at redhat.com Wed Dec 9 20:00:41 2015 From: simo at redhat.com (Simo Sorce) Date: Wed, 9 Dec 2015 15:00:41 -0500 (EST) Subject: [Freeipa-devel] [PATCH 560] Allow to set allowed krb authz data type per user In-Reply-To: <1934255375.26292682.1449688703387.JavaMail.zimbra@redhat.com> References: <1448406570.29102.26.camel@redhat.com> <56555E8C.5020608@redhat.com> <1448473396.17462.4.camel@redhat.com> <20151201080732.GI9605@redhat.com> <1934255375.26292682.1449688703387.JavaMail.zimbra@redhat.com> Message-ID: <1892639799.26312584.1449691241103.JavaMail.zimbra@redhat.com> Sent the wrong patch, attached the one that actually compiles. ----- Original Message ----- > From: "Simo Sorce" > To: "Alexander Bokovoy" > Cc: "Simo Sorce" , "Jan Cholasta" , "freeipa-devel" > Sent: Wednesday, December 9, 2015 2:18:23 PM > Subject: Re: [Freeipa-devel] [PATCH 560] Allow to set allowed krb authz data type per user > > ----- Original Message ----- > > From: "Alexander Bokovoy" > > To: "Simo Sorce" > > Cc: "Jan Cholasta" , "freeipa-devel" > > > > Sent: Tuesday, December 1, 2015 3:07:32 AM > > Subject: Re: [Freeipa-devel] [PATCH 560] Allow to set allowed krb authz > > data type per user > > > > On Wed, 25 Nov 2015, Simo Sorce wrote: > > >On Wed, 2015-11-25 at 08:09 +0100, Jan Cholasta wrote: > > >> On 25.11.2015 00:09, Simo Sorce wrote: > > >> > This patch is untested and mostly an RFC. > > >> > > > >> > I think it is all we need to allow to specify authz data types per > > >> > user > > >> > and by setting the attribute to NONE preventing a user from getting > > >> > MS-PAC data in their ticket. > > >> > > > >> > Alexander you changed quite a bit the code around here so I'd like to > > >> > know if you think the change I made in the KDC will cause any issue > > >> > with > > >> > the special PACs we generate for master's principals. As far as I can > > >> > tell it shouldn't. > > >> > > > >> > Any opinion is welcome. > > >> > > >> Before your change, the server entry was checked for AS requests, now > > >> only the client entry is checked for AS requests. I'm not very familiar > > >> with ipa-kdb, but shouldn't the server entry still be checked as a > > >> fallback when there is no authorization data in the client entry? > > > > > >This is partly why I CCed Alexander, the way the get function works is > > >that it will get policy on the entry itself and if nothing is there it > > >will try with the global policy, so in both cases the global policy is > > >sourced as fallback. > > > > > >For AS requests though you are generally asking for a TGT so the > > >"server" is the krbtgt entry that has no policy. It is through though > > >that a client *can* ask for a ticket directly via an AS request, that is > > >uncommon and it is unclear to me what we should do in that case if > > >client and server have incompatible options. > > > > > >Well this is why it is a RFC after all :) > > Can we source global policy for the direct AS request as well? > > I think I would do this in a separate patch. > > > >> The attribute is exposed in the service plugin, shouldn't it be exposed > > >> in the user plugin as well? > > > > > >I didn't do it on purpose yet but eventually we may want to expose it, > > >indeed. The reason I didn't is that we may want to use something like > > >CoS to populate the attribute based on group membership and I am not > > >sure we want to expose it per user, up top debate. > > I don't want to expose it in the config too. > > Agreed. > > attached find an updated patch as I found a crash bug with the older one in > some situations. > > Simo. > -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-560-3-Allow-to-specify-Kerberos-authz-data-type-per-user.patch Type: text/x-patch Size: 4485 bytes Desc: not available URL: From jcholast at redhat.com Thu Dec 10 06:50:24 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 10 Dec 2015 07:50:24 +0100 Subject: [Freeipa-devel] [PATCH 0071] replica: Fix ipa-replica-install with replica file (domain, level 0). In-Reply-To: <5667D8E0.6070303@redhat.com> References: <5666E31E.1030705@redhat.com> <5666F54C.80801@redhat.com> <5666F831.5040705@redhat.com> <5667D8E0.6070303@redhat.com> Message-ID: <566920B0.90303@redhat.com> On 9.12.2015 08:31, David Kupka wrote: > On 08/12/15 16:33, Tomas Babej wrote: >> >> >> On 12/08/2015 04:20 PM, Oleg Fayans wrote: >>> ACK. The initial issue is fixed. >>> >>> On 12/08/2015 03:03 PM, David Kupka wrote: >>>> https://fedorahosted.org/freeipa/ticket/5531 >>>> >>>> >>> >> >> Can we get some more love for the patch and provide at least a sentence >> worth of commit message before pushing? >> >> It's not obvious from the title what the patch does, other than it fixes >> things. >> >> Tomas >> > I believe it's pretty obvious from linked ticket and attached patch > changing just 5 lines. > But you're right verbosity in commit message could save time later. > Patch with changed commit message attached. ACK. Pushed to master: b7953cda4fc02637f6e3db574b3d7163efc78a98 -- Jan Cholasta From jcholast at redhat.com Thu Dec 10 06:57:24 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 10 Dec 2015 07:57:24 +0100 Subject: [Freeipa-devel] [PATCHES 509-514] replica promotion: use host credentials when setting up replication In-Reply-To: <56684B33.3090103@redhat.com> References: <564DE017.4020408@redhat.com> <1447951434.5132.27.camel@redhat.com> <564F4105.1040204@redhat.com> <1448038723.5132.59.camel@redhat.com> <5652C63E.5090206@redhat.com> <1448289289.7892.13.camel@redhat.com> <565324C4.1090007@redhat.com> <1448290024.7892.15.camel@redhat.com> <565D823E.5090600@redhat.com> <56604D94.7000705@redhat.com> <566167DE.9090204@redhat.com> <56619F5C.5000900@redhat.com> <5661A04E.9080409@redhat.com> <5661A593.5030003@redhat.com> <1449244714.3445.67.camel@redhat.com> <56649B78.1080805@redhat.com> <566531EF.7000603@redhat.com> <56684B33.3090103@redhat.com> Message-ID: <56692254.3060209@redhat.com> On 9.12.2015 16:39, Jan Cholasta wrote: > On 7.12.2015 08:14, Jan Cholasta wrote: >> On 6.12.2015 21:32, Martin Basti wrote: >>> >>> >>> On 04.12.2015 16:58, Simo Sorce wrote: >>>> On Fri, 2015-12-04 at 15:39 +0100, Jan Cholasta wrote: >>>>> On 4.12.2015 15:16, Jan Cholasta wrote: >>>>>> On 4.12.2015 15:12, Jan Cholasta wrote: >>>>>>> On 4.12.2015 11:15, Petr Vobornik wrote: >>>>>>>> On 12/03/2015 03:11 PM, Martin Basti wrote: >>>>>>>>> >>>>>>>>> On 01.12.2015 12:19, Jan Cholasta wrote: >>>>>>>>>> On 23.11.2015 15:47, Simo Sorce wrote: >>>>>>>>>>> On Mon, 2015-11-23 at 15:37 +0100, Jan Cholasta wrote: >>>>>>>>>>>> Ad alternative is to add the host to ipaservers before the >>>>>>>>>>>> checks >>>>>>>>>>>> are >>>>>>>>>>>> done and remove it again if any of them fail. >>>>>>>>>>> Too error prone, I am ok with the current way in your patches >>>>>>>>>>> until/unless I can think of a fail safe way. :-) >>>>>>>>>> Updated patches attached. Note that 520 should be applied >>>>>>>>>> between 509 >>>>>>>>>> and 510. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> Functional ACK >>>>>>>>> >>>>>>>> Simo, do you want to review the ACIs or other things it the >>>>>>>> patches? Or >>>>>>>> can the patches be pushed? >>>>>>> There were no changes in the ACIs since last time. >>>>>> Actually, memberPrincipal was removed from the "IPA server hosts can >>>>>> manage own Custodia secrets" ACI, as per Simo's request. >>>>>> >>>>>>> Rebased patches attached. >>>>> Note that 520 should still be applied between 509 and 510. >>>>> >>>> LGTM >>>> >>> ACK >> >> Thanks. >> >> Pushed to master: 01ddf51df76f3298499973355c5461727e46ab5b > > Martin Babinsky found out that ipaservers is not created early enough > when installing a replica of a 4.2 or older server which causes a crash. > > The attached patch fixes that. Actually I don't like how I fixed that, here's an updated patch. Also, I noticed that replica promotion fails too late in domain level 0. Fixed as well. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-527.1-replica-install-add-ipaservers-if-it-does-not-exist.patch Type: text/x-patch Size: 1678 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-528-replica-promotion-check-domain-level-before-ipaserve.patch Type: text/x-patch Size: 2474 bytes Desc: not available URL: From lslebodn at redhat.com Thu Dec 10 08:05:13 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Thu, 10 Dec 2015 09:05:13 +0100 Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <5666DF65.1030309@redhat.com> References: <20151130163256.GA7759@mail.corp.redhat.com> <565DD881.60401@redhat.com> <565EAAAE.8020306@redhat.com> <565F0332.9000804@redhat.com> <20151202162519.GA19962@mail.corp.redhat.com> <56601AE8.2020100@redhat.com> <5660528E.40904@redhat.com> <149447113.23167965.1449156368210.JavaMail.zimbra@redhat.com> <566060E2.8030903@redhat.com> <5666DF65.1030309@redhat.com> Message-ID: <20151210080513.GA21550@mail.corp.redhat.com> On (08/12/15 14:47), Tomas Babej wrote: > > >On 12/03/2015 04:33 PM, Tomas Babej wrote: >> >> >> On 12/03/2015 04:26 PM, Ale? Mare?ek wrote: >>> Hello, >>> >>> ACK for code >>> NACK for the placing "get_client_ip_with_hostmask" function to test_sudo.py (this function should be in some more general file) >>> >> >> What place would you propose? The task.py is not a good place, as this >> is not really a task. >> >> Nevertheless, I'd rather have it moved when an use case outside >> test_sudo.py actually arises. Right now it would lead to unnecessary >> cluttering. >> >> Tomas >> > >I re-discovered ipatests.test_integration.util (two years after I >created it :D) - which seemed ideal for this function. > >Updated patch attached. > >Tomas >From 33552d6078d75ee99f9ec19ae143df5a61ba84a4 Mon Sep 17 00:00:00 2001 >From: Tomas Babej >Date: Wed, 2 Dec 2015 15:25:49 +0100 >Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on > hostmask > >IPA sudo tests worked under the assumption that the clients >that are executing the sudo commands have their IPs assigned >within 255.255.255.0 hostmask. > >Removes this (invalid) assumption and adds a >dynamic detection of the hostmask of the IPA client. > >https://fedorahosted.org/freeipa/ticket/5501 >--- > ipatests/test_integration/test_sudo.py | 33 +++++++++++++++++++++++++++------ > ipatests/test_integration/util.py | 16 ++++++++++++++++ > 2 files changed, 43 insertions(+), 6 deletions(-) > >diff --git a/ipatests/test_integration/util.py b/ipatests/test_integration/util.py >index 1a1bb3fcc923c9f2721f0a4c1cb7a1ba2ccc2dd8..187f39e80e84af0eb4938fb19ac3d3c7c2280ed9 100644 >--- a/ipatests/test_integration/util.py >+++ b/ipatests/test_integration/util.py >@@ -58,3 +58,19 @@ def run_repeatedly(host, command, assert_zero_rc=True, test=None, > .format(cmd=' '.join(command), > times=timeout / time_step, > timeout=timeout)) >+ >+ >+def get_host_ip_with_hostmask(host): >+ """ >+ Detects the IP of the host including the hostmask. >+ >+ Returns None if the IP could not be detected. >+ """ >+ >+ ip = host.ip >+ result = host.run_command(['ip', 'addr']) >+ full_ip_regex = r'(?P%s/\d{1,2}) ' % re.escape(ip) >+ match = re.search(full_ip_regex, result.stdout_text) ./make-lint ************* Module ipatests.test_integration.util ipatests/test_integration/util.py:72: [E0602(undefined-variable), get_host_ip_with_hostmask] Undefined variable 're') ipatests/test_integration/util.py:73: [E0602(undefined-variable), get_host_ip_with_hostmask] Undefined variable 're') =============================================================================== Errors were found during the static code check. If you are certain that any of the reported errors are false positives, please mark them in the source code according to the pylint documentation. =============================================================================== Makefile:124: recipe for target 'lint' failed LS From lslebodn at redhat.com Thu Dec 10 08:13:58 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Thu, 10 Dec 2015 09:13:58 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <56687174.8000305@redhat.com> References: <56687174.8000305@redhat.com> Message-ID: <20151210081357.GB21550@mail.corp.redhat.com> On (09/12/15 19:22), Martin Basti wrote: >https://fedorahosted.org/freeipa/ticket/5535 > >Patch attached. >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 00:00:00 2001 >From: Martin Basti >Date: Wed, 9 Dec 2015 18:53:35 +0100 >Subject: [PATCH] Fix version comparison > >Use RPM library to compare vendor versions of IPA for redhat platform > >https://fedorahosted.org/freeipa/ticket/5535 >--- > freeipa.spec.in | 2 ++ > ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ > 2 files changed, 21 insertions(+) > >diff --git a/freeipa.spec.in b/freeipa.spec.in >index 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 100644 >--- a/freeipa.spec.in >+++ b/freeipa.spec.in >@@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} > Requires: gzip > Requires: python-gssapi >= 1.1.0 > Requires: custodia >+Requires: rpm-python >+Requires: rpmdevtools Could you explain why do you need the 2nd package? It does not contains any python modules and I cannot see usage of any binary in this patch LS From jcholast at redhat.com Thu Dec 10 08:48:09 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 10 Dec 2015 09:48:09 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <56684B03.2020905@redhat.com> References: <5667EE2C.2050100@redhat.com> <56683238.3050700@redhat.com> <56684B03.2020905@redhat.com> Message-ID: <56693C49.6030508@redhat.com> On 9.12.2015 16:38, Jan Cholasta wrote: > On 9.12.2015 14:52, Jan Cholasta wrote: >> On 9.12.2015 10:02, Jan Cholasta wrote: >>> Hi, >>> >>> the attached patches fix . >> >> Note that this needs selinux-policy fix to work, so put SELinux into >> permissive mode for testing: >> . > > Updated patches attached. I screwed up a change in patch 524 and accidentally included a chunk of code in patch 525 that doesn't belong in it. Updated patches attached. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-523.2-build-put-oddjob-scripts-into-separate-directory.patch Type: text/x-patch Size: 2412 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-524.2-replica-install-add-remote-connection-check-over-API.patch Type: text/x-patch Size: 29543 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-525.2-replica-promotion-use-host-credentials-for-connectio.patch Type: text/x-patch Size: 2344 bytes Desc: not available URL: From jcholast at redhat.com Thu Dec 10 08:51:02 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 10 Dec 2015 09:51:02 +0100 Subject: [Freeipa-devel] [PATCHES 529-530] ca install: use host credentials in domain level 1 Message-ID: <56693CF6.8030104@redhat.com> Hi, the attached patches fix . My patches 523-525 are required for this: . Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-529-aci-merge-domain-and-CA-suffix-replication-agreement.patch Type: text/x-patch Size: 5555 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-530-ca-install-use-host-credentials-in-domain-level-1.patch Type: text/x-patch Size: 4418 bytes Desc: not available URL: From mbabinsk at redhat.com Thu Dec 10 09:14:50 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Thu, 10 Dec 2015 10:14:50 +0100 Subject: [Freeipa-devel] [PATCH 0113] properly add ACIs to custodia container during IPA upgrade In-Reply-To: <5666A6B7.8010705@redhat.com> References: <5666A6B7.8010705@redhat.com> Message-ID: <5669428A.4070303@redhat.com> On 12/08/2015 10:45 AM, Martin Babinsky wrote: > fixes https://fedorahosted.org/freeipa/ticket/5524 > > > Attaching updated patch with simpler fix suggested by Jan. -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0113.1-add-ACIs-for-custodia-container-to-its-parent-during.patch Type: text/x-patch Size: 1893 bytes Desc: not available URL: From mbasti at redhat.com Thu Dec 10 10:23:13 2015 From: mbasti at redhat.com (Martin Basti) Date: Thu, 10 Dec 2015 11:23:13 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <20151210081357.GB21550@mail.corp.redhat.com> References: <56687174.8000305@redhat.com> <20151210081357.GB21550@mail.corp.redhat.com> Message-ID: <56695291.5090705@redhat.com> On 10.12.2015 09:13, Lukas Slebodnik wrote: > On (09/12/15 19:22), Martin Basti wrote: >> https://fedorahosted.org/freeipa/ticket/5535 >> >> Patch attached. > >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 00:00:00 2001 >> From: Martin Basti >> Date: Wed, 9 Dec 2015 18:53:35 +0100 >> Subject: [PATCH] Fix version comparison >> >> Use RPM library to compare vendor versions of IPA for redhat platform >> >> https://fedorahosted.org/freeipa/ticket/5535 >> --- >> freeipa.spec.in | 2 ++ >> ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ >> 2 files changed, 21 insertions(+) >> >> diff --git a/freeipa.spec.in b/freeipa.spec.in >> index 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 100644 >> --- a/freeipa.spec.in >> +++ b/freeipa.spec.in >> @@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} >> Requires: gzip >> Requires: python-gssapi >= 1.1.0 >> Requires: custodia >> +Requires: rpm-python >> +Requires: rpmdevtools > Could you explain why do you need the 2nd package? > It does not contains any python modules > and I cannot see usage of any binary in this patch > > LS Thanks for this catch, it is actually located in yum package, I rather copy stringToVersion function from there to IPA, to avoid dependency hell Updated patch attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0373.2-Fix-version-comparison.patch Type: text/x-patch Size: 3212 bytes Desc: not available URL: From fskola at redhat.com Thu Dec 10 10:29:52 2015 From: fskola at redhat.com (Filip =?UTF-8?B?xaBrb2xh?=) Date: Thu, 10 Dec 2015 11:29:52 +0100 Subject: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin In-Reply-To: <869710129.24860544.1449498571599.JavaMail.zimbra@redhat.com> References: <20151106113201.5e6a7042@dhcp-24-123.brq.redhat.com> <564092F6.9000800@redhat.com> <20151109163512.3d3beb5f@redhat.com> <5641BE6D.4060608@redhat.com> <20151110121350.06295a68@dhcp-24-103.brq.redhat.com> <56448E17.2010409@redhat.com> <20151203173843.37179230@vor2.netbox.priv> <869710129.24860544.1449498571599.JavaMail.zimbra@redhat.com> Message-ID: <20151210112952.3ddb7136@vor2.netbox.priv> Hi, this if fixed. Also issues with test_stageuser_plugin caused by UserTracker changes should be fixed here. Filip On Mon, 7 Dec 2015 09:29:31 -0500 (EST) Ale? Mare?ek wrote: > NACK. > > $ ./make-lint > ************* Module ipatests.test_xmlrpc.test_user_plugin > ipatests/test_xmlrpc/test_user_plugin.py:42: > [E0611(no-name-in-module), ] No name 'ldaptracker' in module > 'ipatests.test_xmlrpc') > > $ grep ldaptracker ipatests/test_xmlrpc/test_user_plugin.py > from ipatests.test_xmlrpc.ldaptracker import Tracker > $ ls ipatests/test_xmlrpc/ldaptracker* > ls: cannot access ipatests/test_xmlrpc/ldaptracker*: No such file or > directory > > > ----- Original Message ----- > > From: "Filip ?kola" > > To: "Milan Kub?k" > > Cc: freeipa-devel at redhat.com > > Sent: Thursday, December 3, 2015 5:38:43 PM > > Subject: Re: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin > > > > Hi, > > > > sending corrected version. > > > > F. > > > > On Thu, 12 Nov 2015 14:03:19 +0100 > > Milan Kub?k wrote: > > > > > On 11/10/2015 12:13 PM, Filip ?kola wrote: > > > > Hi, > > > > > > > > fixed. > > > > > > > > F. > > > > > > > > On Tue, 10 Nov 2015 10:52:45 +0100 > > > > Milan Kub?k wrote: > > > > > > > >> On 11/09/2015 04:35 PM, Filip ?kola wrote: > > > >>> Another patch was applied in the meantime. > > > >>> > > > >>> Attaching an updated version. > > > >>> > > > >>> F. > > > >>> > > > >>> On Mon, 9 Nov 2015 13:35:02 +0100 > > > >>> Milan Kub?k wrote: > > > >>> > > > >>>> On 11/06/2015 11:32 AM, Filip ?kola wrote: > > > >>>> Hi, > > > >>>> the patch doesn't apply. > > > >>>> > > > >> Please fix this. > > > >> > > > >> ipatests/test_xmlrpc/test_user_plugin.py:1419: > > > >> [E0602(undefined-variable), > > > >> TestDeniedBindWithExpiredPrincipal.teardown_class] Undefined > > > >> variable 'user1') > > > >> > > > >> Also, use the version numbers for your changed patches. > > > >> > > > > > > > > > > > Thanks for the patch. Several issues: > > > > > > 1. Use dict.items instead of dict.iteritems, for python3 > > > compatibility > > > > > > 2. What is the purpose of TestPrepare class? The 'purge' methods > > > do not call any ipa commands. > > > Tracker.make_fixture should be used to make the Tracked resources > > > clean themselves up when they're out of scope. > > > > > > 3. Why reference the resources by hardcoded name if they have a > > > fixture representation? > > > > > > 4. Rewrite {create,delete}_test_group to a fixture. You may want > > > to use different scope (or not). > > > > > > 5. In `def atest_rename_to_invalid_login(self, user):` - use > > > pytest.skipif decorator and provide a reason if you must, > > > do not obfuscate method name in order not to run it. > > > > > > > > > > > > -- > > Manage your subscription for the Freeipa-devel mailing list: > > https://www.redhat.com/mailman/listinfo/freeipa-devel > > Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-fskola-0001-4-Refactor-test_user_plugin.patch Type: text/x-patch Size: 98509 bytes Desc: not available URL: From jcholast at redhat.com Thu Dec 10 10:32:26 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 10 Dec 2015 11:32:26 +0100 Subject: [Freeipa-devel] [PATCHES 516-517] spec file: put Python modules into standalone packages In-Reply-To: <56688636.40803@redhat.com> References: <5665A403.9090604@redhat.com> <56688636.40803@redhat.com> Message-ID: <566954BA.6010806@redhat.com> On 9.12.2015 20:51, Petr Vobornik wrote: > On 12/07/2015 04:21 PM, Jan Cholasta wrote: >> Hi, >> >> the attached patches partially fix >> . This is done to allow >> the addition of Python 3 packages, see >> . >> >> See commit messages for more information. >> >> In order to test: >> 1. make rpms >> 2. >> >> >> >> 3. Test with both dnf and yum-deprecated. >> >> Beware that when you run "yum-deprecated clean all", it does not remove >> cache for the on-disk repository created in step 2, you have to remove >> the /var/cache/yum/$basearch/$releasever/$reponame directory manually. >> >> Honza >> > > Shouldn't freeipa-server-dns and freeipa-server-trust-add depend on > freeipa-server? They do not in this patch. IMO they should. Fixed. > > following updates work (all on f23, update from 4.2.3): > dnf update > dnf update freeipa-* > yum-depracated update freeipa-* > > for both client or server with all packages. > > but when I tried to install only client "dnf install freeipa-client" and > then following failed: > dnf update freeipa-client > > The difference was: > Installing: > freeipa-client-common noarch > freeipa-common noarch > python2-ipaclient noarch > python2-ipalib x86_64 > Upgrading: > freeipa-client > > Works: > Installing: > freeipa-client-common noarch > freeipa-common noarch > freeipa-python-compat noarch > replacing freeipa-python.x86_64 4.2.3-1.1.fc23 > python2-ipaclient noarch > python2-ipalib x86_64 > Upgrading: > freeipa-client > > > not sure if it is a problem, otherwise the patch looks OK. Hmm, Fedora packaging guidelines are silent about this. When you run "dnf update freeipa-client freeipa-python" to force freeipa-python update it works fine. I removed Provides added Conflicts on freeipa-python which seem to have fixed it. Updated patch attached. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-516.2-spec-file-remove-config-files-from-freeipa-python.patch Type: text/x-patch Size: 3642 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-517.2-spec-file-put-Python-modules-into-standalone-package.patch Type: text/x-patch Size: 27666 bytes Desc: not available URL: From tbabej at redhat.com Thu Dec 10 10:40:04 2015 From: tbabej at redhat.com (Tomas Babej) Date: Thu, 10 Dec 2015 11:40:04 +0100 Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <20151210080513.GA21550@mail.corp.redhat.com> References: <20151130163256.GA7759@mail.corp.redhat.com> <565DD881.60401@redhat.com> <565EAAAE.8020306@redhat.com> <565F0332.9000804@redhat.com> <20151202162519.GA19962@mail.corp.redhat.com> <56601AE8.2020100@redhat.com> <5660528E.40904@redhat.com> <149447113.23167965.1449156368210.JavaMail.zimbra@redhat.com> <566060E2.8030903@redhat.com> <5666DF65.1030309@redhat.com> <20151210080513.GA21550@mail.corp.redhat.com> Message-ID: <56695684.6050804@redhat.com> On 12/10/2015 09:05 AM, Lukas Slebodnik wrote: > On (08/12/15 14:47), Tomas Babej wrote: >> >> >> On 12/03/2015 04:33 PM, Tomas Babej wrote: >>> >>> >>> On 12/03/2015 04:26 PM, Ale? Mare?ek wrote: >>>> Hello, >>>> >>>> ACK for code >>>> NACK for the placing "get_client_ip_with_hostmask" function to test_sudo.py (this function should be in some more general file) >>>> >>> >>> What place would you propose? The task.py is not a good place, as this >>> is not really a task. >>> >>> Nevertheless, I'd rather have it moved when an use case outside >>> test_sudo.py actually arises. Right now it would lead to unnecessary >>> cluttering. >>> >>> Tomas >>> >> >> I re-discovered ipatests.test_integration.util (two years after I >> created it :D) - which seemed ideal for this function. >> >> Updated patch attached. >> >> Tomas > >>From 33552d6078d75ee99f9ec19ae143df5a61ba84a4 Mon Sep 17 00:00:00 2001 >> From: Tomas Babej >> Date: Wed, 2 Dec 2015 15:25:49 +0100 >> Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on >> hostmask >> >> IPA sudo tests worked under the assumption that the clients >> that are executing the sudo commands have their IPs assigned >> within 255.255.255.0 hostmask. >> >> Removes this (invalid) assumption and adds a >> dynamic detection of the hostmask of the IPA client. >> >> https://fedorahosted.org/freeipa/ticket/5501 >> --- >> ipatests/test_integration/test_sudo.py | 33 +++++++++++++++++++++++++++------ >> ipatests/test_integration/util.py | 16 ++++++++++++++++ >> 2 files changed, 43 insertions(+), 6 deletions(-) >> >> diff --git a/ipatests/test_integration/util.py b/ipatests/test_integration/util.py >> index 1a1bb3fcc923c9f2721f0a4c1cb7a1ba2ccc2dd8..187f39e80e84af0eb4938fb19ac3d3c7c2280ed9 100644 >> --- a/ipatests/test_integration/util.py >> +++ b/ipatests/test_integration/util.py >> @@ -58,3 +58,19 @@ def run_repeatedly(host, command, assert_zero_rc=True, test=None, >> .format(cmd=' '.join(command), >> times=timeout / time_step, >> timeout=timeout)) >> + >> + >> +def get_host_ip_with_hostmask(host): >> + """ >> + Detects the IP of the host including the hostmask. >> + >> + Returns None if the IP could not be detected. >> + """ >> + >> + ip = host.ip >> + result = host.run_command(['ip', 'addr']) >> + full_ip_regex = r'(?P%s/\d{1,2}) ' % re.escape(ip) >> + match = re.search(full_ip_regex, result.stdout_text) > ./make-lint > ************* Module ipatests.test_integration.util > ipatests/test_integration/util.py:72: [E0602(undefined-variable), get_host_ip_with_hostmask] Undefined variable 're') > ipatests/test_integration/util.py:73: [E0602(undefined-variable), get_host_ip_with_hostmask] Undefined variable 're') > =============================================================================== > Errors were found during the static code check. > If you are certain that any of the reported errors are false positives, please > mark them in the source code according to the pylint documentation. > =============================================================================== > Makefile:124: recipe for target 'lint' failed > > LS > Nothing can break when moving a function, right? Missing import fixed. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0388-7-tests-Add-hostmask-detection-for-sudo-rules-validati.patch Type: text/x-patch Size: 4663 bytes Desc: not available URL: From jcholast at redhat.com Thu Dec 10 12:51:47 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 10 Dec 2015 13:51:47 +0100 Subject: [Freeipa-devel] [PATCHES 516-517] spec file: put Python modules into standalone packages In-Reply-To: <566954BA.6010806@redhat.com> References: <5665A403.9090604@redhat.com> <56688636.40803@redhat.com> <566954BA.6010806@redhat.com> Message-ID: <56697563.4040406@redhat.com> On 10.12.2015 11:32, Jan Cholasta wrote: > On 9.12.2015 20:51, Petr Vobornik wrote: >> On 12/07/2015 04:21 PM, Jan Cholasta wrote: >>> Hi, >>> >>> the attached patches partially fix >>> . This is done to allow >>> the addition of Python 3 packages, see >>> . >>> >>> >>> See commit messages for more information. >>> >>> In order to test: >>> 1. make rpms >>> 2. >>> >>> >>> >>> >>> 3. Test with both dnf and yum-deprecated. >>> >>> Beware that when you run "yum-deprecated clean all", it does not remove >>> cache for the on-disk repository created in step 2, you have to remove >>> the /var/cache/yum/$basearch/$releasever/$reponame directory manually. >>> >>> Honza >>> >> >> Shouldn't freeipa-server-dns and freeipa-server-trust-add depend on >> freeipa-server? They do not in this patch. IMO they should. > > Fixed. > >> >> following updates work (all on f23, update from 4.2.3): >> dnf update >> dnf update freeipa-* >> yum-depracated update freeipa-* >> >> for both client or server with all packages. >> >> but when I tried to install only client "dnf install freeipa-client" and >> then following failed: >> dnf update freeipa-client >> >> The difference was: >> Installing: >> freeipa-client-common noarch >> freeipa-common noarch >> python2-ipaclient noarch >> python2-ipalib x86_64 >> Upgrading: >> freeipa-client >> >> Works: >> Installing: >> freeipa-client-common noarch >> freeipa-common noarch >> freeipa-python-compat noarch >> replacing freeipa-python.x86_64 4.2.3-1.1.fc23 >> python2-ipaclient noarch >> python2-ipalib x86_64 >> Upgrading: >> freeipa-client >> >> >> not sure if it is a problem, otherwise the patch looks OK. > > Hmm, Fedora packaging guidelines are silent about this. > > When you run "dnf update freeipa-client freeipa-python" to force > freeipa-python update it works fine. > > I removed Provides added Conflicts on freeipa-python which seem to have > fixed it. Actually I think it would be better to keep the Provides and do Conflicts < 4.2.91 - fixed. Also updated %alt_name dependencies to be in sync with %name dependencies and added arch-specific python-ipalib Provides to python2-ipalib. Updated patches attached. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-516.3-spec-file-remove-config-files-from-freeipa-python.patch Type: text/x-patch Size: 3642 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-517.3-spec-file-put-Python-modules-into-standalone-package.patch Type: text/x-patch Size: 27880 bytes Desc: not available URL: From tbabej at redhat.com Thu Dec 10 13:18:27 2015 From: tbabej at redhat.com (Tomas Babej) Date: Thu, 10 Dec 2015 14:18:27 +0100 Subject: [Freeipa-devel] [PATCH 0395] replicainstall: Make sure the enrollment state is preserved Message-ID: <56697BA3.6030209@redhat.com> Hi, During the promote_check phase, the subsequent checks after the machine is enrolled may cause the installation to abort, hence leaving it enrolled even though it might not have been prior to the execution of the ipa-replica-install command. Make sure that ipa-client-install --uninstall is called on the machine that has not been enrolled before in case of failure during the promote_check phase. https://fedorahosted.org/freeipa/ticket/5529 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0395-replicainstall-Make-sure-the-enrollment-state-is-pre.patch Type: text/x-patch Size: 2739 bytes Desc: not available URL: From tbabej at redhat.com Thu Dec 10 13:22:46 2015 From: tbabej at redhat.com (Tomas Babej) Date: Thu, 10 Dec 2015 14:22:46 +0100 Subject: [Freeipa-devel] [PATCH 0395] replicainstall: Make sure the enrollment state is preserved In-Reply-To: <56697BA3.6030209@redhat.com> References: <56697BA3.6030209@redhat.com> Message-ID: <56697CA6.80500@redhat.com> On 12/10/2015 02:18 PM, Tomas Babej wrote: > Hi, > > During the promote_check phase, the subsequent checks after the machine > is enrolled may cause the installation to abort, hence leaving it > enrolled even though it might not have been prior to the execution of > the ipa-replica-install command. > > Make sure that ipa-client-install --uninstall is called on the machine > that has not been enrolled before in case of failure during the > promote_check phase. > > https://fedorahosted.org/freeipa/ticket/5529 > Self-NACK, installer is redundant for uninstall_client. Updated patch attached. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0395-2-replicainstall-Make-sure-the-enrollment-state-is-pre.patch Type: text/x-patch Size: 2721 bytes Desc: not available URL: From lslebodn at redhat.com Thu Dec 10 13:47:34 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Thu, 10 Dec 2015 14:47:34 +0100 Subject: [Freeipa-devel] [PATCH] ca-less tests updated - POC In-Reply-To: <563CA575.1030808@redhat.com> References: <56337745.109@redhat.com> <563B091B.3010501@redhat.com> <563B6A96.8090606@redhat.com> <563BABEC.8080304@redhat.com> <563C5B1A.1090803@redhat.com> <563C5E6D.4060307@redhat.com> <563CA575.1030808@redhat.com> Message-ID: <20151210134716.GA15692@mail.corp.redhat.com> On (06/11/15 14:04), Oleg Fayans wrote: >Hi Jan, > >On 11/06/2015 09:01 AM, Jan Cholasta wrote: >>Actually it might be better to keep them, but fix them to expect >>ipa-server-certinstall to success. > >Done. Updated patch attached. >Also in the patch 0013 I removed a trailing whitespace which caused lint to >complain > >Now with domain level 0 the test output looks like this: > >[11:40:51]ofayans at vm-076:~]$ ipa-run-tests test_integration/test_caless.py >==================================================================================== >test session starts ===================================================================================== >platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.6.4 >plugins: multihost, sourceorder >collected 88 items > >test_integration/test_caless.py ......xx......ss...............xx........ss....................xx......ss............... > >===================================================================== 76 >passed, 6 skipped, 6 xfailed in 7871.10 seconds >===================================================================== > > >> >>On 6.11.2015 08:47, Jan Cholasta wrote: >>>Hi Oleg, >>> >>>I think you can just remove >>>TestCertinstall.test_{http,ds}_intermediate_ca, the certificates are >>>imported correctly in this case and I didn't see anything break. >>> >>>Honza >>> >>>On 5.11.2015 20:20, Oleg Fayans wrote: >>>>Patch 0014 updated and passes lint >>>> >>>>On 11/05/2015 03:41 PM, Oleg Fayans wrote: >>>>>Wait a bit, the patch has problems with pylint: it does not build :) >>>>>The updated version (without the setupmaster nonsense) is being tested >>>>>now. >>>>> >>>>>On 11/05/2015 08:45 AM, Oleg Fayans wrote: >>>>>>Hi Jan, >>>>>> >>>>>>Could you take a look at these, whenever you are free? >>>>>> >>>>>>On 10/30/2015 02:57 PM, Oleg Fayans wrote: >>>>>>>Hi, >>>>>>> >>>>>>>The following patches contain updates to ca-less integration tests. >>>>>>>It's still a proof of concept: 2 tests still fail seemingly due to >>>>>>>the >>>>>>>change in target system logic (marked as xfail with "ask jcholast >>>>>>>comment") >>>>>>> >>>>>>>The test output looks like this: >>>>>>> >>>>>>>$ ipa-run-tests test_integration/test_caless.py --pdb >>>>>>>==================================================================================== >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>test session starts >>>>>>>===================================================================================== >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.6.4 >>>>>>>plugins: multihost, sourceorder >>>>>>>collected 88 items >>>>>>> >>>>>>>test_integration/test_caless.py >>>>>>>......xx......ss............sssssssssssssssssss.ssssss.........xx......ssxx............. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>==================================================================== >>>>>>>53 >>>>>>> >>>>>>>passed, 29 skipped, 6 xfailed in 5620.17 seconds >>>>>>>===================================================================== >>>>>>> >>>>>>>Numerous skips correspond to the tests related to ipa-replica-prepare >>>>>>>(unsupported under domain level 1) >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>>> >>> >>> >> >> > >-- >Oleg Fayans >Quality Engineer >FreeIPA team >RedHat. >From 3142e9ab937b602a687639e7972422001e887211 Mon Sep 17 00:00:00 2001 >From: Oleg Fayans >Date: Thu, 5 Nov 2015 16:25:29 +0100 >Subject: [PATCH] Updated the script creating test certificate chains > >https://fedorahosted.org/freeipa/ticket/4589 >--- > .../test_integration/scripts/caless-create-pki | 29 ++++++++++++++-------- > 1 file changed, 18 insertions(+), 11 deletions(-) > >diff --git a/ipatests/test_integration/scripts/caless-create-pki b/ipatests/test_integration/scripts/caless-create-pki >index f428ebae16e05644a875a35faf192f75eb149740..4c37077ffdecfb0c70663c7c4817f102154d3b26 100644 >--- a/ipatests/test_integration/scripts/caless-create-pki >+++ b/ipatests/test_integration/scripts/caless-create-pki >@@ -3,7 +3,17 @@ > profile_ca=(-t CT,C,C -v 120) > profile_server=(-t ,, -v 12) > >-crl_path=${crl_path-$(readlink -f $dbdir)} >+# crl_path=${crl_path-$(readlink -f $dbdir)} >+profile_ca_request_options=(-1 -2 -4) >+profile_ca_request_input="\$'0\n1\n5\n6\n9\ny\ny\n\ny\n1\n7\nfile://'\$(readlink -f \$dbdir)/\$ca.crl\$'\n-1\n-1\n-1\nn\nn\n'" >+profile_ca_create_options=(-v 120) >+profile_ca_add_options=(-t ,,) >+ >+profile_server_request_options=(-4) >+profile_server_request_input="\$'1\n7\nfile://'\$(readlink -f \$dbdir)/\$ca.crl\$'\n-1\n-1\n-1\nn\nn\n'" >+profile_server_create_options=(-v 12) >+profile_server_add_options=(-t ,,) >+ > > serial_number=0 > >@@ -18,7 +28,11 @@ gen_cert() { > ca="$nick" > fi > >+ echo $profile > eval "options=(\"\${profile_$profile[@]}\")" >+ eval "request_options=(\"\${profile_${profile}_request_options[@]}\")" >+ eval "eval request_input=(\"\${profile_${profile}_request_input[@]}\")" >+ > if [ "$ca" = "$nick" ]; then > options=("${options[@]}" -x -m 1) > else >@@ -38,16 +52,7 @@ gen_cert() { > > csr="$(mktemp)" > crt="$(mktemp)" >- certutil -R -d "$dbdir" -s "$subject" -f "$pwfile" -z "$noise" -o "$csr" -4 >/dev/null <-1 >-7 >-file://$crl_path/$ca.crl >--1 >--1 >--1 >-n >-n >-EOF >+ certutil -R -d "$dbdir" -s "$subject" -f "$pwfile" -z "$noise" -o "$csr" "${request_options[@]}" >/dev/null <<<"$request_input" > serial_number=$(($serial_number+1)) > certutil -C -d "$dbdir" -f "$pwfile" -m "$serial_number" -i "$csr" -o "$crt" "${options[@]}" "$@" > certutil -A -d "$dbdir" -n "$nick" -f "$pwfile" -i "$crt" "${options[@]}" >@@ -117,3 +122,5 @@ gen_cert server replica-selfsign "CN=$server2,O=Self-signed" > gen_subtree ca1 'Example Organization' > gen_subtree ca1/subca 'Subsidiary Example Organization' > gen_subtree ca2 'Other Example Organization' >+gen_subtree ca3 'Unknown Organization' >+certutil -D -d "$dbdir" -n ca3 >-- >2.4.3 > >From e101c70bca6ed9d54b7aa6f19f0a76425cf411ea Mon Sep 17 00:00:00 2001 >From: Oleg Fayans >Date: Fri, 6 Nov 2015 13:43:05 +0100 >Subject: [PATCH] Updated ca-less tests. > >A preview. All tests except 2 pass. Those 2 failing ones need a consulting from >jcholast (so far marked as xfail). > >https://fedorahosted.org/freeipa/ticket/4589 >--- I had to use 3 way merge to apply this patch. but I can still see failures with this patch @see _______________________ TestReplicaInstall.test_no_certs _______________________ self = def test_no_certs(self): "IPA replica install without certificates" replica = self.replicas[0] if config.domain_level == 0: result = self.master.run_command(['ipa-replica-prepare', self.replicas[0].hostname, '-p', self.env['dirman_password']], raiseonerr=False) assert result.returncode > 0 assert ('Cannot issue certificates: a CA is not installed. Use the ' '--http-cert-file, --dirsrv-cert-file options to provide ' 'custom certificates.' in result.stderr_text), \ result.stderr_text else: args = ["ipa-replica-install", "-U", "-p", replica.config.dirman_password, "-w", replica.config.admin_password, "--ip-address", replica.ip, "--setup-ca"] tasks.install_client(self.master, replica) result = replica.run_command(args, raiseonerr=False) > assert ("The remote master does not have a CA installed," " can't proceed without certs" in result.stderr_text), \ result.stderr_text E AssertionError: ipa : ERROR Reverse DNS resolution of address 10.16.65.13 (dell-pe1850-04.testrelm.test) failed. Clients may not function properly. Please check your DNS setup. (Note that this check queries IPA DNS directly and ignores /etc/hosts.) E ipa : ERROR The IP address 2620:52:0:1040:214:22ff:fe21:ca12 of host dell-pe1850-04.testrelm.test resolves to: dell-pe1850-04.rhts.eng.bos.redhat.com.. Clients may not function properly. Please check your DNS setup. (Note that this check queries IPA DNS directly and ignores /etc/hosts.) E ipa : ERROR Cannot issue certificates: a CA is not installed. Use the --http-cert-file, --dirsrv-cert-file options to provide custom certificates. E ipa.ipapython.install.cli.install_tool(Replica): ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information E E assert "The remote master does not have a CA installed, can't proceed without certs" in 'ipa : ERROR Reverse DNS resolution of address 10.16.65.13 (dell-pe1850-04.testrelm.test) failed. Clients m...(Replica): ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information\n' E + where 'ipa : ERROR Reverse DNS resolution of address 10.16.65.13 (dell-pe1850-04.testrelm.test) failed. Clients m...(Replica): ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information\n' = .stderr_text test_integration/test_caless.py:785: AssertionError ---------------------------- Captured stdout setup ----------------------------- LS From mbabinsk at redhat.com Thu Dec 10 14:01:01 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Thu, 10 Dec 2015 15:01:01 +0100 Subject: [Freeipa-devel] [PATCHES 509-514] replica promotion: use host credentials when setting up replication In-Reply-To: <56692254.3060209@redhat.com> References: <564DE017.4020408@redhat.com> <1447951434.5132.27.camel@redhat.com> <564F4105.1040204@redhat.com> <1448038723.5132.59.camel@redhat.com> <5652C63E.5090206@redhat.com> <1448289289.7892.13.camel@redhat.com> <565324C4.1090007@redhat.com> <1448290024.7892.15.camel@redhat.com> <565D823E.5090600@redhat.com> <56604D94.7000705@redhat.com> <566167DE.9090204@redhat.com> <56619F5C.5000900@redhat.com> <5661A04E.9080409@redhat.com> <5661A593.5030003@redhat.com> <1449244714.3445.67.camel@redhat.com> <56649B78.1080805@redhat.com> <566531EF.7000603@redhat.com> <56684B33.3090103@redhat.com> <56692254.3060209@redhat.com> Message-ID: <5669859D.2050202@redhat.com> On 12/10/2015 07:57 AM, Jan Cholasta wrote: > On 9.12.2015 16:39, Jan Cholasta wrote: >> On 7.12.2015 08:14, Jan Cholasta wrote: >>> On 6.12.2015 21:32, Martin Basti wrote: >>>> >>>> >>>> On 04.12.2015 16:58, Simo Sorce wrote: >>>>> On Fri, 2015-12-04 at 15:39 +0100, Jan Cholasta wrote: >>>>>> On 4.12.2015 15:16, Jan Cholasta wrote: >>>>>>> On 4.12.2015 15:12, Jan Cholasta wrote: >>>>>>>> On 4.12.2015 11:15, Petr Vobornik wrote: >>>>>>>>> On 12/03/2015 03:11 PM, Martin Basti wrote: >>>>>>>>>> >>>>>>>>>> On 01.12.2015 12:19, Jan Cholasta wrote: >>>>>>>>>>> On 23.11.2015 15:47, Simo Sorce wrote: >>>>>>>>>>>> On Mon, 2015-11-23 at 15:37 +0100, Jan Cholasta wrote: >>>>>>>>>>>>> Ad alternative is to add the host to ipaservers before the >>>>>>>>>>>>> checks >>>>>>>>>>>>> are >>>>>>>>>>>>> done and remove it again if any of them fail. >>>>>>>>>>>> Too error prone, I am ok with the current way in your patches >>>>>>>>>>>> until/unless I can think of a fail safe way. :-) >>>>>>>>>>> Updated patches attached. Note that 520 should be applied >>>>>>>>>>> between 509 >>>>>>>>>>> and 510. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> Functional ACK >>>>>>>>>> >>>>>>>>> Simo, do you want to review the ACIs or other things it the >>>>>>>>> patches? Or >>>>>>>>> can the patches be pushed? >>>>>>>> There were no changes in the ACIs since last time. >>>>>>> Actually, memberPrincipal was removed from the "IPA server hosts can >>>>>>> manage own Custodia secrets" ACI, as per Simo's request. >>>>>>> >>>>>>>> Rebased patches attached. >>>>>> Note that 520 should still be applied between 509 and 510. >>>>>> >>>>> LGTM >>>>> >>>> ACK >>> >>> Thanks. >>> >>> Pushed to master: 01ddf51df76f3298499973355c5461727e46ab5b >> >> Martin Babinsky found out that ipaservers is not created early enough >> when installing a replica of a 4.2 or older server which causes a crash. >> >> The attached patch fixes that. > > Actually I don't like how I fixed that, here's an updated patch. > > Also, I noticed that replica promotion fails too late in domain level 0. > Fixed as well. > > > ACK to both patches. -- Martin^3 Babinsky From mbasti at redhat.com Thu Dec 10 14:18:26 2015 From: mbasti at redhat.com (Martin Basti) Date: Thu, 10 Dec 2015 15:18:26 +0100 Subject: [Freeipa-devel] [PATCH 0374-0375] Fix permissions on newly created directories Message-ID: <566989B2.2080502@redhat.com> Hello, patch 0374 fixes the ticket, but I found more issues with directory permission, I fixed them in 0375 https://fedorahosted.org/freeipa/ticket/5520 Patches attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0374-DNS-fix-file-permissions.patch Type: text/x-patch Size: 1514 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0375-Explicitly-call-chmod-on-newly-created-directories.patch Type: text/x-patch Size: 4931 bytes Desc: not available URL: From tbabej at redhat.com Thu Dec 10 14:49:07 2015 From: tbabej at redhat.com (Tomas Babej) Date: Thu, 10 Dec 2015 15:49:07 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <56695291.5090705@redhat.com> References: <56687174.8000305@redhat.com> <20151210081357.GB21550@mail.corp.redhat.com> <56695291.5090705@redhat.com> Message-ID: <566990E3.2080903@redhat.com> On 12/10/2015 11:23 AM, Martin Basti wrote: > > > On 10.12.2015 09:13, Lukas Slebodnik wrote: >> On (09/12/15 19:22), Martin Basti wrote: >>> https://fedorahosted.org/freeipa/ticket/5535 >>> >>> Patch attached. >> >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 00:00:00 2001 >>> From: Martin Basti >>> Date: Wed, 9 Dec 2015 18:53:35 +0100 >>> Subject: [PATCH] Fix version comparison >>> >>> Use RPM library to compare vendor versions of IPA for redhat platform >>> >>> https://fedorahosted.org/freeipa/ticket/5535 >>> --- >>> freeipa.spec.in | 2 ++ >>> ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ >>> 2 files changed, 21 insertions(+) >>> >>> diff --git a/freeipa.spec.in b/freeipa.spec.in >>> index >>> 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 >>> 100644 >>> --- a/freeipa.spec.in >>> +++ b/freeipa.spec.in >>> @@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} >>> Requires: gzip >>> Requires: python-gssapi >= 1.1.0 >>> Requires: custodia >>> +Requires: rpm-python >>> +Requires: rpmdevtools >> Could you explain why do you need the 2nd package? >> It does not contains any python modules >> and I cannot see usage of any binary in this patch >> >> LS > Thanks for this catch, it is actually located in yum package, I rather > copy stringToVersion function from there to IPA, to avoid dependency hell > > Updated patch attached. > > Looking good. The __cmp__ function, however, is not available in Python 3. As we will eventually support python3 in RHEL as well, maybe we should make sure even platform-dependent parts are python3 compatible? For the future's sake. Tomas From amarecek at redhat.com Thu Dec 10 14:52:15 2015 From: amarecek at redhat.com (=?utf-8?Q?Ale=C5=A1_Mare=C4=8Dek?=) Date: Thu, 10 Dec 2015 09:52:15 -0500 (EST) Subject: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin In-Reply-To: <20151210112952.3ddb7136@vor2.netbox.priv> References: <20151106113201.5e6a7042@dhcp-24-123.brq.redhat.com> <20151109163512.3d3beb5f@redhat.com> <5641BE6D.4060608@redhat.com> <20151110121350.06295a68@dhcp-24-103.brq.redhat.com> <56448E17.2010409@redhat.com> <20151203173843.37179230@vor2.netbox.priv> <869710129.24860544.1449498571599.JavaMail.zimbra@redhat.com> <20151210112952.3ddb7136@vor2.netbox.priv> Message-ID: <891788766.26644767.1449759135974.JavaMail.zimbra@redhat.com> Ahoj Filipe, zase tam vidim to '\r\n'... Prisel jsi na to, co to zpusobuje? Jinak patche jdu otestovat... - alich - ----- Original Message ----- > From: "Filip ?kola" > To: "Ale? Mare?ek" > Cc: freeipa-devel at redhat.com, "Milan Kub?k" > Sent: Thursday, December 10, 2015 11:29:52 AM > Subject: Re: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin > > Hi, > > this if fixed. Also issues with test_stageuser_plugin caused by > UserTracker changes should be fixed here. > > Filip > > > On Mon, 7 Dec 2015 09:29:31 -0500 (EST) > Ale? Mare?ek wrote: > > > NACK. > > > > $ ./make-lint > > ************* Module ipatests.test_xmlrpc.test_user_plugin > > ipatests/test_xmlrpc/test_user_plugin.py:42: > > [E0611(no-name-in-module), ] No name 'ldaptracker' in module > > 'ipatests.test_xmlrpc') > > > > $ grep ldaptracker ipatests/test_xmlrpc/test_user_plugin.py > > from ipatests.test_xmlrpc.ldaptracker import Tracker > > $ ls ipatests/test_xmlrpc/ldaptracker* > > ls: cannot access ipatests/test_xmlrpc/ldaptracker*: No such file or > > directory > > > > > > ----- Original Message ----- > > > From: "Filip ?kola" > > > To: "Milan Kub?k" > > > Cc: freeipa-devel at redhat.com > > > Sent: Thursday, December 3, 2015 5:38:43 PM > > > Subject: Re: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin > > > > > > Hi, > > > > > > sending corrected version. > > > > > > F. > > > > > > On Thu, 12 Nov 2015 14:03:19 +0100 > > > Milan Kub?k wrote: > > > > > > > On 11/10/2015 12:13 PM, Filip ?kola wrote: > > > > > Hi, > > > > > > > > > > fixed. > > > > > > > > > > F. > > > > > > > > > > On Tue, 10 Nov 2015 10:52:45 +0100 > > > > > Milan Kub?k wrote: > > > > > > > > > >> On 11/09/2015 04:35 PM, Filip ?kola wrote: > > > > >>> Another patch was applied in the meantime. > > > > >>> > > > > >>> Attaching an updated version. > > > > >>> > > > > >>> F. > > > > >>> > > > > >>> On Mon, 9 Nov 2015 13:35:02 +0100 > > > > >>> Milan Kub?k wrote: > > > > >>> > > > > >>>> On 11/06/2015 11:32 AM, Filip ?kola wrote: > > > > >>>> Hi, > > > > >>>> the patch doesn't apply. > > > > >>>> > > > > >> Please fix this. > > > > >> > > > > >> ipatests/test_xmlrpc/test_user_plugin.py:1419: > > > > >> [E0602(undefined-variable), > > > > >> TestDeniedBindWithExpiredPrincipal.teardown_class] Undefined > > > > >> variable 'user1') > > > > >> > > > > >> Also, use the version numbers for your changed patches. > > > > >> > > > > > > > > > > > > > > Thanks for the patch. Several issues: > > > > > > > > 1. Use dict.items instead of dict.iteritems, for python3 > > > > compatibility > > > > > > > > 2. What is the purpose of TestPrepare class? The 'purge' methods > > > > do not call any ipa commands. > > > > Tracker.make_fixture should be used to make the Tracked resources > > > > clean themselves up when they're out of scope. > > > > > > > > 3. Why reference the resources by hardcoded name if they have a > > > > fixture representation? > > > > > > > > 4. Rewrite {create,delete}_test_group to a fixture. You may want > > > > to use different scope (or not). > > > > > > > > 5. In `def atest_rename_to_invalid_login(self, user):` - use > > > > pytest.skipif decorator and provide a reason if you must, > > > > do not obfuscate method name in order not to run it. > > > > > > > > > > > > > > > > > -- > > > Manage your subscription for the Freeipa-devel mailing list: > > > https://www.redhat.com/mailman/listinfo/freeipa-devel > > > Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code > > From mbabinsk at redhat.com Thu Dec 10 14:56:49 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Thu, 10 Dec 2015 15:56:49 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <56693C49.6030508@redhat.com> References: <5667EE2C.2050100@redhat.com> <56683238.3050700@redhat.com> <56684B03.2020905@redhat.com> <56693C49.6030508@redhat.com> Message-ID: <566992B1.6030702@redhat.com> On 12/10/2015 09:48 AM, Jan Cholasta wrote: > On 9.12.2015 16:38, Jan Cholasta wrote: >> On 9.12.2015 14:52, Jan Cholasta wrote: >>> On 9.12.2015 10:02, Jan Cholasta wrote: >>>> Hi, >>>> >>>> the attached patches fix >>>> . >>> >>> Note that this needs selinux-policy fix to work, so put SELinux into >>> permissive mode for testing: >>> . >> >> Updated patches attached. > > I screwed up a change in patch 524 and accidentally included a chunk of > code in patch 525 that doesn't belong in it. > > Updated patches attached. > > > Patches work as expected and I was not able to find any functional problem. I have a question about the naming of the oddjob helper script: the one related to trusts is named 'com.redhat.idm.trust-fetch-domains', and the conncheck runner is named 'org.freeipa.server.conncheck'. I don't want to start another bikeshedding conversation but shouldn't we named them in a consistent fashion (either rename the first one in separate patch or rename the new helper to com.redhat.idm.server.conncheck)? I understand that as an upstream, we should go with the 'org.freeipa.*' convention, but having two helpers with different prefixes makes me sad. That is a nitpick though, it does not affect the overall functionality of the patches so ACK. -- Martin^3 Babinsky From amarecek at redhat.com Thu Dec 10 15:11:47 2015 From: amarecek at redhat.com (=?utf-8?Q?Ale=C5=A1_Mare=C4=8Dek?=) Date: Thu, 10 Dec 2015 10:11:47 -0500 (EST) Subject: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin In-Reply-To: <20151210112952.3ddb7136@vor2.netbox.priv> References: <20151106113201.5e6a7042@dhcp-24-123.brq.redhat.com> <20151109163512.3d3beb5f@redhat.com> <5641BE6D.4060608@redhat.com> <20151110121350.06295a68@dhcp-24-103.brq.redhat.com> <56448E17.2010409@redhat.com> <20151203173843.37179230@vor2.netbox.priv> <869710129.24860544.1449498571599.JavaMail.zimbra@redhat.com> <20151210112952.3ddb7136@vor2.netbox.priv> Message-ID: <964966782.26674717.1449760307850.JavaMail.zimbra@redhat.com> Ah, sorry, haven't realized there had been devel list attached. Ok, there is some problem with \r\n in the patch. Filip, please take a look at it... Thanks... - alich - ----- Original Message ----- > From: "Filip ?kola" > To: "Ale? Mare?ek" > Cc: freeipa-devel at redhat.com, "Milan Kub?k" > Sent: Thursday, December 10, 2015 11:29:52 AM > Subject: Re: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin > > Hi, > > this if fixed. Also issues with test_stageuser_plugin caused by > UserTracker changes should be fixed here. > > Filip > > > On Mon, 7 Dec 2015 09:29:31 -0500 (EST) > Ale? Mare?ek wrote: > > > NACK. > > > > $ ./make-lint > > ************* Module ipatests.test_xmlrpc.test_user_plugin > > ipatests/test_xmlrpc/test_user_plugin.py:42: > > [E0611(no-name-in-module), ] No name 'ldaptracker' in module > > 'ipatests.test_xmlrpc') > > > > $ grep ldaptracker ipatests/test_xmlrpc/test_user_plugin.py > > from ipatests.test_xmlrpc.ldaptracker import Tracker > > $ ls ipatests/test_xmlrpc/ldaptracker* > > ls: cannot access ipatests/test_xmlrpc/ldaptracker*: No such file or > > directory > > > > > > ----- Original Message ----- > > > From: "Filip ?kola" > > > To: "Milan Kub?k" > > > Cc: freeipa-devel at redhat.com > > > Sent: Thursday, December 3, 2015 5:38:43 PM > > > Subject: Re: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin > > > > > > Hi, > > > > > > sending corrected version. > > > > > > F. > > > > > > On Thu, 12 Nov 2015 14:03:19 +0100 > > > Milan Kub?k wrote: > > > > > > > On 11/10/2015 12:13 PM, Filip ?kola wrote: > > > > > Hi, > > > > > > > > > > fixed. > > > > > > > > > > F. > > > > > > > > > > On Tue, 10 Nov 2015 10:52:45 +0100 > > > > > Milan Kub?k wrote: > > > > > > > > > >> On 11/09/2015 04:35 PM, Filip ?kola wrote: > > > > >>> Another patch was applied in the meantime. > > > > >>> > > > > >>> Attaching an updated version. > > > > >>> > > > > >>> F. > > > > >>> > > > > >>> On Mon, 9 Nov 2015 13:35:02 +0100 > > > > >>> Milan Kub?k wrote: > > > > >>> > > > > >>>> On 11/06/2015 11:32 AM, Filip ?kola wrote: > > > > >>>> Hi, > > > > >>>> the patch doesn't apply. > > > > >>>> > > > > >> Please fix this. > > > > >> > > > > >> ipatests/test_xmlrpc/test_user_plugin.py:1419: > > > > >> [E0602(undefined-variable), > > > > >> TestDeniedBindWithExpiredPrincipal.teardown_class] Undefined > > > > >> variable 'user1') > > > > >> > > > > >> Also, use the version numbers for your changed patches. > > > > >> > > > > > > > > > > > > > > Thanks for the patch. Several issues: > > > > > > > > 1. Use dict.items instead of dict.iteritems, for python3 > > > > compatibility > > > > > > > > 2. What is the purpose of TestPrepare class? The 'purge' methods > > > > do not call any ipa commands. > > > > Tracker.make_fixture should be used to make the Tracked resources > > > > clean themselves up when they're out of scope. > > > > > > > > 3. Why reference the resources by hardcoded name if they have a > > > > fixture representation? > > > > > > > > 4. Rewrite {create,delete}_test_group to a fixture. You may want > > > > to use different scope (or not). > > > > > > > > 5. In `def atest_rename_to_invalid_login(self, user):` - use > > > > pytest.skipif decorator and provide a reason if you must, > > > > do not obfuscate method name in order not to run it. > > > > > > > > > > > > > > > > > -- > > > Manage your subscription for the Freeipa-devel mailing list: > > > https://www.redhat.com/mailman/listinfo/freeipa-devel > > > Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code > > From mbabinsk at redhat.com Thu Dec 10 15:35:45 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Thu, 10 Dec 2015 16:35:45 +0100 Subject: [Freeipa-devel] [PATCH 0116] CI tests: remove '-p' option from ipa-dns-install calls Message-ID: <56699BD1.7080804@redhat.com> See commit message. -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0116-CI-tests-remove-p-option-from-ipa-dns-install-calls.patch Type: text/x-patch Size: 4344 bytes Desc: not available URL: From mbasti at redhat.com Thu Dec 10 16:09:55 2015 From: mbasti at redhat.com (Martin Basti) Date: Thu, 10 Dec 2015 17:09:55 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <566990E3.2080903@redhat.com> References: <56687174.8000305@redhat.com> <20151210081357.GB21550@mail.corp.redhat.com> <56695291.5090705@redhat.com> <566990E3.2080903@redhat.com> Message-ID: <5669A3D3.7050608@redhat.com> On 10.12.2015 15:49, Tomas Babej wrote: > > On 12/10/2015 11:23 AM, Martin Basti wrote: >> >> On 10.12.2015 09:13, Lukas Slebodnik wrote: >>> On (09/12/15 19:22), Martin Basti wrote: >>>> https://fedorahosted.org/freeipa/ticket/5535 >>>> >>>> Patch attached. >>> >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 00:00:00 2001 >>>> From: Martin Basti >>>> Date: Wed, 9 Dec 2015 18:53:35 +0100 >>>> Subject: [PATCH] Fix version comparison >>>> >>>> Use RPM library to compare vendor versions of IPA for redhat platform >>>> >>>> https://fedorahosted.org/freeipa/ticket/5535 >>>> --- >>>> freeipa.spec.in | 2 ++ >>>> ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ >>>> 2 files changed, 21 insertions(+) >>>> >>>> diff --git a/freeipa.spec.in b/freeipa.spec.in >>>> index >>>> 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 >>>> 100644 >>>> --- a/freeipa.spec.in >>>> +++ b/freeipa.spec.in >>>> @@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} >>>> Requires: gzip >>>> Requires: python-gssapi >= 1.1.0 >>>> Requires: custodia >>>> +Requires: rpm-python >>>> +Requires: rpmdevtools >>> Could you explain why do you need the 2nd package? >>> It does not contains any python modules >>> and I cannot see usage of any binary in this patch >>> >>> LS >> Thanks for this catch, it is actually located in yum package, I rather >> copy stringToVersion function from there to IPA, to avoid dependency hell >> >> Updated patch attached. >> >> > Looking good. The __cmp__ function, however, is not available in Python > 3. As we will eventually support python3 in RHEL as well, maybe we > should make sure even platform-dependent parts are python3 compatible? > For the future's sake. > > Tomas > Thanks, python 3 compatible patch attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0373.3-Fix-version-comparison.patch Type: text/x-patch Size: 3592 bytes Desc: not available URL: From dkupka at redhat.com Thu Dec 10 16:31:20 2015 From: dkupka at redhat.com (David Kupka) Date: Thu, 10 Dec 2015 17:31:20 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <56686AFE.4000401@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> <5665843C.2090507@redhat.com> <56658C89.8050103@redhat.com> <56668D30.8060308@redhat.com> <5666BCBC.9040005@redhat.com> <5666E822.90103@redhat.com> <56682093.3010505@redhat.com> <56686AFE.4000401@redhat.com> Message-ID: <5669A8D8.5030405@redhat.com> On 09/12/15 18:55, Petr Spacek wrote: > On 9.12.2015 13:37, David Kupka wrote: >> On 08/12/15 15:24, Petr Spacek wrote: >>> On 8.12.2015 12:19, David Kupka wrote: >>>> On 08/12/15 08:56, Petr Spacek wrote: >>>>> On 7.12.2015 14:41, David Kupka wrote: >>>>>> +def is_host_resolvable(fqdn): >>>>>> + if not isinstance(fqdn, DNSName): >>>>>> + fqdn = DNSName(fqdn) >>>>>> + for rdtype in (rdatatype.A, rdatatype.AAAA): >>>>>> + try: >>>>>> + resolver.query(fqdn.make_absolute(), rdtype) >>>>>> + except DNSException: >>>>>> + continue >>>>>> + else: >>>>>> + return True >>>>>> + >>>>>> + return False >>>>>> >>>>> >>>>> NACK, you are re-introducing duplicate function which was removed in >>>>> 498471e4aed1367b72cd74d15811d0584a6ee268. >>>>> >>>>> Please amend the patch ASAP to use new verify_host_resolvable() function so I >>>>> can test it and get it into 4.3. >>>>> >>>> Updated patches attached. >>> >>> Hmm, my review script screams: >>> >>> Detected base branch: origin/master >>> Checks will be made against following base commit: 848912a add missing >>> /ipaplatform/constants.py to .gitignore >>> Writing API to API.txt >>> ./ipalib/plugins/dns.py:2127:9: E124 closing bracket does not match visual >>> indentation >>> ./ipalib/plugins/dns.py:2711:13: E128 continuation line under-indented for >>> visual indent >>> ./ipalib/plugins/dns.py:2713:9: E124 closing bracket does not match visual >>> indentation >>> ./ipalib/plugins/dns.py:2716:13: E128 continuation line under-indented for >>> visual indent >>> ./ipapython/ipautil.py:928:1: E302 expected 2 blank lines, found 1 >>> ./ipapython/ipautil.py:948:17: E128 continuation line under-indented for >>> visual indent >>> ./ipapython/ipautil.py:956:1: E302 expected 2 blank lines, found 1 >>> ./ipapython/ipautil.py:997:80: E501 line too long (83 > 79 characters) >>> ./ipapython/ipautil.py:998:80: E501 line too long (83 > 79 characters) >>> ./ipaserver/install/bindinstance.py:49:9: E128 continuation line >>> under-indented for visual indent >>> ./ipaserver/install/bindinstance.py:292:80: E501 line too long (80 > 79 >>> characters) >>> ./ipaserver/install/bindinstance.py:438:19: E128 continuation line >>> under-indented for visual indent >>> ./ipaserver/install/server/common.py:271:63: E261 at least two spaces before >>> inline comment >>> ./ipaserver/install/server/common.py:271:80: E501 line too long (90 > 79 >>> characters) >>> ERROR: PEP8 --diff failed, continuing ... >>> ERROR: API.txt was changed without a change in VERSION, continuing ... >>> Summary of detected errors: >>> ERROR: PEP8 --diff failed >>> ERROR: API.txt was changed without a change in VERSION >>> >>> Please fix it :-) >>> >>> Make make this more pleasant for you, you can use (and review) my attached >>> patch. It adds 'review' target to Makefile, it will do the same checks as I do. >>> >> >> Unfortunately your tool does not work for me (output w/ -o xtrace attached). >> Anyway I have incremented API version and fixed most of the pep8 errors except >> for E124 twice in ipalib/plugins/dns.py as it seems to be convention in the >> file and E501 twice in ipapython/ipautil.py because IMO breaking string >> constant into multiple lines does not help readability. >> >> Updated patches also attached. > > We are almost there, but NACK for now. > > 1) Following attempt to install IPA explodes. Please note that > dom-245.idm.lab.eng.brq.redhat.com DNS domain is delegated to the IPA server > before installation is started, so it gives 'timeout' or possibly SERVFAIL. > > # ipa-server-install --ds-password=root4lab --admin-password=root4lab > --setup-dns --forwarder=10.38.5.26 --no-reverse --allow-zone-overlap > --domain=dom-245.idm.lab.eng.brq.redhat.com > --realm=DOM-245.IDM.LAB.ENG.BRQ.REDHAT.COM > [...] > > Configuring DNS (named) > [1/11]: generating rndc key file > [2/11]: adding DNS container > [3/11]: setting up our zone > [error] InvocationError: DNS check for domain > dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation timed out after > 30.000453949 seconds. Please make sure that the domain is properly delegated > to this IPA server. > ipa.ipapython.install.cli.install_tool(Server): ERROR DNS check for domain > dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation timed out after > 30.000453949 seconds. Please make sure that the domain is properly delegated > to this IPA server. > ipa.ipapython.install.cli.install_tool(Server): ERROR The > ipa-server-install command failed. See /var/log/ipaserver-install.log for more > information > > 2015-12-09T17:15:37Z DEBUG File > "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in execute > return_value = self.run() > File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line 318, > in run > cfgr.run() > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 310, > in run > self.execute() > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 332, > in execute > for nothing in self._executor(): > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, > in __runner > self._handle_exception(exc_info) > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, > in _handle_exception > six.reraise(*exc_info) > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, > in __runner > step() > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, > in > step = lambda: next(self.__gen) > File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, > in run_generator_with_yield_from > six.reraise(*exc_info) > File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, > in run_generator_with_yield_from > value = gen.send(prev_value) > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 571, > in _configure > next(executor) > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, > in __runner > self._handle_exception(exc_info) > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 449, > in _handle_exception > self.__parent._handle_exception(exc_info) > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, > in _handle_exception > six.reraise(*exc_info) > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 446, > in _handle_exception > super(ComponentBase, self)._handle_exception(exc_info) > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, > in _handle_exception > six.reraise(*exc_info) > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, > in __runner > step() > File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, > in > step = lambda: next(self.__gen) > File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, > in run_generator_with_yield_from > six.reraise(*exc_info) > File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, > in run_generator_with_yield_from > value = gen.send(prev_value) > > File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line > 63, in _install > for nothing in self._installer(self.parent): > File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", > line 1471, in main > install(self) > File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", > line 265, in decorated > func(installer) > File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", > line 958, in install > dns.install(False, False, options) > File "/usr/lib/python2.7/site-packages/ipaserver/install/dns.py", line 322, > in install > bind.create_instance() > File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", > line 680, in create_instance > self.start_creation() > File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line > 447, in start_creation > run_step(full_msg, method) > File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line > 437, in run_step > method() > File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", > line 805, in __setup_zone > ns_hostname=self.api.env.host, force=True, api=self.api) > File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", > line 331, in add_zone > force=force) > File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 447, in > __call__ > ret = self.run(*args, **options) > File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 764, in run > return self.execute(*args, **options) > File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line 2781, in > execute > result = super(dnszone_add, self).execute(*keys, **options) > File "/usr/lib/python2.7/site-packages/ipalib/plugins/baseldap.py", line > 1233, in execute > *keys, **options) > File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line 2747, in > pre_callback > ldap, dn, entry_attrs, attrs_list, *keys, **options) > File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line 2153, in > pre_callback > raise errors.InvocationError(e.message) > > > 2) Please print 'Checking DNS domain , please wait ...' when validating > domain parameter in installer. The timeout is 30 seconds and users may be > nervous when the installed blocks for 30 seconds without a visible reason. > > Precedent for this is in check_forwarders() in > ipaserver/install/bindinstance.py . Encapsulating check_zone_overlap() in > auxiliary method may be an option. > > > 3) Timeout is a fatal error instead of warning. We have been discussing this > and it should really be just a warning. > > > 4) Nitpicks are attached in .patch file. > > > 5) ipa dnszone-add checks work as expected, good job! > > > Thank you for patience! > Updated patches attached. Added patch introducing --auto-reverse option that should generate needed reverse zones automatically. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0072.0-dns-Add-auto-reverse-option.patch Type: text/x-patch Size: 5177 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0058.10-dns-do-not-add-forward-zone-if-it-is-already-resolva.patch Type: text/x-patch Size: 12748 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0064.7-dns-Check-if-domain-already-exists.patch Type: text/x-patch Size: 17048 bytes Desc: not available URL: From pspacek at redhat.com Thu Dec 10 17:04:27 2015 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 10 Dec 2015 18:04:27 +0100 Subject: [Freeipa-devel] [PATCH 0069] Add 'review' target for make In-Reply-To: <56683B03.40703@redhat.com> References: <56683B03.40703@redhat.com> Message-ID: <5669B09B.9000001@redhat.com> On 9.12.2015 15:30, Petr Spacek wrote: > Hello, > > this patch automates some of sanity checks proposed by Petr Vobornik. > > 'make review' should be used in root of clean Git tree which has patches under > review applied. > > Magic in review.sh attempts to detect nearest remote branch which can be used > as diff base for review. Please see review.sh for further details. And here is the patch! :-) -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0069-Add-review-target-for-make.-It-automates-following-t.patch Type: text/x-patch Size: 3163 bytes Desc: not available URL: From pspacek at redhat.com Thu Dec 10 17:10:38 2015 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 10 Dec 2015 18:10:38 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <5669A8D8.5030405@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> <5665843C.2090507@redhat.com> <56658C89.8050103@redhat.com> <56668D30.8060308@redhat.com> <5666BCBC.9040005@redhat.com> <5666E822.90103@redhat.com> <56682093.3010505@redhat.com> <56686AFE.4000401@redhat.com> <5669A8D8.5030405@redhat.com> Message-ID: <5669B20E.8000803@redhat.com> On 10.12.2015 17:31, David Kupka wrote: > On 09/12/15 18:55, Petr Spacek wrote: >> On 9.12.2015 13:37, David Kupka wrote: >>> On 08/12/15 15:24, Petr Spacek wrote: >>>> On 8.12.2015 12:19, David Kupka wrote: >>>>> On 08/12/15 08:56, Petr Spacek wrote: >>>>>> On 7.12.2015 14:41, David Kupka wrote: >>>>>>> +def is_host_resolvable(fqdn): >>>>>>> + if not isinstance(fqdn, DNSName): >>>>>>> + fqdn = DNSName(fqdn) >>>>>>> + for rdtype in (rdatatype.A, rdatatype.AAAA): >>>>>>> + try: >>>>>>> + resolver.query(fqdn.make_absolute(), rdtype) >>>>>>> + except DNSException: >>>>>>> + continue >>>>>>> + else: >>>>>>> + return True >>>>>>> + >>>>>>> + return False >>>>>>> >>>>>> >>>>>> NACK, you are re-introducing duplicate function which was removed in >>>>>> 498471e4aed1367b72cd74d15811d0584a6ee268. >>>>>> >>>>>> Please amend the patch ASAP to use new verify_host_resolvable() function >>>>>> so I >>>>>> can test it and get it into 4.3. >>>>>> >>>>> Updated patches attached. >>>> >>>> Hmm, my review script screams: >>>> >>>> Detected base branch: origin/master >>>> Checks will be made against following base commit: 848912a add missing >>>> /ipaplatform/constants.py to .gitignore >>>> Writing API to API.txt >>>> ./ipalib/plugins/dns.py:2127:9: E124 closing bracket does not match visual >>>> indentation >>>> ./ipalib/plugins/dns.py:2711:13: E128 continuation line under-indented for >>>> visual indent >>>> ./ipalib/plugins/dns.py:2713:9: E124 closing bracket does not match visual >>>> indentation >>>> ./ipalib/plugins/dns.py:2716:13: E128 continuation line under-indented for >>>> visual indent >>>> ./ipapython/ipautil.py:928:1: E302 expected 2 blank lines, found 1 >>>> ./ipapython/ipautil.py:948:17: E128 continuation line under-indented for >>>> visual indent >>>> ./ipapython/ipautil.py:956:1: E302 expected 2 blank lines, found 1 >>>> ./ipapython/ipautil.py:997:80: E501 line too long (83 > 79 characters) >>>> ./ipapython/ipautil.py:998:80: E501 line too long (83 > 79 characters) >>>> ./ipaserver/install/bindinstance.py:49:9: E128 continuation line >>>> under-indented for visual indent >>>> ./ipaserver/install/bindinstance.py:292:80: E501 line too long (80 > 79 >>>> characters) >>>> ./ipaserver/install/bindinstance.py:438:19: E128 continuation line >>>> under-indented for visual indent >>>> ./ipaserver/install/server/common.py:271:63: E261 at least two spaces before >>>> inline comment >>>> ./ipaserver/install/server/common.py:271:80: E501 line too long (90 > 79 >>>> characters) >>>> ERROR: PEP8 --diff failed, continuing ... >>>> ERROR: API.txt was changed without a change in VERSION, continuing ... >>>> Summary of detected errors: >>>> ERROR: PEP8 --diff failed >>>> ERROR: API.txt was changed without a change in VERSION >>>> >>>> Please fix it :-) >>>> >>>> Make make this more pleasant for you, you can use (and review) my attached >>>> patch. It adds 'review' target to Makefile, it will do the same checks as >>>> I do. >>>> >>> >>> Unfortunately your tool does not work for me (output w/ -o xtrace attached). >>> Anyway I have incremented API version and fixed most of the pep8 errors except >>> for E124 twice in ipalib/plugins/dns.py as it seems to be convention in the >>> file and E501 twice in ipapython/ipautil.py because IMO breaking string >>> constant into multiple lines does not help readability. >>> >>> Updated patches also attached. >> >> We are almost there, but NACK for now. >> >> 1) Following attempt to install IPA explodes. Please note that >> dom-245.idm.lab.eng.brq.redhat.com DNS domain is delegated to the IPA server >> before installation is started, so it gives 'timeout' or possibly SERVFAIL. >> >> # ipa-server-install --ds-password=root4lab --admin-password=root4lab >> --setup-dns --forwarder=10.38.5.26 --no-reverse --allow-zone-overlap >> --domain=dom-245.idm.lab.eng.brq.redhat.com >> --realm=DOM-245.IDM.LAB.ENG.BRQ.REDHAT.COM >> [...] >> >> Configuring DNS (named) >> [1/11]: generating rndc key file >> [2/11]: adding DNS container >> [3/11]: setting up our zone >> [error] InvocationError: DNS check for domain >> dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation timed out after >> 30.000453949 seconds. Please make sure that the domain is properly delegated >> to this IPA server. >> ipa.ipapython.install.cli.install_tool(Server): ERROR DNS check for domain >> dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation timed out after >> 30.000453949 seconds. Please make sure that the domain is properly delegated >> to this IPA server. >> ipa.ipapython.install.cli.install_tool(Server): ERROR The >> ipa-server-install command failed. See /var/log/ipaserver-install.log for more >> information >> >> 2015-12-09T17:15:37Z DEBUG File >> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in execute >> return_value = self.run() >> File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line 318, >> in run >> cfgr.run() >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 310, >> in run >> self.execute() >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 332, >> in execute >> for nothing in self._executor(): >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, >> in __runner >> self._handle_exception(exc_info) >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, >> in _handle_exception >> six.reraise(*exc_info) >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, >> in __runner >> step() >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, >> in >> step = lambda: next(self.__gen) >> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, >> in run_generator_with_yield_from >> six.reraise(*exc_info) >> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, >> in run_generator_with_yield_from >> value = gen.send(prev_value) >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 571, >> in _configure >> next(executor) >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, >> in __runner >> self._handle_exception(exc_info) >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 449, >> in _handle_exception >> self.__parent._handle_exception(exc_info) >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, >> in _handle_exception >> six.reraise(*exc_info) >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 446, >> in _handle_exception >> super(ComponentBase, self)._handle_exception(exc_info) >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, >> in _handle_exception >> six.reraise(*exc_info) >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, >> in __runner >> step() >> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, >> in >> step = lambda: next(self.__gen) >> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, >> in run_generator_with_yield_from >> six.reraise(*exc_info) >> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, >> in run_generator_with_yield_from >> value = gen.send(prev_value) >> >> File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line >> 63, in _install >> for nothing in self._installer(self.parent): >> File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >> line 1471, in main >> install(self) >> File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >> line 265, in decorated >> func(installer) >> File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >> line 958, in install >> dns.install(False, False, options) >> File "/usr/lib/python2.7/site-packages/ipaserver/install/dns.py", line 322, >> in install >> bind.create_instance() >> File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >> line 680, in create_instance >> self.start_creation() >> File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line >> 447, in start_creation >> run_step(full_msg, method) >> File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line >> 437, in run_step >> method() >> File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >> line 805, in __setup_zone >> ns_hostname=self.api.env.host, force=True, api=self.api) >> File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >> line 331, in add_zone >> force=force) >> File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 447, in >> __call__ >> ret = self.run(*args, **options) >> File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 764, in run >> return self.execute(*args, **options) >> File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line 2781, in >> execute >> result = super(dnszone_add, self).execute(*keys, **options) >> File "/usr/lib/python2.7/site-packages/ipalib/plugins/baseldap.py", line >> 1233, in execute >> *keys, **options) >> File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line 2747, in >> pre_callback >> ldap, dn, entry_attrs, attrs_list, *keys, **options) >> File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line 2153, in >> pre_callback >> raise errors.InvocationError(e.message) >> >> >> 2) Please print 'Checking DNS domain , please wait ...' when validating >> domain parameter in installer. The timeout is 30 seconds and users may be >> nervous when the installed blocks for 30 seconds without a visible reason. >> >> Precedent for this is in check_forwarders() in >> ipaserver/install/bindinstance.py . Encapsulating check_zone_overlap() in >> auxiliary method may be an option. >> >> >> 3) Timeout is a fatal error instead of warning. We have been discussing this >> and it should really be just a warning. >> >> >> 4) Nitpicks are attached in .patch file. >> >> >> 5) ipa dnszone-add checks work as expected, good job! >> >> >> Thank you for patience! >> > Updated patches attached. Added patch introducing --auto-reverse option that > should generate needed reverse zones automatically. > NACK: $ ipa-server-install --setup-dns Do you want to configure the reverse zone? [yes]: ipa.ipapython.install.cli.install_tool(Server): ERROR 'CheckedIPAddress' object has no attribute 'split' -- Petr^2 Spacek From lslebodn at redhat.com Thu Dec 10 17:27:10 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Thu, 10 Dec 2015 18:27:10 +0100 Subject: [Freeipa-devel] [PATCH 0115] fix error message assertion in negative forced client reenrollment tests In-Reply-To: <56670D22.1050109@redhat.com> References: <56670D22.1050109@redhat.com> Message-ID: <20151210172710.GA15788@mail.corp.redhat.com> On (08/12/15 18:02), Martin Babinsky wrote: >This patch fixes the assertions in negative test cases of >'test_forced_client_reenrollment' CI test suite. > >On ipa-4-2 it fixes https://fedorahosted.org/freeipa/ticket/5511 and makes >all 8 tests in this suite green, shiny and happy again. > >It also fixes negative test cases on master branch, but the positive cases >are broken there due to https://fedorahosted.org/freeipa/ticket/5528 > >-- >Martin^3 Babinsky >From eb152f6996a8b653d8676ade826e806898fdf556 Mon Sep 17 00:00:00 2001 >From: Martin Babinsky >Date: Tue, 8 Dec 2015 17:00:11 +0100 >Subject: [PATCH] fix error message assertion in negative forced client > reenrollment tests > >https://fedorahosted.org/freeipa/ticket/5511 > >--- > ipatests/test_integration/test_forced_client_reenrollment.py | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > >diff --git a/ipatests/test_integration/test_forced_client_reenrollment.py b/ipatests/test_integration/test_forced_client_reenrollment.py >index e1edff9b7f2d535a341d1e8ca55917012943818e..df0e90526c5c8dd78d3af9d7ddb7c9bdbf5a2268 100644 >--- a/ipatests/test_integration/test_forced_client_reenrollment.py >+++ b/ipatests/test_integration/test_forced_client_reenrollment.py >@@ -198,7 +198,7 @@ class TestForcedClientReenrollment(IntegrationTest): > assert 'IPA Server: %s' % server.hostname in result.stderr_text > > if expect_fail: >- err_msg = 'Kerberos authentication failed using keytab' >+ err_msg = "Kerberos authentication failed: " Test passed for freeipa-4.2: * fedora 23 with krb5-1.14 * rhel7.2 with krb5-1.13 ACK LS From mbasti at redhat.com Thu Dec 10 18:40:02 2015 From: mbasti at redhat.com (Martin Basti) Date: Thu, 10 Dec 2015 19:40:02 +0100 Subject: [Freeipa-devel] [PATCH 0376] KRA: add RA cert during replica promotion Message-ID: <5669C702.4080204@redhat.com> https://fedorahosted.org/freeipa/ticket/5512 patch attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0376-Install-RA-cert-during-replica-promotion.patch Type: text/x-patch Size: 2241 bytes Desc: not available URL: From jcholast at redhat.com Fri Dec 11 06:08:53 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 11 Dec 2015 07:08:53 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <566992B1.6030702@redhat.com> References: <5667EE2C.2050100@redhat.com> <56683238.3050700@redhat.com> <56684B03.2020905@redhat.com> <56693C49.6030508@redhat.com> <566992B1.6030702@redhat.com> Message-ID: <566A6875.1070300@redhat.com> On 10.12.2015 15:56, Martin Babinsky wrote: > On 12/10/2015 09:48 AM, Jan Cholasta wrote: >> On 9.12.2015 16:38, Jan Cholasta wrote: >>> On 9.12.2015 14:52, Jan Cholasta wrote: >>>> On 9.12.2015 10:02, Jan Cholasta wrote: >>>>> Hi, >>>>> >>>>> the attached patches fix >>>>> . >>>> >>>> Note that this needs selinux-policy fix to work, so put SELinux into >>>> permissive mode for testing: >>>> . >>> >>> Updated patches attached. >> >> I screwed up a change in patch 524 and accidentally included a chunk of >> code in patch 525 that doesn't belong in it. >> >> Updated patches attached. >> >> >> > > Patches work as expected and I was not able to find any functional problem. > > I have a question about the naming of the oddjob helper script: the one > related to trusts is named 'com.redhat.idm.trust-fetch-domains', and the > conncheck runner is named 'org.freeipa.server.conncheck'. I don't want > to start another bikeshedding conversation but shouldn't we named them > in a consistent fashion (either rename the first one in separate patch > or rename the new helper to com.redhat.idm.server.conncheck)? > > I understand that as an upstream, we should go with the 'org.freeipa.*' > convention, but having two helpers with different prefixes makes me sad. If you look at the larger picture, org.freeipa is the consistent name. It makes me sad as well, but mistakes should be corrected. This is similar to how we use PEP8 in new code, but do not fix it in old code just for the sake of fixing it. > > That is a nitpick though, it does not affect the overall functionality > of the patches so ACK. Thanks for the review. The current patch 523 breaks the trusts oddjob with SELinux in enforcing mode, I will send an update which corrects that, until bug 1289930 is fixed. -- Jan Cholasta From jcholast at redhat.com Fri Dec 11 06:12:23 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 11 Dec 2015 07:12:23 +0100 Subject: [Freeipa-devel] [PATCHES 509-514] replica promotion: use host credentials when setting up replication In-Reply-To: <5669859D.2050202@redhat.com> References: <564DE017.4020408@redhat.com> <1447951434.5132.27.camel@redhat.com> <564F4105.1040204@redhat.com> <1448038723.5132.59.camel@redhat.com> <5652C63E.5090206@redhat.com> <1448289289.7892.13.camel@redhat.com> <565324C4.1090007@redhat.com> <1448290024.7892.15.camel@redhat.com> <565D823E.5090600@redhat.com> <56604D94.7000705@redhat.com> <566167DE.9090204@redhat.com> <56619F5C.5000900@redhat.com> <5661A04E.9080409@redhat.com> <5661A593.5030003@redhat.com> <1449244714.3445.67.camel@redhat.com> <56649B78.1080805@redhat.com> <566531EF.7000603@redhat.com> <56684B33.3090103@redhat.com> <56692254.3060209@redhat.com> <5669859D.2050202@redhat.com> Message-ID: <566A6947.2010803@redhat.com> On 10.12.2015 15:01, Martin Babinsky wrote: > On 12/10/2015 07:57 AM, Jan Cholasta wrote: >> On 9.12.2015 16:39, Jan Cholasta wrote: >>> On 7.12.2015 08:14, Jan Cholasta wrote: >>>> On 6.12.2015 21:32, Martin Basti wrote: >>>>> >>>>> >>>>> On 04.12.2015 16:58, Simo Sorce wrote: >>>>>> On Fri, 2015-12-04 at 15:39 +0100, Jan Cholasta wrote: >>>>>>> On 4.12.2015 15:16, Jan Cholasta wrote: >>>>>>>> On 4.12.2015 15:12, Jan Cholasta wrote: >>>>>>>>> On 4.12.2015 11:15, Petr Vobornik wrote: >>>>>>>>>> On 12/03/2015 03:11 PM, Martin Basti wrote: >>>>>>>>>>> >>>>>>>>>>> On 01.12.2015 12:19, Jan Cholasta wrote: >>>>>>>>>>>> On 23.11.2015 15:47, Simo Sorce wrote: >>>>>>>>>>>>> On Mon, 2015-11-23 at 15:37 +0100, Jan Cholasta wrote: >>>>>>>>>>>>>> Ad alternative is to add the host to ipaservers before the >>>>>>>>>>>>>> checks >>>>>>>>>>>>>> are >>>>>>>>>>>>>> done and remove it again if any of them fail. >>>>>>>>>>>>> Too error prone, I am ok with the current way in your patches >>>>>>>>>>>>> until/unless I can think of a fail safe way. :-) >>>>>>>>>>>> Updated patches attached. Note that 520 should be applied >>>>>>>>>>>> between 509 >>>>>>>>>>>> and 510. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> Functional ACK >>>>>>>>>>> >>>>>>>>>> Simo, do you want to review the ACIs or other things it the >>>>>>>>>> patches? Or >>>>>>>>>> can the patches be pushed? >>>>>>>>> There were no changes in the ACIs since last time. >>>>>>>> Actually, memberPrincipal was removed from the "IPA server hosts >>>>>>>> can >>>>>>>> manage own Custodia secrets" ACI, as per Simo's request. >>>>>>>> >>>>>>>>> Rebased patches attached. >>>>>>> Note that 520 should still be applied between 509 and 510. >>>>>>> >>>>>> LGTM >>>>>> >>>>> ACK >>>> >>>> Thanks. >>>> >>>> Pushed to master: 01ddf51df76f3298499973355c5461727e46ab5b >>> >>> Martin Babinsky found out that ipaservers is not created early enough >>> when installing a replica of a 4.2 or older server which causes a crash. >>> >>> The attached patch fixes that. >> >> Actually I don't like how I fixed that, here's an updated patch. >> >> Also, I noticed that replica promotion fails too late in domain level 0. >> Fixed as well. >> >> >> > > ACK to both patches. Thanks. Pushed to master: b4a78db4e7d06237a715f088d1b914b47eccf32f -- Jan Cholasta From jcholast at redhat.com Fri Dec 11 06:19:32 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 11 Dec 2015 07:19:32 +0100 Subject: [Freeipa-devel] [PATCH 0098-0099] domain level 1 topology checks during IPA server uninstall In-Reply-To: <56684726.7020108@redhat.com> References: <564E04BE.1020304@redhat.com> <56586187.5090402@redhat.com> <565C3399.5030109@redhat.com> <565C6E8C.2090900@redhat.com> <565C844B.7050901@redhat.com> <565C8A57.2000305@redhat.com> <565CA4B8.1010701@redhat.com> <565DA35D.6010804@redhat.com> <565EBDA3.40600@redhat.com> <565EED67.3040405@redhat.com> <565EEDDC.8060304@redhat.com> <565EEFEC.1080703@redhat.com> <56683F2B.8040000@redhat.com> <56684726.7020108@redhat.com> Message-ID: <566A6AF4.8010806@redhat.com> On 9.12.2015 16:22, Martin Babinsky wrote: > On 12/09/2015 03:48 PM, Jan Cholasta wrote: >> On 2.12.2015 14:19, Martin Basti wrote: >>> >>> >>> On 02.12.2015 14:10, Martin Basti wrote: >>>> >>>> >>>> On 02.12.2015 14:08, Martin Babinsky wrote: >>>>> On 12/02/2015 10:45 AM, Martin Babinsky wrote: >>>>>> On 12/01/2015 02:40 PM, Martin Babinsky wrote: >>>>>>> On 11/30/2015 08:34 PM, Martin Basti wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 30.11.2015 18:41, Martin Babinsky wrote: >>>>>>>>> On 11/30/2015 06:15 PM, Martin Basti wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 30.11.2015 16:43, Martin Babinsky wrote: >>>>>>>>>>> On 11/30/2015 12:31 PM, Jan Cholasta wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> On 27.11.2015 14:58, Martin Babinsky wrote: >>>>>>>>>>>>> On 11/19/2015 06:19 PM, Martin Babinsky wrote: >>>>>>>>>>>>>> These two patches fix the following tickets: >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5377 >>>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5409 >>>>>>>>>>>>>> >>>>>>>>>>>>>> I have added a new option '--ignore-disconnected-topology' >>>>>>>>>>>>>> which >>>>>>>>>>>>>> forces >>>>>>>>>>>>>> IPA master uninstall despite reported errors in topology. >>>>>>>>>>>>>> I'm not >>>>>>>>>>>>>> quite >>>>>>>>>>>>>> sure if we want to flood ipa-server-install with >>>>>>>>>>>>>> uninstall-specific >>>>>>>>>>>>>> options, maybe it is better to skip the check in unattended >>>>>>>>>>>>>> mode >>>>>>>>>>>>>> and >>>>>>>>>>>>>> just print a warning about disconnected topology and what >>>>>>>>>>>>>> to do >>>>>>>>>>>>>> about >>>>>>>>>>>>>> it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I would like to hear your opinions about this. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> Attaching rebased and updated patches. >>>>>>>>>>>> >>>>>>>>>>>> Patch 0098: LGTM >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Patch 0099: >>>>>>>>>>>> >>>>>>>>>>>> a) This check should be done in Server.__init__() rather than >>>>>>>>>>>> install_check(): >>>>>>>>>>>> >>>>>>>>>>>> + if options.ignore_disconnected_topology: >>>>>>>>>>>> + print("'--ignore-disconnected-topology' is used only >>>>>>>>>>>> during " >>>>>>>>>>>> + "uninstallation") >>>>>>>>>>>> + sys.exit(1) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> b) >>>>>>>>>>>> s/--ignore-disconnected-topology/--ignore-topology-disconnect/, >>>>>>>>>>>> for >>>>>>>>>>>> consistency with other options, e.g. --no-ui-redirect. >>>>>>>>>>>> >>>>>>>>>>>> Maybe even shorten it to --ignore-topology? But we probably >>>>>>>>>>>> don't >>>>>>>>>>>> want >>>>>>>>>>>> people to use this option much, so it might be better to >>>>>>>>>>>> keep it >>>>>>>>>>>> long? >>>>>>>>>>>> >>>>>>>>>>> I would rather leave it with the long option name, it is more >>>>>>>>>>> apparent >>>>>>>>>>> what this switch should be around. >>>>>>>>>>>> >>>>>>>>>>>> c) I'm fine with uninstall options, you can remove the TODO: >>>>>>>>>>>> >>>>>>>>>>>> + # TODO: ask jcholast about uninstallation options >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Honza >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Attaching updated patches. >>>>>>>>>>> >>>>>>>>>> NACK >>>>>>>>>> >>>>>>>>>> ipa-server-install --uninstall >>>>>>>>>> >>>>>>>>>> 2015-11-30T17:14:30Z DEBUG Destroyed connection >>>>>>>>>> context.ldap2_140081152041808 >>>>>>>>>> 2015-11-30T17:14:30Z DEBUG Traceback (most recent call last): >>>>>>>>>> File >>>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>>>>>>>> line 91, in _handle_exception >>>>>>>>>> super(Continuous, self)._handle_exception(exc_info) >>>>>>>>>> File >>>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>>>> line 387, in _handle_exception >>>>>>>>>> six.reraise(*exc_info) >>>>>>>>>> File >>>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>>>> line 439, in _handle_exception >>>>>>>>>> super(ComponentBase, self)._handle_exception(exc_info) >>>>>>>>>> File >>>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>>>> line 387, in _handle_exception >>>>>>>>>> six.reraise(*exc_info) >>>>>>>>>> File >>>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>>>> line 355, in __runner >>>>>>>>>> step() >>>>>>>>>> File >>>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>>>> line 352, in >>>>>>>>>> step = lambda: next(self.__gen) >>>>>>>>>> File >>>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>>>>> line 81, in run_generator_with_yield_from >>>>>>>>>> six.reraise(*exc_info) >>>>>>>>>> File >>>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>>>>> line 59, in run_generator_with_yield_from >>>>>>>>>> value = gen.send(prev_value) >>>>>>>>>> File >>>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", >>>>>>>>>> line 71, in _uninstall >>>>>>>>>> for nothing in self._uninstaller(self.parent): >>>>>>>>>> File >>>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> line 1409, in main >>>>>>>>>> uninstall_check(self) >>>>>>>>>> File >>>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> line 265, in decorated >>>>>>>>>> func(installer) >>>>>>>>>> File >>>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> line 1140, in uninstall_check >>>>>>>>>> api, masters, options.ignore_disconnected_topology) >>>>>>>>>> AttributeError: 'uninstaller(Server)' object has no attribute >>>>>>>>>> 'ignore_disconnected_topology' >>>>>>>>>> >>>>>>>>>> 2015-11-30T17:14:30Z ERROR 'uninstaller(Server)' object has no >>>>>>>>>> attribute >>>>>>>>>> 'ignore_disconnected_topology' >>>>>>>>>> 2015-11-30T17:14:30Z INFO The ipa-server-install command was >>>>>>>>>> successful >>>>>>>>>> >>>>>>>>> >>>>>>>>> Sorry I have failed horribly during option rename. Attaching patch >>>>>>>>> that should actually work. >>>>>>>>> >>>>>>>> functional ACK >>>>>>> Attaching rebased patches reflecting the recent changes in the >>>>>>> handling >>>>>>> of managed topology suffixes handling. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> Jan had some more suggestions to the patches. Attaching updated >>>>>> version. >>>>>> >>>>>> >>>>>> >>>>> Attaching updated patch 99 with fixed error message. >>>>> >>>> Pushed to master: b8c619a7139bd7b65caa03b68431e22791ff19bf >>>> >>> ACK :-) >> >> I was doing some unrelated testing with domain level 0 and forgot to >> remove --ignore-topology disconnect from my command line before >> uninstalling server, which gave me an error that the option cannot be >> used in domain level 0 and I had to re-run ipa-server-install >> --uninstall with the option removed. >> >> IMO it would be better for UX if the option was ignored in domain level >> 0, since topology disconnects are *always* ignored in domain level 0. >> (Right?) >> >> The attached patch fixes that. >> > > ACK. Thanks. Pushed to master: ccb2f523134af5246450edd04874af2d34d896cc -- Jan Cholasta From jcholast at redhat.com Fri Dec 11 07:03:57 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 11 Dec 2015 08:03:57 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <566A6875.1070300@redhat.com> References: <5667EE2C.2050100@redhat.com> <56683238.3050700@redhat.com> <56684B03.2020905@redhat.com> <56693C49.6030508@redhat.com> <566992B1.6030702@redhat.com> <566A6875.1070300@redhat.com> Message-ID: <566A755D.6000701@redhat.com> On 11.12.2015 07:08, Jan Cholasta wrote: > On 10.12.2015 15:56, Martin Babinsky wrote: >> On 12/10/2015 09:48 AM, Jan Cholasta wrote: >>> On 9.12.2015 16:38, Jan Cholasta wrote: >>>> On 9.12.2015 14:52, Jan Cholasta wrote: >>>>> On 9.12.2015 10:02, Jan Cholasta wrote: >>>>>> Hi, >>>>>> >>>>>> the attached patches fix >>>>>> . >>>>> >>>>> Note that this needs selinux-policy fix to work, so put SELinux into >>>>> permissive mode for testing: >>>>> . >>>> >>>> Updated patches attached. >>> >>> I screwed up a change in patch 524 and accidentally included a chunk of >>> code in patch 525 that doesn't belong in it. >>> >>> Updated patches attached. >>> >>> >>> >> >> Patches work as expected and I was not able to find any functional >> problem. >> >> I have a question about the naming of the oddjob helper script: the one >> related to trusts is named 'com.redhat.idm.trust-fetch-domains', and the >> conncheck runner is named 'org.freeipa.server.conncheck'. I don't want >> to start another bikeshedding conversation but shouldn't we named them >> in a consistent fashion (either rename the first one in separate patch >> or rename the new helper to com.redhat.idm.server.conncheck)? >> >> I understand that as an upstream, we should go with the 'org.freeipa.*' >> convention, but having two helpers with different prefixes makes me sad. > > If you look at the larger picture, org.freeipa is the consistent name. > It makes me sad as well, but mistakes should be corrected. This is > similar to how we use PEP8 in new code, but do not fix it in old code > just for the sake of fixing it. > >> >> That is a nitpick though, it does not affect the overall functionality >> of the patches so ACK. > > Thanks for the review. The current patch 523 breaks the trusts oddjob > with SELinux in enforcing mode, I will send an update which corrects > that, until bug 1289930 is fixed. Updated patches attached. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-523.3-build-put-oddjob-scripts-into-separate-directory.patch Type: text/x-patch Size: 1690 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-524.3-replica-install-add-remote-connection-check-over-API.patch Type: text/x-patch Size: 29120 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-525.3-replica-promotion-use-host-credentials-for-connectio.patch Type: text/x-patch Size: 2344 bytes Desc: not available URL: From lslebodn at redhat.com Fri Dec 11 08:22:39 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Fri, 11 Dec 2015 09:22:39 +0100 Subject: [Freeipa-devel] [PATCH 0069] Add 'review' target for make In-Reply-To: <5669B09B.9000001@redhat.com> References: <56683B03.40703@redhat.com> <5669B09B.9000001@redhat.com> Message-ID: <20151211082239.GA13173@mail.corp.redhat.com> On (10/12/15 18:04), Petr Spacek wrote: >On 9.12.2015 15:30, Petr Spacek wrote: >> Hello, >> >> this patch automates some of sanity checks proposed by Petr Vobornik. >> >> 'make review' should be used in root of clean Git tree which has patches under >> review applied. >> +1 for automatisation -1 for name. Your script does not review[1] anything. Code review(peer review) is a job for humans. What do you think about alternative names: "review-helper", "sanity-check" ... >> Magic in review.sh attempts to detect nearest remote branch which can be used >> as diff base for review. Please see review.sh for further details. > >And here is the patch! :-) > >-- >Petr^2 Spacek [1] https://en.wikipedia.org/wiki/Code_review >From 52393bf2bb0ad74dbe37e496b1fd41a6ab22bd90 Mon Sep 17 00:00:00 2001 >From: Petr Spacek >Date: Tue, 8 Dec 2015 12:06:33 +0100 >Subject: [PATCH] Add 'review' target for make. It automates following tasks: > >- check if ACI.txt and API.txt are up-to-date >- check if VERSION was changed if API was changed >- pep8 --diff does not produce new errors when ran on diff from origin/branch >- make lint does not produce errors >--- > Makefile | 4 ++++ > review.sh | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 68 insertions(+) > create mode 100755 review.sh > >diff --git a/Makefile b/Makefile >index d2c37f1597a011af2bd9ef1a4f7ce87ac59620a3..b85b3a5d6362150dcebf16745414c3d416c9547f 100644 >--- a/Makefile >+++ b/Makefile >@@ -79,6 +79,10 @@ check: bootstrap-autogen server tests > (cd $$subdir && $(MAKE) check) || exit 1; \ > done > >+# works only in Git tree >+review: version-update >+ ./review.sh >+ > bootstrap-autogen: version-update client-autogen > @echo "Building IPA $(IPA_VERSION)" > cd asn1; if [ ! -e Makefile ]; then ../autogen.sh --prefix=/usr --sysconfdir=/etc --localstatedir=/var --libdir=$(LIBDIR); fi >diff --git a/review.sh b/review.sh >new file mode 100755 >index 0000000000000000000000000000000000000000..3a5114fbe88d3aba8e926e049500f5ac5f41a83e >--- /dev/null >+++ b/review.sh >@@ -0,0 +1,64 @@ >+#!/bin/bash >+is_tree_clean() { >+ test "$(git status --porcelain "$@")" == "" >+ return $? >+} >+ >+log_error() { >+ echo "ERROR: ${1}, continuing ..." >+ errs=(${errs[@]} "ERROR: ${1}\n") >+} >+ >+set -o errexit -o nounset #-o xtrace >+ >+LOGFILE="$(mktemp --suff=.log)" >+exec > >(tee -i ${LOGFILE}) >+exec 2>&1 >+ >+echo -n "make lint is running ... " >+make --silent lint || log_error "make lint failed" >+ >+# Go backwards in history until you find a remote branch from which current >+# branch was created. This will be used as base for git diff. >+PATCHCNT=0 >+BASEBRANCH="" If base branch means the remote branch which was used for cloning the cerrent git HEAD than you can simplify it a littl bit. git rev-parse --symbolic-full-name @{upstream} >+CURRBRANCH="$(git branch --remote --contains)" >+while [ "${BASEBRANCH}" == "" ] >+do >+ BASEBRANCH="$(git branch --remote --contains "HEAD~${PATCHCNT}" | grep -v "^. ${CURRBRANCH}$" || :)" >+ PATCHCNT="$(expr "${PATCHCNT}" + 1)" Then for patch count you can use PATCHCNT=$(git log --oneline $BASEBRANCH..HEAD | wc -l) LS From pspacek at redhat.com Fri Dec 11 08:35:34 2015 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 11 Dec 2015 09:35:34 +0100 Subject: [Freeipa-devel] [PATCH 0070] Makefile: disable parallel build Message-ID: <566A8AD6.7070102@redhat.com> Hello, Makefile: disable parallel build IPA build system cannot cope with parallel build anyway, so this patch disables parallel build explicitly so it does not blow up when user has -j specified in default MAKEOPTS. -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0070-Makefile-disable-parallel-build.patch Type: text/x-patch Size: 781 bytes Desc: not available URL: From mkosek at redhat.com Fri Dec 11 08:36:50 2015 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 11 Dec 2015 09:36:50 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <5669A3D3.7050608@redhat.com> References: <56687174.8000305@redhat.com> <20151210081357.GB21550@mail.corp.redhat.com> <56695291.5090705@redhat.com> <566990E3.2080903@redhat.com> <5669A3D3.7050608@redhat.com> Message-ID: <566A8B22.5020101@redhat.com> On 12/10/2015 05:09 PM, Martin Basti wrote: > > > On 10.12.2015 15:49, Tomas Babej wrote: >> >> On 12/10/2015 11:23 AM, Martin Basti wrote: >>> >>> On 10.12.2015 09:13, Lukas Slebodnik wrote: >>>> On (09/12/15 19:22), Martin Basti wrote: >>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>> >>>>> Patch attached. >>>> >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 00:00:00 2001 >>>>> From: Martin Basti >>>>> Date: Wed, 9 Dec 2015 18:53:35 +0100 >>>>> Subject: [PATCH] Fix version comparison >>>>> >>>>> Use RPM library to compare vendor versions of IPA for redhat platform >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>> --- >>>>> freeipa.spec.in | 2 ++ >>>>> ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ >>>>> 2 files changed, 21 insertions(+) >>>>> >>>>> diff --git a/freeipa.spec.in b/freeipa.spec.in >>>>> index >>>>> 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 >>>>> >>>>> 100644 >>>>> --- a/freeipa.spec.in >>>>> +++ b/freeipa.spec.in >>>>> @@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} >>>>> Requires: gzip >>>>> Requires: python-gssapi >= 1.1.0 >>>>> Requires: custodia >>>>> +Requires: rpm-python >>>>> +Requires: rpmdevtools >>>> Could you explain why do you need the 2nd package? >>>> It does not contains any python modules >>>> and I cannot see usage of any binary in this patch >>>> >>>> LS >>> Thanks for this catch, it is actually located in yum package, I rather >>> copy stringToVersion function from there to IPA, to avoid dependency hell >>> >>> Updated patch attached. >>> >>> >> Looking good. The __cmp__ function, however, is not available in Python >> 3. As we will eventually support python3 in RHEL as well, maybe we >> should make sure even platform-dependent parts are python3 compatible? >> For the future's sake. >> >> Tomas >> > Thanks, > > python 3 compatible patch attached. I wonder why we have to add so much code here and reimplement RPM version checking if it is already provided by rpmdevtools: ~~~ $ /usr/bin/rpmdev-vercmp ipa-4.2.0-15.el7 4.2.0-15.el7_2.3; echo $? WARNING: hyphen in release1: 4.2.0-15.el7 rpmdev-vercmp rpmdev-vercmp rpmdev-vercmp # with no arguments, prompt Exit status is 0 if the EVR's are equal, 11 if EVR1 is newer, and 12 if EVR2 is newer. Other exit statuses indicate problems. ipa-4.2.0-15.el7 < 4.2.0-15.el7_2.3 12 ~~~ which could be directly called from __eq__ or __lt__, since we are in platform specific code anyway already. Martin From pspacek at redhat.com Fri Dec 11 08:46:43 2015 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 11 Dec 2015 09:46:43 +0100 Subject: [Freeipa-devel] [PATCH 0069] Add 'review' target for make In-Reply-To: <20151211082239.GA13173@mail.corp.redhat.com> References: <56683B03.40703@redhat.com> <5669B09B.9000001@redhat.com> <20151211082239.GA13173@mail.corp.redhat.com> Message-ID: <566A8D73.3080309@redhat.com> On 11.12.2015 09:22, Lukas Slebodnik wrote: > On (10/12/15 18:04), Petr Spacek wrote: >> On 9.12.2015 15:30, Petr Spacek wrote: >>> Hello, >>> >>> this patch automates some of sanity checks proposed by Petr Vobornik. >>> >>> 'make review' should be used in root of clean Git tree which has patches under >>> review applied. >>> > +1 for automatisation > -1 for name. > > Your script does not review[1] anything. Code review(peer review) is a job > for humans. What do you think about alternative > names: "review-helper", "sanity-check" ... I like short name but I respect your point, so I've updated the script to print: "No problems detected using lint, pep8, ./make{api,aci}, and VERSION" at the end. Now it should be clear what it did, so reviewer knows what steps were already done and what can be skipped during further manual review. Attached patch also adds 'assert-clean-tree' target which is run before lint target, so no time is wasted if the tree is not clean. Parallel build will execute and terminate both targets at the same time, but it saves time in both cases, the output is just not so nice. And IPA cannot be built in parallel anyway :-) >>> Magic in review.sh attempts to detect nearest remote branch which can be used >>> as diff base for review. Please see review.sh for further details. >> >> And here is the patch! :-) >> >> -- >> Petr^2 Spacek > > [1] https://en.wikipedia.org/wiki/Code_review > >>From 52393bf2bb0ad74dbe37e496b1fd41a6ab22bd90 Mon Sep 17 00:00:00 2001 >> From: Petr Spacek >> Date: Tue, 8 Dec 2015 12:06:33 +0100 >> Subject: [PATCH] Add 'review' target for make. It automates following tasks: >> >> - check if ACI.txt and API.txt are up-to-date >> - check if VERSION was changed if API was changed >> - pep8 --diff does not produce new errors when ran on diff from origin/branch >> - make lint does not produce errors >> --- >> Makefile | 4 ++++ >> review.sh | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> 2 files changed, 68 insertions(+) >> create mode 100755 review.sh >> >> diff --git a/Makefile b/Makefile >> index d2c37f1597a011af2bd9ef1a4f7ce87ac59620a3..b85b3a5d6362150dcebf16745414c3d416c9547f 100644 >> --- a/Makefile >> +++ b/Makefile >> @@ -79,6 +79,10 @@ check: bootstrap-autogen server tests >> (cd $$subdir && $(MAKE) check) || exit 1; \ >> done >> >> +# works only in Git tree >> +review: version-update >> + ./review.sh >> + >> bootstrap-autogen: version-update client-autogen >> @echo "Building IPA $(IPA_VERSION)" >> cd asn1; if [ ! -e Makefile ]; then ../autogen.sh --prefix=/usr --sysconfdir=/etc --localstatedir=/var --libdir=$(LIBDIR); fi >> diff --git a/review.sh b/review.sh >> new file mode 100755 >> index 0000000000000000000000000000000000000000..3a5114fbe88d3aba8e926e049500f5ac5f41a83e >> --- /dev/null >> +++ b/review.sh >> @@ -0,0 +1,64 @@ >> +#!/bin/bash >> +is_tree_clean() { >> + test "$(git status --porcelain "$@")" == "" >> + return $? >> +} >> + >> +log_error() { >> + echo "ERROR: ${1}, continuing ..." >> + errs=(${errs[@]} "ERROR: ${1}\n") >> +} >> + >> +set -o errexit -o nounset #-o xtrace >> + >> +LOGFILE="$(mktemp --suff=.log)" >> +exec > >(tee -i ${LOGFILE}) >> +exec 2>&1 >> + >> +echo -n "make lint is running ... " >> +make --silent lint || log_error "make lint failed" >> + >> +# Go backwards in history until you find a remote branch from which current >> +# branch was created. This will be used as base for git diff. >> +PATCHCNT=0 >> +BASEBRANCH="" > If base branch means the remote branch which was used for cloning the cerrent > git HEAD than you can simplify it a littl bit. > git rev-parse --symbolic-full-name @{upstream} Nice! I did not know this trick. Unfortunately it does not work for my use-case: I clone upstream repo, pick a branch to base review on, create my own branch from the upstream one, and then apply patches on top of that. Imagine that I'm going to run the tool on branch 'review-tool'. Git history tree looks like this: * d036f3e (HEAD -> review-tool) fixup! Add 'review' target for make. * c6d75b0 Makefile: disable parallel build | * 667d50b (dkupka-review) dns: Add --auto-reverse option. | * c85f176 dns: Check if domain already exists. | * a55cd76 dns: do not add (forward)zone if it is already resolvable. | * 0b0c529 (dns-no-forwarders) DNS: Make --no-forwarders option default. |/ * d2e8470 Add 'review' target for make. It automates following tasks: | * 101ffae (mbasti-review) Fix DNS tests: dns-resolve returns warning | * 52393bf Add 'review' target for make. It automates following tasks: |/ * 848912a (origin/master, master) add missing /ipaplatform/constants.py to .gitignore $ git rev-parse --symbolic-full-name @{upstream} errors out like this: fatal: no upstream configured for branch 'review-tool' I definitely agree that the code above is just an ugly hack, but I would like to keep current behavior so I do not need to change my workflow. >> +CURRBRANCH="$(git branch --remote --contains)" >> +while [ "${BASEBRANCH}" == "" ] >> +do >> + BASEBRANCH="$(git branch --remote --contains "HEAD~${PATCHCNT}" | grep -v "^. ${CURRBRANCH}$" || :)" >> + PATCHCNT="$(expr "${PATCHCNT}" + 1)" > Then for patch count you can use > PATCHCNT=$(git log --oneline $BASEBRANCH..HEAD | wc -l) -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0069-3-Add-review-target-for-make.patch Type: text/x-patch Size: 781 bytes Desc: not available URL: From dkupka at redhat.com Fri Dec 11 09:37:39 2015 From: dkupka at redhat.com (David Kupka) Date: Fri, 11 Dec 2015 10:37:39 +0100 Subject: [Freeipa-devel] [PATCH 0113] properly add ACIs to custodia container during IPA upgrade In-Reply-To: <5669428A.4070303@redhat.com> References: <5666A6B7.8010705@redhat.com> <5669428A.4070303@redhat.com> Message-ID: <566A9963.7050500@redhat.com> On 10/12/15 10:14, Martin Babinsky wrote: > On 12/08/2015 10:45 AM, Martin Babinsky wrote: >> fixes https://fedorahosted.org/freeipa/ticket/5524 >> >> >> > > Attaching updated patch with simpler fix suggested by Jan. > > > Thanks for the patch. Works for me, ACK. -- David Kupka From dkupka at redhat.com Fri Dec 11 11:35:33 2015 From: dkupka at redhat.com (David Kupka) Date: Fri, 11 Dec 2015 12:35:33 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <5669B20E.8000803@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> <5665843C.2090507@redhat.com> <56658C89.8050103@redhat.com> <56668D30.8060308@redhat.com> <5666BCBC.9040005@redhat.com> <5666E822.90103@redhat.com> <56682093.3010505@redhat.com> <56686AFE.4000401@redhat.com> <5669A8D8.5030405@redhat.com> <5669B20E.8000803@redhat.com> Message-ID: <566AB505.3060508@redhat.com> On 10/12/15 18:10, Petr Spacek wrote: > On 10.12.2015 17:31, David Kupka wrote: >> On 09/12/15 18:55, Petr Spacek wrote: >>> On 9.12.2015 13:37, David Kupka wrote: >>>> On 08/12/15 15:24, Petr Spacek wrote: >>>>> On 8.12.2015 12:19, David Kupka wrote: >>>>>> On 08/12/15 08:56, Petr Spacek wrote: >>>>>>> On 7.12.2015 14:41, David Kupka wrote: >>>>>>>> +def is_host_resolvable(fqdn): >>>>>>>> + if not isinstance(fqdn, DNSName): >>>>>>>> + fqdn = DNSName(fqdn) >>>>>>>> + for rdtype in (rdatatype.A, rdatatype.AAAA): >>>>>>>> + try: >>>>>>>> + resolver.query(fqdn.make_absolute(), rdtype) >>>>>>>> + except DNSException: >>>>>>>> + continue >>>>>>>> + else: >>>>>>>> + return True >>>>>>>> + >>>>>>>> + return False >>>>>>>> >>>>>>> >>>>>>> NACK, you are re-introducing duplicate function which was removed in >>>>>>> 498471e4aed1367b72cd74d15811d0584a6ee268. >>>>>>> >>>>>>> Please amend the patch ASAP to use new verify_host_resolvable() function >>>>>>> so I >>>>>>> can test it and get it into 4.3. >>>>>>> >>>>>> Updated patches attached. >>>>> >>>>> Hmm, my review script screams: >>>>> >>>>> Detected base branch: origin/master >>>>> Checks will be made against following base commit: 848912a add missing >>>>> /ipaplatform/constants.py to .gitignore >>>>> Writing API to API.txt >>>>> ./ipalib/plugins/dns.py:2127:9: E124 closing bracket does not match visual >>>>> indentation >>>>> ./ipalib/plugins/dns.py:2711:13: E128 continuation line under-indented for >>>>> visual indent >>>>> ./ipalib/plugins/dns.py:2713:9: E124 closing bracket does not match visual >>>>> indentation >>>>> ./ipalib/plugins/dns.py:2716:13: E128 continuation line under-indented for >>>>> visual indent >>>>> ./ipapython/ipautil.py:928:1: E302 expected 2 blank lines, found 1 >>>>> ./ipapython/ipautil.py:948:17: E128 continuation line under-indented for >>>>> visual indent >>>>> ./ipapython/ipautil.py:956:1: E302 expected 2 blank lines, found 1 >>>>> ./ipapython/ipautil.py:997:80: E501 line too long (83 > 79 characters) >>>>> ./ipapython/ipautil.py:998:80: E501 line too long (83 > 79 characters) >>>>> ./ipaserver/install/bindinstance.py:49:9: E128 continuation line >>>>> under-indented for visual indent >>>>> ./ipaserver/install/bindinstance.py:292:80: E501 line too long (80 > 79 >>>>> characters) >>>>> ./ipaserver/install/bindinstance.py:438:19: E128 continuation line >>>>> under-indented for visual indent >>>>> ./ipaserver/install/server/common.py:271:63: E261 at least two spaces before >>>>> inline comment >>>>> ./ipaserver/install/server/common.py:271:80: E501 line too long (90 > 79 >>>>> characters) >>>>> ERROR: PEP8 --diff failed, continuing ... >>>>> ERROR: API.txt was changed without a change in VERSION, continuing ... >>>>> Summary of detected errors: >>>>> ERROR: PEP8 --diff failed >>>>> ERROR: API.txt was changed without a change in VERSION >>>>> >>>>> Please fix it :-) >>>>> >>>>> Make make this more pleasant for you, you can use (and review) my attached >>>>> patch. It adds 'review' target to Makefile, it will do the same checks as >>>>> I do. >>>>> >>>> >>>> Unfortunately your tool does not work for me (output w/ -o xtrace attached). >>>> Anyway I have incremented API version and fixed most of the pep8 errors except >>>> for E124 twice in ipalib/plugins/dns.py as it seems to be convention in the >>>> file and E501 twice in ipapython/ipautil.py because IMO breaking string >>>> constant into multiple lines does not help readability. >>>> >>>> Updated patches also attached. >>> >>> We are almost there, but NACK for now. >>> >>> 1) Following attempt to install IPA explodes. Please note that >>> dom-245.idm.lab.eng.brq.redhat.com DNS domain is delegated to the IPA server >>> before installation is started, so it gives 'timeout' or possibly SERVFAIL. >>> >>> # ipa-server-install --ds-password=root4lab --admin-password=root4lab >>> --setup-dns --forwarder=10.38.5.26 --no-reverse --allow-zone-overlap >>> --domain=dom-245.idm.lab.eng.brq.redhat.com >>> --realm=DOM-245.IDM.LAB.ENG.BRQ.REDHAT.COM >>> [...] >>> >>> Configuring DNS (named) >>> [1/11]: generating rndc key file >>> [2/11]: adding DNS container >>> [3/11]: setting up our zone >>> [error] InvocationError: DNS check for domain >>> dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation timed out after >>> 30.000453949 seconds. Please make sure that the domain is properly delegated >>> to this IPA server. >>> ipa.ipapython.install.cli.install_tool(Server): ERROR DNS check for domain >>> dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation timed out after >>> 30.000453949 seconds. Please make sure that the domain is properly delegated >>> to this IPA server. >>> ipa.ipapython.install.cli.install_tool(Server): ERROR The >>> ipa-server-install command failed. See /var/log/ipaserver-install.log for more >>> information >>> >>> 2015-12-09T17:15:37Z DEBUG File >>> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in execute >>> return_value = self.run() >>> File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line 318, >>> in run >>> cfgr.run() >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 310, >>> in run >>> self.execute() >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 332, >>> in execute >>> for nothing in self._executor(): >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, >>> in __runner >>> self._handle_exception(exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, >>> in _handle_exception >>> six.reraise(*exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, >>> in __runner >>> step() >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, >>> in >>> step = lambda: next(self.__gen) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, >>> in run_generator_with_yield_from >>> six.reraise(*exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, >>> in run_generator_with_yield_from >>> value = gen.send(prev_value) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 571, >>> in _configure >>> next(executor) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, >>> in __runner >>> self._handle_exception(exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 449, >>> in _handle_exception >>> self.__parent._handle_exception(exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, >>> in _handle_exception >>> six.reraise(*exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 446, >>> in _handle_exception >>> super(ComponentBase, self)._handle_exception(exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, >>> in _handle_exception >>> six.reraise(*exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, >>> in __runner >>> step() >>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, >>> in >>> step = lambda: next(self.__gen) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, >>> in run_generator_with_yield_from >>> six.reraise(*exc_info) >>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, >>> in run_generator_with_yield_from >>> value = gen.send(prev_value) >>> >>> File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line >>> 63, in _install >>> for nothing in self._installer(self.parent): >>> File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>> line 1471, in main >>> install(self) >>> File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>> line 265, in decorated >>> func(installer) >>> File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>> line 958, in install >>> dns.install(False, False, options) >>> File "/usr/lib/python2.7/site-packages/ipaserver/install/dns.py", line 322, >>> in install >>> bind.create_instance() >>> File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>> line 680, in create_instance >>> self.start_creation() >>> File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line >>> 447, in start_creation >>> run_step(full_msg, method) >>> File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line >>> 437, in run_step >>> method() >>> File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>> line 805, in __setup_zone >>> ns_hostname=self.api.env.host, force=True, api=self.api) >>> File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>> line 331, in add_zone >>> force=force) >>> File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 447, in >>> __call__ >>> ret = self.run(*args, **options) >>> File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 764, in run >>> return self.execute(*args, **options) >>> File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line 2781, in >>> execute >>> result = super(dnszone_add, self).execute(*keys, **options) >>> File "/usr/lib/python2.7/site-packages/ipalib/plugins/baseldap.py", line >>> 1233, in execute >>> *keys, **options) >>> File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line 2747, in >>> pre_callback >>> ldap, dn, entry_attrs, attrs_list, *keys, **options) >>> File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line 2153, in >>> pre_callback >>> raise errors.InvocationError(e.message) >>> >>> >>> 2) Please print 'Checking DNS domain , please wait ...' when validating >>> domain parameter in installer. The timeout is 30 seconds and users may be >>> nervous when the installed blocks for 30 seconds without a visible reason. >>> >>> Precedent for this is in check_forwarders() in >>> ipaserver/install/bindinstance.py . Encapsulating check_zone_overlap() in >>> auxiliary method may be an option. >>> >>> >>> 3) Timeout is a fatal error instead of warning. We have been discussing this >>> and it should really be just a warning. >>> >>> >>> 4) Nitpicks are attached in .patch file. >>> >>> >>> 5) ipa dnszone-add checks work as expected, good job! >>> >>> >>> Thank you for patience! >>> >> Updated patches attached. Added patch introducing --auto-reverse option that >> should generate needed reverse zones automatically. >> > NACK: > > $ ipa-server-install --setup-dns > > Do you want to configure the reverse zone? [yes]: > ipa.ipapython.install.cli.install_tool(Server): ERROR 'CheckedIPAddress' > object has no attribute 'split' > Updated patches attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0064.9-dns-Check-if-domain-already-exists.patch Type: text/x-patch Size: 18424 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0058.12-dns-do-not-add-forward-zone-if-it-is-already-resolva.patch Type: text/x-patch Size: 12115 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0072.2-dns-Add-auto-reverse-option.patch Type: text/x-patch Size: 6372 bytes Desc: not available URL: From mbasti at redhat.com Fri Dec 11 11:36:31 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 11 Dec 2015 12:36:31 +0100 Subject: [Freeipa-devel] [TESTS][PATCH 0006] Add comments to stageuser plugin tests In-Reply-To: <5668131B.1070207@redhat.com> References: <564C7A23.5010500@redhat.com> <564C9E77.4050406@redhat.com> <564D88AF.8010500@redhat.com> <564D97AC.6050901@redhat.com> <564F004A.2060806@redhat.com> <5667E942.7030305@redhat.com> <5667F0A6.1090104@redhat.com> <5668028E.60902@redhat.com> <5668131B.1070207@redhat.com> Message-ID: <566AB53F.4090504@redhat.com> On 09.12.2015 12:40, Martin Babinsky wrote: > On 12/09/2015 11:29 AM, Lenka Doudova wrote: >> >> >> On 12/09/2015 10:13 AM, Martin Basti wrote: >>> >>> >>> On 09.12.2015 09:41, Lenka Doudova wrote: >>>> Hi, >>>> >>>> attaching fixed patches for master and ipa-4-2 branch. >>>> Also fixes test accordingly to >>>> https://fedorahosted.org/freeipa/ticket/5387. >>>> >>>> Lenka >>>> >>>> On 11/20/2015 12:13 PM, Martin Babinsky wrote: >>>>> On 11/19/2015 10:34 AM, Petr Viktorin wrote: >>>>>> On 11/19/2015 09:30 AM, Lenka Doudova wrote: >>>>>>> On 11/18/2015 04:51 PM, Martin Babinsky wrote: >>>>>>>> On 11/18/2015 02:16 PM, Lenka Doudova wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> here's a patch that adds a few comments to stageuser tests in >>>>>>>>> order to >>>>>>>>> allow easier determining of a problem when tests fail. >>>>>>>>> >>>>>>>>> Lenka >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> Hi Lenka, >>>>>>>> >>>>>>>> Firstly a technical detail: Python indexes lists from 0, so the >>>>>>>> comments in 'options_ok' do not correctly map to the test names >>>>>>>> anyway. >>>>>>>> >>>>>>>> I am also not sure if this patch is worth reviewing and pushing >>>>>>>> as it >>>>>>>> IMHO doesn't help in the identification of failed tests at all. >>>>>>>> >>>>>>>> This should be solved at more fundamental level. >>>>>>>> >>>>>>> Ouch, good point, I didn't realize. Sorry. >>>>>>> >>>>>>> Anyway, the issue is that even if tests are run in verbose mode, >>>>>>> you get >>>>>>> output like this: >>>>>>> >>>>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser27] >>>>>>> >>>>>>> >>>>>>> PASSED >>>>>>> >>>>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser28] >>>>>>> >>>>>>> >>>>>>> PASSED >>>>>>> >>>>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser29] >>>>>>> >>>>>>> >>>>>>> PASSED >>>>>>> >>>>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser210] >>>>>>> >>>>>>> >>>>>>> PASSED >>>>>>> >>>>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser211] >>>>>>> >>>>>>> >>>>>>> PASSED >>>>>>> >>>>>>> ipatests/test_xmlrpc/test_stageuser_plugin.py::TestStagedUser::test_create_attr[stageduser212] >>>>>>> >>>>>>> >>>>>>> PASSED >>>>>>> >>>>>>> >>>>>>> If some test fails, you can't really tell which command was the one >>>>>>> responsible for the fail. This should be solved by >>>>>>> https://fedorahosted.org/freeipa/ticket/5449. Until that's done, >>>>>>> though, >>>>>>> the only way to find out which command failed is looking at the >>>>>>> code and >>>>>>> finding which parameters were put into the command, which was not >>>>>>> much >>>>>>> possible without better commenting the test case (as I realized >>>>>>> last >>>>>>> week when David Kupka asked me to help him find the reason for >>>>>>> failed >>>>>>> tests). >>>>>>> Obviously I can rewrite the tests so there's 27 separate test >>>>>>> cases, one >>>>>>> for each parameter, instead of one method that 'unwraps' into 27 >>>>>>> test >>>>>>> cases, which would entirely eliminate the confusion. So far I >>>>>>> don't know >>>>>>> of a way to put 27 similar test cases in one method which would >>>>>>> allow >>>>>>> easy recognition of the test cases. >>>>>>> I'll wait with fixing the patch until further discussion. >>>>>> >>>>>> Hello, >>>>>> Pytest wants you to be a bit more explicit about how to name the >>>>>> parameters. (It "hides" dicts by default, because large dicts would >>>>>> make >>>>>> the output even more confusing than the numbers.) >>>>>> >>>>>> Please try the attached patch. >>>>>> Docs are at >>>>>> https://pytest.org/latest/fixture.html#parametrizing-a-fixture >>>>>> >>>>>> >>>>>> >>>>> This looks like a better approach IMHO, you can then see which >>>>> attribute/value was being checked. >>>>> >>>>> I would very much favor more descriptive test/fixture names in the >>>>> beginning. >>>>> >>>> >>>> >>>> >>> Hello, >>> >>> we use usually bottom posting on freeipa-devel please try to keep >>> reply in this way. >>> >>> Patch: >>> >>> I do not like the idea of separated lists, IMO it is hard to manage >>> and is easy to create mistakes. >>> >>> How about this (untested, just from top of my head): >>> http://fpaste.org/298994/65184014/ >>> >>> Martin >> >> Great idea, thanks. Fixed patches attached. >> >> Lenka >> >> > > Tests pass and code looks good, ACK. > Pushed to master: a66a2c5160dbc23cdeec55d17422812028939e16 Pushed to ipa-4-2: 75675fc71a148dcc17b025a80123aa01644f5297 From amarecek at redhat.com Fri Dec 11 11:40:11 2015 From: amarecek at redhat.com (=?utf-8?Q?Ale=C5=A1_Mare=C4=8Dek?=) Date: Fri, 11 Dec 2015 06:40:11 -0500 (EST) Subject: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin In-Reply-To: <964966782.26674717.1449760307850.JavaMail.zimbra@redhat.com> References: <20151106113201.5e6a7042@dhcp-24-123.brq.redhat.com> <5641BE6D.4060608@redhat.com> <20151110121350.06295a68@dhcp-24-103.brq.redhat.com> <56448E17.2010409@redhat.com> <20151203173843.37179230@vor2.netbox.priv> <869710129.24860544.1449498571599.JavaMail.zimbra@redhat.com> <20151210112952.3ddb7136@vor2.netbox.priv> <964966782.26674717.1449760307850.JavaMail.zimbra@redhat.com> Message-ID: <273613290.27075436.1449834011983.JavaMail.zimbra@redhat.com> There is some issue with "test_rename_to_too_long_login" test. It fails but actually this is false positive because it is possible to create login upto 255 characters. I don't know why test mentions 32 characters without any other modified setup. NACK for now. - alich - ----- Original Message ----- > From: "Ale? Mare?ek" > To: "Filip ?kola" > Cc: freeipa-devel at redhat.com, "Milan Kub?k" > Sent: Thursday, December 10, 2015 4:11:47 PM > Subject: Re: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin > > Ah, sorry, haven't realized there had been devel list attached. > Ok, there is some problem with \r\n in the patch. > Filip, please take a look at it... > Thanks... > - alich - > > ----- Original Message ----- > > From: "Filip ?kola" > > To: "Ale? Mare?ek" > > Cc: freeipa-devel at redhat.com, "Milan Kub?k" > > Sent: Thursday, December 10, 2015 11:29:52 AM > > Subject: Re: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin > > > > Hi, > > > > this if fixed. Also issues with test_stageuser_plugin caused by > > UserTracker changes should be fixed here. > > > > Filip > > > > > > On Mon, 7 Dec 2015 09:29:31 -0500 (EST) > > Ale? Mare?ek wrote: > > > > > NACK. > > > > > > $ ./make-lint > > > ************* Module ipatests.test_xmlrpc.test_user_plugin > > > ipatests/test_xmlrpc/test_user_plugin.py:42: > > > [E0611(no-name-in-module), ] No name 'ldaptracker' in module > > > 'ipatests.test_xmlrpc') > > > > > > $ grep ldaptracker ipatests/test_xmlrpc/test_user_plugin.py > > > from ipatests.test_xmlrpc.ldaptracker import Tracker > > > $ ls ipatests/test_xmlrpc/ldaptracker* > > > ls: cannot access ipatests/test_xmlrpc/ldaptracker*: No such file or > > > directory > > > > > > > > > ----- Original Message ----- > > > > From: "Filip ?kola" > > > > To: "Milan Kub?k" > > > > Cc: freeipa-devel at redhat.com > > > > Sent: Thursday, December 3, 2015 5:38:43 PM > > > > Subject: Re: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin > > > > > > > > Hi, > > > > > > > > sending corrected version. > > > > > > > > F. > > > > > > > > On Thu, 12 Nov 2015 14:03:19 +0100 > > > > Milan Kub?k wrote: > > > > > > > > > On 11/10/2015 12:13 PM, Filip ?kola wrote: > > > > > > Hi, > > > > > > > > > > > > fixed. > > > > > > > > > > > > F. > > > > > > > > > > > > On Tue, 10 Nov 2015 10:52:45 +0100 > > > > > > Milan Kub?k wrote: > > > > > > > > > > > >> On 11/09/2015 04:35 PM, Filip ?kola wrote: > > > > > >>> Another patch was applied in the meantime. > > > > > >>> > > > > > >>> Attaching an updated version. > > > > > >>> > > > > > >>> F. > > > > > >>> > > > > > >>> On Mon, 9 Nov 2015 13:35:02 +0100 > > > > > >>> Milan Kub?k wrote: > > > > > >>> > > > > > >>>> On 11/06/2015 11:32 AM, Filip ?kola wrote: > > > > > >>>> Hi, > > > > > >>>> the patch doesn't apply. > > > > > >>>> > > > > > >> Please fix this. > > > > > >> > > > > > >> ipatests/test_xmlrpc/test_user_plugin.py:1419: > > > > > >> [E0602(undefined-variable), > > > > > >> TestDeniedBindWithExpiredPrincipal.teardown_class] Undefined > > > > > >> variable 'user1') > > > > > >> > > > > > >> Also, use the version numbers for your changed patches. > > > > > >> > > > > > > > > > > > > > > > > > Thanks for the patch. Several issues: > > > > > > > > > > 1. Use dict.items instead of dict.iteritems, for python3 > > > > > compatibility > > > > > > > > > > 2. What is the purpose of TestPrepare class? The 'purge' methods > > > > > do not call any ipa commands. > > > > > Tracker.make_fixture should be used to make the Tracked resources > > > > > clean themselves up when they're out of scope. > > > > > > > > > > 3. Why reference the resources by hardcoded name if they have a > > > > > fixture representation? > > > > > > > > > > 4. Rewrite {create,delete}_test_group to a fixture. You may want > > > > > to use different scope (or not). > > > > > > > > > > 5. In `def atest_rename_to_invalid_login(self, user):` - use > > > > > pytest.skipif decorator and provide a reason if you must, > > > > > do not obfuscate method name in order not to run it. > > > > > > > > > > > > > > > > > > > > > > -- > > > > Manage your subscription for the Freeipa-devel mailing list: > > > > https://www.redhat.com/mailman/listinfo/freeipa-devel > > > > Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code > > > > > From tbabej at redhat.com Fri Dec 11 11:50:39 2015 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 11 Dec 2015 12:50:39 +0100 Subject: [Freeipa-devel] [PATCH 0113] properly add ACIs to custodia container during IPA upgrade In-Reply-To: <566A9963.7050500@redhat.com> References: <5666A6B7.8010705@redhat.com> <5669428A.4070303@redhat.com> <566A9963.7050500@redhat.com> Message-ID: <566AB88F.5000104@redhat.com> On 12/11/2015 10:37 AM, David Kupka wrote: > On 10/12/15 10:14, Martin Babinsky wrote: >> On 12/08/2015 10:45 AM, Martin Babinsky wrote: >>> fixes https://fedorahosted.org/freeipa/ticket/5524 >>> >>> >>> >> >> Attaching updated patch with simpler fix suggested by Jan. >> >> >> > Thanks for the patch. Works for me, ACK. > I was also finally able to reproduce it on a clear machine. Pushed to master: a66a2c5160dbc23cdeec55d17422812028939e16 From tbabej at redhat.com Fri Dec 11 11:55:53 2015 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 11 Dec 2015 12:55:53 +0100 Subject: [Freeipa-devel] [PATCH 0113] properly add ACIs to custodia container during IPA upgrade In-Reply-To: <566AB88F.5000104@redhat.com> References: <5666A6B7.8010705@redhat.com> <5669428A.4070303@redhat.com> <566A9963.7050500@redhat.com> <566AB88F.5000104@redhat.com> Message-ID: <566AB9C9.8040508@redhat.com> On 12/11/2015 12:50 PM, Tomas Babej wrote: > > > On 12/11/2015 10:37 AM, David Kupka wrote: >> On 10/12/15 10:14, Martin Babinsky wrote: >>> On 12/08/2015 10:45 AM, Martin Babinsky wrote: >>>> fixes https://fedorahosted.org/freeipa/ticket/5524 >>>> >>>> >>>> >>> >>> Attaching updated patch with simpler fix suggested by Jan. >>> >>> >>> >> Thanks for the patch. Works for me, ACK. >> > > I was also finally able to reproduce it on a clear machine. > > Pushed to master: a66a2c5160dbc23cdeec55d17422812028939e16 > Martin actually pushed this 30 minutes ago, actual commit hash is e130d35687a05cb3d2dd8708b76e7745e337c0c0. From mbasti at redhat.com Fri Dec 11 12:13:20 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 11 Dec 2015 13:13:20 +0100 Subject: [Freeipa-devel] [PATCH 0115] fix error message assertion in negative forced client reenrollment tests In-Reply-To: <56670D22.1050109@redhat.com> References: <56670D22.1050109@redhat.com> Message-ID: <566ABDE0.20409@redhat.com> On 08.12.2015 18:02, Martin Babinsky wrote: > This patch fixes the assertions in negative test cases of > 'test_forced_client_reenrollment' CI test suite. > > On ipa-4-2 it fixes https://fedorahosted.org/freeipa/ticket/5511 and > makes all 8 tests in this suite green, shiny and happy again. > > It also fixes negative test cases on master branch, but the positive > cases are broken there due to > https://fedorahosted.org/freeipa/ticket/5528 > > > I haven't received ACK email, but according this https://www.redhat.com/archives/freeipa-devel/2015-December/msg00367.html, ACK is already there Pushed to: master: 7c4ce9a09863d5364b4634fac03e83a4e9caae42 ipa-4-2: 742ffcde9920379da28c8b51060064c1bff03dc1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From lslebodn at redhat.com Fri Dec 11 12:30:56 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Fri, 11 Dec 2015 13:30:56 +0100 Subject: [Freeipa-devel] [PATCH 0069] Add 'review' target for make In-Reply-To: <566A8D73.3080309@redhat.com> References: <56683B03.40703@redhat.com> <5669B09B.9000001@redhat.com> <20151211082239.GA13173@mail.corp.redhat.com> <566A8D73.3080309@redhat.com> Message-ID: <20151211123056.GA17431@mail.corp.redhat.com> On (11/12/15 09:46), Petr Spacek wrote: >On 11.12.2015 09:22, Lukas Slebodnik wrote: >> On (10/12/15 18:04), Petr Spacek wrote: >>> On 9.12.2015 15:30, Petr Spacek wrote: >>>> Hello, >>>> >>>> this patch automates some of sanity checks proposed by Petr Vobornik. >>>> >>>> 'make review' should be used in root of clean Git tree which has patches under >>>> review applied. >>>> >> +1 for automatisation >> -1 for name. >> >> Your script does not review[1] anything. Code review(peer review) is a job >> for humans. What do you think about alternative >> names: "review-helper", "sanity-check" ... > >I like short name but I respect your point, so I've updated the script to print: >"No problems detected using lint, pep8, ./make{api,aci}, and VERSION" at the end. >Now it should be clear what it did, so reviewer knows what steps were already >done and what can be skipped during further manual review. > "review-helper", "sanity-check" are not long. long would be "aotomated-part-of-sanity-checks-from-FreeIPA-developers's-check-list" And pdf document https://pvoborni.fedorapeople.org/FreeIPAdeveloperschecklist.pdf already mentioned "Sanity checks" Please consider to not use confusing name. Neither as make target nor as name of script. BTW I cannot see sanity check for spec file. If it will be added to FreeIPAdeveloperschecklist.pdf than you might add it to the sript as well. >Attached patch also adds 'assert-clean-tree' target which is run before lint >target, so no time is wasted if the tree is not clean. Parallel build will >execute and terminate both targets at the same time, but it saves time in both >cases, the output is just not so nice. > >And IPA cannot be built in parallel anyway :-) > >>>> Magic in review.sh attempts to detect nearest remote branch which can be used >>>> as diff base for review. Please see review.sh for further details. >>> >>> And here is the patch! :-) >>> >>> -- >>> Petr^2 Spacek >> >> [1] https://en.wikipedia.org/wiki/Code_review >> >>>From 52393bf2bb0ad74dbe37e496b1fd41a6ab22bd90 Mon Sep 17 00:00:00 2001 >>> From: Petr Spacek >>> Date: Tue, 8 Dec 2015 12:06:33 +0100 >>> Subject: [PATCH] Add 'review' target for make. It automates following tasks: >>> >>> - check if ACI.txt and API.txt are up-to-date >>> - check if VERSION was changed if API was changed >>> - pep8 --diff does not produce new errors when ran on diff from origin/branch >>> - make lint does not produce errors >>> --- >>> Makefile | 4 ++++ >>> review.sh | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> 2 files changed, 68 insertions(+) >>> create mode 100755 review.sh >>> >>> diff --git a/Makefile b/Makefile >>> index d2c37f1597a011af2bd9ef1a4f7ce87ac59620a3..b85b3a5d6362150dcebf16745414c3d416c9547f 100644 >>> --- a/Makefile >>> +++ b/Makefile >>> @@ -79,6 +79,10 @@ check: bootstrap-autogen server tests >>> (cd $$subdir && $(MAKE) check) || exit 1; \ >>> done >>> >>> +# works only in Git tree >>> +review: version-update >>> + ./review.sh >>> + >>> bootstrap-autogen: version-update client-autogen >>> @echo "Building IPA $(IPA_VERSION)" >>> cd asn1; if [ ! -e Makefile ]; then ../autogen.sh --prefix=/usr --sysconfdir=/etc --localstatedir=/var --libdir=$(LIBDIR); fi >>> diff --git a/review.sh b/review.sh >>> new file mode 100755 >>> index 0000000000000000000000000000000000000000..3a5114fbe88d3aba8e926e049500f5ac5f41a83e >>> --- /dev/null >>> +++ b/review.sh >>> @@ -0,0 +1,64 @@ >>> +#!/bin/bash >>> +is_tree_clean() { >>> + test "$(git status --porcelain "$@")" == "" >>> + return $? >>> +} >>> + >>> +log_error() { >>> + echo "ERROR: ${1}, continuing ..." >>> + errs=(${errs[@]} "ERROR: ${1}\n") >>> +} >>> + >>> +set -o errexit -o nounset #-o xtrace >>> + >>> +LOGFILE="$(mktemp --suff=.log)" >>> +exec > >(tee -i ${LOGFILE}) >>> +exec 2>&1 >>> + >>> +echo -n "make lint is running ... " >>> +make --silent lint || log_error "make lint failed" >>> + >>> +# Go backwards in history until you find a remote branch from which current >>> +# branch was created. This will be used as base for git diff. >>> +PATCHCNT=0 >>> +BASEBRANCH="" >> If base branch means the remote branch which was used for cloning the cerrent >> git HEAD than you can simplify it a littl bit. >> git rev-parse --symbolic-full-name @{upstream} > >Nice! I did not know this trick. Unfortunately it does not work for my >use-case: I clone upstream repo, pick a branch to base review on, create my >own branch from the upstream one, and then apply patches on top of that. > >Imagine that I'm going to run the tool on branch 'review-tool'. Git history >tree looks like this: > >* d036f3e (HEAD -> review-tool) fixup! Add 'review' target for make. >* c6d75b0 Makefile: disable parallel build >| * 667d50b (dkupka-review) dns: Add --auto-reverse option. >| * c85f176 dns: Check if domain already exists. >| * a55cd76 dns: do not add (forward)zone if it is already resolvable. >| * 0b0c529 (dns-no-forwarders) DNS: Make --no-forwarders option default. >|/ >* d2e8470 Add 'review' target for make. It automates following tasks: >| * 101ffae (mbasti-review) Fix DNS tests: dns-resolve returns warning >| * 52393bf Add 'review' target for make. It automates following tasks: >|/ >* 848912a (origin/master, master) add missing /ipaplatform/constants.py to >.gitignore > >$ git rev-parse --symbolic-full-name @{upstream} >errors out like this: >fatal: no upstream configured for branch 'review-tool' > >I definitely agree that the code above is just an ugly hack, but I would like >to keep current behavior so I do not need to change my workflow. > good point. But work with arrays nad complicated logic in bash is not very nice :-) Almost impossible to find out what it does. If you do not plan to rewrite it to normal scripting language :-) than please use bash function for this as well. LS From mbasti at redhat.com Fri Dec 11 12:39:48 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 11 Dec 2015 13:39:48 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <566A8B22.5020101@redhat.com> References: <56687174.8000305@redhat.com> <20151210081357.GB21550@mail.corp.redhat.com> <56695291.5090705@redhat.com> <566990E3.2080903@redhat.com> <5669A3D3.7050608@redhat.com> <566A8B22.5020101@redhat.com> Message-ID: <566AC414.6030305@redhat.com> On 11.12.2015 09:36, Martin Kosek wrote: > On 12/10/2015 05:09 PM, Martin Basti wrote: >> >> >> On 10.12.2015 15:49, Tomas Babej wrote: >>> >>> On 12/10/2015 11:23 AM, Martin Basti wrote: >>>> >>>> On 10.12.2015 09:13, Lukas Slebodnik wrote: >>>>> On (09/12/15 19:22), Martin Basti wrote: >>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>> >>>>>> Patch attached. >>>>> >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 00:00:00 >>>>> 2001 >>>>>> From: Martin Basti >>>>>> Date: Wed, 9 Dec 2015 18:53:35 +0100 >>>>>> Subject: [PATCH] Fix version comparison >>>>>> >>>>>> Use RPM library to compare vendor versions of IPA for redhat >>>>>> platform >>>>>> >>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>> --- >>>>>> freeipa.spec.in | 2 ++ >>>>>> ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ >>>>>> 2 files changed, 21 insertions(+) >>>>>> >>>>>> diff --git a/freeipa.spec.in b/freeipa.spec.in >>>>>> index >>>>>> 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 >>>>>> >>>>>> >>>>>> 100644 >>>>>> --- a/freeipa.spec.in >>>>>> +++ b/freeipa.spec.in >>>>>> @@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} >>>>>> Requires: gzip >>>>>> Requires: python-gssapi >= 1.1.0 >>>>>> Requires: custodia >>>>>> +Requires: rpm-python >>>>>> +Requires: rpmdevtools >>>>> Could you explain why do you need the 2nd package? >>>>> It does not contains any python modules >>>>> and I cannot see usage of any binary in this patch >>>>> >>>>> LS >>>> Thanks for this catch, it is actually located in yum package, I rather >>>> copy stringToVersion function from there to IPA, to avoid >>>> dependency hell >>>> >>>> Updated patch attached. >>>> >>>> >>> Looking good. The __cmp__ function, however, is not available in Python >>> 3. As we will eventually support python3 in RHEL as well, maybe we >>> should make sure even platform-dependent parts are python3 compatible? >>> For the future's sake. >>> >>> Tomas >>> >> Thanks, >> >> python 3 compatible patch attached. > > I wonder why we have to add so much code here and reimplement RPM > version checking if it is already provided by rpmdevtools: > > ~~~ > $ /usr/bin/rpmdev-vercmp ipa-4.2.0-15.el7 4.2.0-15.el7_2.3; echo $? > WARNING: hyphen in release1: 4.2.0-15.el7 > > rpmdev-vercmp > rpmdev-vercmp > rpmdev-vercmp # with no arguments, prompt > > Exit status is 0 if the EVR's are equal, 11 if EVR1 is newer, and 12 > if EVR2 > is newer. Other exit statuses indicate problems. > > ipa-4.2.0-15.el7 < 4.2.0-15.el7_2.3 > 12 > ~~~ > > which could be directly called from __eq__ or __lt__, since we are in > platform specific code anyway already. > > Martin I do not like the idea of calling external tool, but I have no other objections except my personal feeling. Martin^2 From lkrispen at redhat.com Fri Dec 11 12:56:52 2015 From: lkrispen at redhat.com (Ludwig Krispenz) Date: Fri, 11 Dec 2015 13:56:52 +0100 Subject: [Freeipa-devel] [PATCH 0022] topology plugin prevents deletes but does not prevent moddn Message-ID: <566AC814.8010108@redhat.com> Ticket: https://fedorahosted.org/freeipa/ticket/5536 Patch attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-lkrispen-0022-prevent-moving-of-topology-entries-out-of-managed-sc.patch Type: text/x-patch Size: 4122 bytes Desc: not available URL: From pvoborni at redhat.com Fri Dec 11 12:56:48 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 11 Dec 2015 13:56:48 +0100 Subject: [Freeipa-devel] [PATCHES 516-517] spec file: put Python modules into standalone packages In-Reply-To: <56697563.4040406@redhat.com> References: <5665A403.9090604@redhat.com> <56688636.40803@redhat.com> <566954BA.6010806@redhat.com> <56697563.4040406@redhat.com> Message-ID: <566AC810.7040104@redhat.com> On 12/10/2015 01:51 PM, Jan Cholasta wrote: > On 10.12.2015 11:32, Jan Cholasta wrote: >> On 9.12.2015 20:51, Petr Vobornik wrote: >>> On 12/07/2015 04:21 PM, Jan Cholasta wrote: >>>> Hi, >>>> >>>> the attached patches partially fix >>>> . This is done to allow >>>> the addition of Python 3 packages, see >>>> . >>>> >>>> >>>> >>>> See commit messages for more information. >>>> >>>> In order to test: >>>> 1. make rpms >>>> 2. >>>> >>>> >>>> >>>> >>>> >>>> 3. Test with both dnf and yum-deprecated. >>>> >>>> Beware that when you run "yum-deprecated clean all", it does not remove >>>> cache for the on-disk repository created in step 2, you have to remove >>>> the /var/cache/yum/$basearch/$releasever/$reponame directory manually. >>>> >>>> Honza >>>> >>> >>> Shouldn't freeipa-server-dns and freeipa-server-trust-add depend on >>> freeipa-server? They do not in this patch. IMO they should. >> >> Fixed. >> >>> >>> following updates work (all on f23, update from 4.2.3): >>> dnf update >>> dnf update freeipa-* >>> yum-depracated update freeipa-* >>> >>> for both client or server with all packages. >>> >>> but when I tried to install only client "dnf install freeipa-client" and >>> then following failed: >>> dnf update freeipa-client >>> >>> The difference was: >>> Installing: >>> freeipa-client-common noarch >>> freeipa-common noarch >>> python2-ipaclient noarch >>> python2-ipalib x86_64 >>> Upgrading: >>> freeipa-client >>> >>> Works: >>> Installing: >>> freeipa-client-common noarch >>> freeipa-common noarch >>> freeipa-python-compat noarch >>> replacing freeipa-python.x86_64 4.2.3-1.1.fc23 >>> python2-ipaclient noarch >>> python2-ipalib x86_64 >>> Upgrading: >>> freeipa-client >>> >>> >>> not sure if it is a problem, otherwise the patch looks OK. >> >> Hmm, Fedora packaging guidelines are silent about this. >> >> When you run "dnf update freeipa-client freeipa-python" to force >> freeipa-python update it works fine. >> >> I removed Provides added Conflicts on freeipa-python which seem to have >> fixed it. > > Actually I think it would be better to keep the Provides and do > Conflicts < 4.2.91 - fixed. > > Also updated %alt_name dependencies to be in sync with %name > dependencies and added arch-specific python-ipalib Provides to > python2-ipalib. > > Updated patches attached. > ACK from me. What surprised me a bit is that I don't see any other responses here. If somebody didn't ready the mail, here is the new package list: freeipa-admintools freeipa-client freeipa-client-common freeipa-common freeipa-debuginfo freeipa-python-compat freeipa-server freeipa-server-common freeipa-server-dns freeipa-server-trust-ad python2-ipaclient python2-ipalib python2-ipaserver python2-ipatests -- Petr Vobornik From lslebodn at redhat.com Fri Dec 11 13:01:48 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Fri, 11 Dec 2015 14:01:48 +0100 Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <56695684.6050804@redhat.com> References: <565EAAAE.8020306@redhat.com> <565F0332.9000804@redhat.com> <20151202162519.GA19962@mail.corp.redhat.com> <56601AE8.2020100@redhat.com> <5660528E.40904@redhat.com> <149447113.23167965.1449156368210.JavaMail.zimbra@redhat.com> <566060E2.8030903@redhat.com> <5666DF65.1030309@redhat.com> <20151210080513.GA21550@mail.corp.redhat.com> <56695684.6050804@redhat.com> Message-ID: <20151211130148.GA21332@mail.corp.redhat.com> On (10/12/15 11:40), Tomas Babej wrote: >On 12/10/2015 09:05 AM, Lukas Slebodnik wrote: >> On (08/12/15 14:47), Tomas Babej wrote: >>> >>> >>> On 12/03/2015 04:33 PM, Tomas Babej wrote: >>>> >>>> >>>> On 12/03/2015 04:26 PM, Ale? Mare?ek wrote: >>>>> Hello, >>>>> >>>>> ACK for code >>>>> NACK for the placing "get_client_ip_with_hostmask" function to test_sudo.py (this function should be in some more general file) >>>>> >>>> >>>> What place would you propose? The task.py is not a good place, as this >>>> is not really a task. >>>> >>>> Nevertheless, I'd rather have it moved when an use case outside >>>> test_sudo.py actually arises. Right now it would lead to unnecessary >>>> cluttering. >>>> >>>> Tomas >>>> >>> >>> I re-discovered ipatests.test_integration.util (two years after I >>> created it :D) - which seemed ideal for this function. >>> >>> Updated patch attached. >>> >>> Tomas >> >>>From 33552d6078d75ee99f9ec19ae143df5a61ba84a4 Mon Sep 17 00:00:00 2001 >>> From: Tomas Babej >>> Date: Wed, 2 Dec 2015 15:25:49 +0100 >>> Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on >>> hostmask >>> >>> IPA sudo tests worked under the assumption that the clients >>> that are executing the sudo commands have their IPs assigned >>> within 255.255.255.0 hostmask. >>> >>> Removes this (invalid) assumption and adds a >>> dynamic detection of the hostmask of the IPA client. >>> >>> https://fedorahosted.org/freeipa/ticket/5501 >>> --- >>> ipatests/test_integration/test_sudo.py | 33 +++++++++++++++++++++++++++------ >>> ipatests/test_integration/util.py | 16 ++++++++++++++++ >>> 2 files changed, 43 insertions(+), 6 deletions(-) >>> >>> diff --git a/ipatests/test_integration/util.py b/ipatests/test_integration/util.py >>> index 1a1bb3fcc923c9f2721f0a4c1cb7a1ba2ccc2dd8..187f39e80e84af0eb4938fb19ac3d3c7c2280ed9 100644 >>> --- a/ipatests/test_integration/util.py >>> +++ b/ipatests/test_integration/util.py >>> @@ -58,3 +58,19 @@ def run_repeatedly(host, command, assert_zero_rc=True, test=None, >>> .format(cmd=' '.join(command), >>> times=timeout / time_step, >>> timeout=timeout)) >>> + >>> + >>> +def get_host_ip_with_hostmask(host): >>> + """ >>> + Detects the IP of the host including the hostmask. >>> + >>> + Returns None if the IP could not be detected. >>> + """ >>> + >>> + ip = host.ip >>> + result = host.run_command(['ip', 'addr']) >>> + full_ip_regex = r'(?P%s/\d{1,2}) ' % re.escape(ip) >>> + match = re.search(full_ip_regex, result.stdout_text) >> ./make-lint >> ************* Module ipatests.test_integration.util >> ipatests/test_integration/util.py:72: [E0602(undefined-variable), get_host_ip_with_hostmask] Undefined variable 're') >> ipatests/test_integration/util.py:73: [E0602(undefined-variable), get_host_ip_with_hostmask] Undefined variable 're') >> =============================================================================== >> Errors were found during the static code check. >> If you are certain that any of the reported errors are false positives, please >> mark them in the source code according to the pylint documentation. >> =============================================================================== >> Makefile:124: recipe for target 'lint' failed >> >> LS >> > >Nothing can break when moving a function, right? Missing import fixed. > >Tomas >From c176ff1ab9ea1c56dc0c5d44bc490d925fad1497 Mon Sep 17 00:00:00 2001 >From: Tomas Babej >Date: Wed, 2 Dec 2015 15:25:49 +0100 >Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on > hostmask > >IPA sudo tests worked under the assumption that the clients >that are executing the sudo commands have their IPs assigned >within 255.255.255.0 hostmask. > >Removes this (invalid) assumption and adds a >dynamic detection of the hostmask of the IPA client. > >https://fedorahosted.org/freeipa/ticket/5501 >--- > ipatests/test_integration/test_sudo.py | 32 ++++++++++++++++++++++++++------ > ipatests/test_integration/util.py | 17 +++++++++++++++++ > 2 files changed, 43 insertions(+), 6 deletions(-) > Thank you very much. ACK LS From tbabej at redhat.com Fri Dec 11 13:03:12 2015 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 11 Dec 2015 14:03:12 +0100 Subject: [Freeipa-devel] [PATCH 0395] replicainstall: Make sure the enrollment state is preserved In-Reply-To: <56697CA6.80500@redhat.com> References: <56697BA3.6030209@redhat.com> <56697CA6.80500@redhat.com> Message-ID: <566AC990.1070605@redhat.com> On 12/10/2015 02:22 PM, Tomas Babej wrote: > > > On 12/10/2015 02:18 PM, Tomas Babej wrote: >> Hi, >> >> During the promote_check phase, the subsequent checks after the machine >> is enrolled may cause the installation to abort, hence leaving it >> enrolled even though it might not have been prior to the execution of >> the ipa-replica-install command. >> >> Make sure that ipa-client-install --uninstall is called on the machine >> that has not been enrolled before in case of failure during the >> promote_check phase. >> >> https://fedorahosted.org/freeipa/ticket/5529 >> > > Self-NACK, installer is redundant for uninstall_client. > > Updated patch attached. > > Tomas > > > Attaching rebased version. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0395-3-replicainstall-Make-sure-the-enrollment-state-is-pre.patch Type: text/x-patch Size: 2648 bytes Desc: not available URL: From tbabej at redhat.com Thu Dec 10 10:40:04 2015 From: tbabej at redhat.com (Tomas Babej) Date: Thu, 10 Dec 2015 11:40:04 +0100 Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <20151210080513.GA21550@mail.corp.redhat.com> References: <20151130163256.GA7759@mail.corp.redhat.com> <565DD881.60401@redhat.com> <565EAAAE.8020306@redhat.com> <565F0332.9000804@redhat.com> <20151202162519.GA19962@mail.corp.redhat.com> <56601AE8.2020100@redhat.com> <5660528E.40904@redhat.com> <149447113.23167965.1449156368210.JavaMail.zimbra@redhat.com> <566060E2.8030903@redhat.com> <5666DF65.1030309@redhat.com> <20151210080513.GA21550@mail.corp.redhat.com> Message-ID: <56695684.6050804@redhat.com> On 12/10/2015 09:05 AM, Lukas Slebodnik wrote: > On (08/12/15 14:47), Tomas Babej wrote: >> >> >> On 12/03/2015 04:33 PM, Tomas Babej wrote: >>> >>> >>> On 12/03/2015 04:26 PM, Ale? Mare?ek wrote: >>>> Hello, >>>> >>>> ACK for code >>>> NACK for the placing "get_client_ip_with_hostmask" function to test_sudo.py (this function should be in some more general file) >>>> >>> >>> What place would you propose? The task.py is not a good place, as this >>> is not really a task. >>> >>> Nevertheless, I'd rather have it moved when an use case outside >>> test_sudo.py actually arises. Right now it would lead to unnecessary >>> cluttering. >>> >>> Tomas >>> >> >> I re-discovered ipatests.test_integration.util (two years after I >> created it :D) - which seemed ideal for this function. >> >> Updated patch attached. >> >> Tomas > >>From 33552d6078d75ee99f9ec19ae143df5a61ba84a4 Mon Sep 17 00:00:00 2001 >> From: Tomas Babej >> Date: Wed, 2 Dec 2015 15:25:49 +0100 >> Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on >> hostmask >> >> IPA sudo tests worked under the assumption that the clients >> that are executing the sudo commands have their IPs assigned >> within 255.255.255.0 hostmask. >> >> Removes this (invalid) assumption and adds a >> dynamic detection of the hostmask of the IPA client. >> >> https://fedorahosted.org/freeipa/ticket/5501 >> --- >> ipatests/test_integration/test_sudo.py | 33 +++++++++++++++++++++++++++------ >> ipatests/test_integration/util.py | 16 ++++++++++++++++ >> 2 files changed, 43 insertions(+), 6 deletions(-) >> >> diff --git a/ipatests/test_integration/util.py b/ipatests/test_integration/util.py >> index 1a1bb3fcc923c9f2721f0a4c1cb7a1ba2ccc2dd8..187f39e80e84af0eb4938fb19ac3d3c7c2280ed9 100644 >> --- a/ipatests/test_integration/util.py >> +++ b/ipatests/test_integration/util.py >> @@ -58,3 +58,19 @@ def run_repeatedly(host, command, assert_zero_rc=True, test=None, >> .format(cmd=' '.join(command), >> times=timeout / time_step, >> timeout=timeout)) >> + >> + >> +def get_host_ip_with_hostmask(host): >> + """ >> + Detects the IP of the host including the hostmask. >> + >> + Returns None if the IP could not be detected. >> + """ >> + >> + ip = host.ip >> + result = host.run_command(['ip', 'addr']) >> + full_ip_regex = r'(?P%s/\d{1,2}) ' % re.escape(ip) >> + match = re.search(full_ip_regex, result.stdout_text) > ./make-lint > ************* Module ipatests.test_integration.util > ipatests/test_integration/util.py:72: [E0602(undefined-variable), get_host_ip_with_hostmask] Undefined variable 're') > ipatests/test_integration/util.py:73: [E0602(undefined-variable), get_host_ip_with_hostmask] Undefined variable 're') > =============================================================================== > Errors were found during the static code check. > If you are certain that any of the reported errors are false positives, please > mark them in the source code according to the pylint documentation. > =============================================================================== > Makefile:124: recipe for target 'lint' failed > > LS > Nothing can break when moving a function, right? Missing import fixed. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0388-7-tests-Add-hostmask-detection-for-sudo-rules-validati.patch Type: text/x-patch Size: 4663 bytes Desc: not available URL: From tbabej at redhat.com Fri Dec 11 13:27:22 2015 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 11 Dec 2015 14:27:22 +0100 Subject: [Freeipa-devel] [PATCH 0388] tests: Add hostmask detection for sudo rules validating In-Reply-To: <20151211130148.GA21332@mail.corp.redhat.com> References: <565EAAAE.8020306@redhat.com> <565F0332.9000804@redhat.com> <20151202162519.GA19962@mail.corp.redhat.com> <56601AE8.2020100@redhat.com> <5660528E.40904@redhat.com> <149447113.23167965.1449156368210.JavaMail.zimbra@redhat.com> <566060E2.8030903@redhat.com> <5666DF65.1030309@redhat.com> <20151210080513.GA21550@mail.corp.redhat.com> <56695684.6050804@redhat.com> <20151211130148.GA21332@mail.corp.redhat.com> Message-ID: <566ACF3A.6000409@redhat.com> On 12/11/2015 02:01 PM, Lukas Slebodnik wrote: > On (10/12/15 11:40), Tomas Babej wrote: >> On 12/10/2015 09:05 AM, Lukas Slebodnik wrote: >>> On (08/12/15 14:47), Tomas Babej wrote: >>>> >>>> >>>> On 12/03/2015 04:33 PM, Tomas Babej wrote: >>>>> >>>>> >>>>> On 12/03/2015 04:26 PM, Ale? Mare?ek wrote: >>>>>> Hello, >>>>>> >>>>>> ACK for code >>>>>> NACK for the placing "get_client_ip_with_hostmask" function to test_sudo.py (this function should be in some more general file) >>>>>> >>>>> >>>>> What place would you propose? The task.py is not a good place, as this >>>>> is not really a task. >>>>> >>>>> Nevertheless, I'd rather have it moved when an use case outside >>>>> test_sudo.py actually arises. Right now it would lead to unnecessary >>>>> cluttering. >>>>> >>>>> Tomas >>>>> >>>> >>>> I re-discovered ipatests.test_integration.util (two years after I >>>> created it :D) - which seemed ideal for this function. >>>> >>>> Updated patch attached. >>>> >>>> Tomas >>> >>> >From 33552d6078d75ee99f9ec19ae143df5a61ba84a4 Mon Sep 17 00:00:00 2001 >>>> From: Tomas Babej >>>> Date: Wed, 2 Dec 2015 15:25:49 +0100 >>>> Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on >>>> hostmask >>>> >>>> IPA sudo tests worked under the assumption that the clients >>>> that are executing the sudo commands have their IPs assigned >>>> within 255.255.255.0 hostmask. >>>> >>>> Removes this (invalid) assumption and adds a >>>> dynamic detection of the hostmask of the IPA client. >>>> >>>> https://fedorahosted.org/freeipa/ticket/5501 >>>> --- >>>> ipatests/test_integration/test_sudo.py | 33 +++++++++++++++++++++++++++------ >>>> ipatests/test_integration/util.py | 16 ++++++++++++++++ >>>> 2 files changed, 43 insertions(+), 6 deletions(-) >>>> >>>> diff --git a/ipatests/test_integration/util.py b/ipatests/test_integration/util.py >>>> index 1a1bb3fcc923c9f2721f0a4c1cb7a1ba2ccc2dd8..187f39e80e84af0eb4938fb19ac3d3c7c2280ed9 100644 >>>> --- a/ipatests/test_integration/util.py >>>> +++ b/ipatests/test_integration/util.py >>>> @@ -58,3 +58,19 @@ def run_repeatedly(host, command, assert_zero_rc=True, test=None, >>>> .format(cmd=' '.join(command), >>>> times=timeout / time_step, >>>> timeout=timeout)) >>>> + >>>> + >>>> +def get_host_ip_with_hostmask(host): >>>> + """ >>>> + Detects the IP of the host including the hostmask. >>>> + >>>> + Returns None if the IP could not be detected. >>>> + """ >>>> + >>>> + ip = host.ip >>>> + result = host.run_command(['ip', 'addr']) >>>> + full_ip_regex = r'(?P%s/\d{1,2}) ' % re.escape(ip) >>>> + match = re.search(full_ip_regex, result.stdout_text) >>> ./make-lint >>> ************* Module ipatests.test_integration.util >>> ipatests/test_integration/util.py:72: [E0602(undefined-variable), get_host_ip_with_hostmask] Undefined variable 're') >>> ipatests/test_integration/util.py:73: [E0602(undefined-variable), get_host_ip_with_hostmask] Undefined variable 're') >>> =============================================================================== >>> Errors were found during the static code check. >>> If you are certain that any of the reported errors are false positives, please >>> mark them in the source code according to the pylint documentation. >>> =============================================================================== >>> Makefile:124: recipe for target 'lint' failed >>> >>> LS >>> >> >> Nothing can break when moving a function, right? Missing import fixed. >> >> Tomas > >>From c176ff1ab9ea1c56dc0c5d44bc490d925fad1497 Mon Sep 17 00:00:00 2001 >> From: Tomas Babej >> Date: Wed, 2 Dec 2015 15:25:49 +0100 >> Subject: [PATCH] tests: Add hostmask detection for sudo rules validating on >> hostmask >> >> IPA sudo tests worked under the assumption that the clients >> that are executing the sudo commands have their IPs assigned >> within 255.255.255.0 hostmask. >> >> Removes this (invalid) assumption and adds a >> dynamic detection of the hostmask of the IPA client. >> >> https://fedorahosted.org/freeipa/ticket/5501 >> --- >> ipatests/test_integration/test_sudo.py | 32 ++++++++++++++++++++++++++------ >> ipatests/test_integration/util.py | 17 +++++++++++++++++ >> 2 files changed, 43 insertions(+), 6 deletions(-) >> > Thank you very much. > > ACK > > LS > Pushed to: master: a02f83ff9c6a1920cedebee69dc6857c3521f161 ipa-4-2: f676b122848f7d843e200484ac06568d71f2f4ef From mbasti at redhat.com Fri Dec 11 13:45:23 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 11 Dec 2015 14:45:23 +0100 Subject: [Freeipa-devel] [PATCH 0377] CI: fix test_vault installation on domain level 1 Message-ID: <566AD373.3060505@redhat.com> Patch attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0377-CI-fix-ipa-kra-install-on-domain-level-1.patch Type: text/x-patch Size: 1120 bytes Desc: not available URL: From jcholast at redhat.com Fri Dec 11 14:00:38 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 11 Dec 2015 15:00:38 +0100 Subject: [Freeipa-devel] [PATCHES 529-530] ca install: use host credentials in domain level 1 In-Reply-To: <56693CF6.8030104@redhat.com> References: <56693CF6.8030104@redhat.com> Message-ID: <566AD706.90800@redhat.com> On 10.12.2015 09:51, Jan Cholasta wrote: > Hi, > > the attached patches fix . > > My patches 523-525 are required for this: > . > > > Honza Rebased patches attached. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-529.1-aci-merge-domain-and-CA-suffix-replication-agreement.patch Type: text/x-patch Size: 5555 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-530.1-ca-install-use-host-credentials-in-domain-level-1.patch Type: text/x-patch Size: 4418 bytes Desc: not available URL: From pspacek at redhat.com Fri Dec 11 14:00:31 2015 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 11 Dec 2015 15:00:31 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <566AB505.3060508@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> <5665843C.2090507@redhat.com> <56658C89.8050103@redhat.com> <56668D30.8060308@redhat.com> <5666BCBC.9040005@redhat.com> <5666E822.90103@redhat.com> <56682093.3010505@redhat.com> <56686AFE.4000401@redhat.com> <5669A8D8.5030405@redhat.com> <5669B20E.8000803@redhat.com> <566AB505.3060508@redhat.com> Message-ID: <566AD6FF.7020409@redhat.com> On 11.12.2015 12:35, David Kupka wrote: > On 10/12/15 18:10, Petr Spacek wrote: >> On 10.12.2015 17:31, David Kupka wrote: >>> On 09/12/15 18:55, Petr Spacek wrote: >>>> On 9.12.2015 13:37, David Kupka wrote: >>>>> On 08/12/15 15:24, Petr Spacek wrote: >>>>>> On 8.12.2015 12:19, David Kupka wrote: >>>>>>> On 08/12/15 08:56, Petr Spacek wrote: >>>>>>>> On 7.12.2015 14:41, David Kupka wrote: >>>>>>>>> +def is_host_resolvable(fqdn): >>>>>>>>> + if not isinstance(fqdn, DNSName): >>>>>>>>> + fqdn = DNSName(fqdn) >>>>>>>>> + for rdtype in (rdatatype.A, rdatatype.AAAA): >>>>>>>>> + try: >>>>>>>>> + resolver.query(fqdn.make_absolute(), rdtype) >>>>>>>>> + except DNSException: >>>>>>>>> + continue >>>>>>>>> + else: >>>>>>>>> + return True >>>>>>>>> + >>>>>>>>> + return False >>>>>>>>> >>>>>>>> >>>>>>>> NACK, you are re-introducing duplicate function which was removed in >>>>>>>> 498471e4aed1367b72cd74d15811d0584a6ee268. >>>>>>>> >>>>>>>> Please amend the patch ASAP to use new verify_host_resolvable() function >>>>>>>> so I >>>>>>>> can test it and get it into 4.3. >>>>>>>> >>>>>>> Updated patches attached. >>>>>> >>>>>> Hmm, my review script screams: >>>>>> >>>>>> Detected base branch: origin/master >>>>>> Checks will be made against following base commit: 848912a add missing >>>>>> /ipaplatform/constants.py to .gitignore >>>>>> Writing API to API.txt >>>>>> ./ipalib/plugins/dns.py:2127:9: E124 closing bracket does not match visual >>>>>> indentation >>>>>> ./ipalib/plugins/dns.py:2711:13: E128 continuation line under-indented for >>>>>> visual indent >>>>>> ./ipalib/plugins/dns.py:2713:9: E124 closing bracket does not match visual >>>>>> indentation >>>>>> ./ipalib/plugins/dns.py:2716:13: E128 continuation line under-indented for >>>>>> visual indent >>>>>> ./ipapython/ipautil.py:928:1: E302 expected 2 blank lines, found 1 >>>>>> ./ipapython/ipautil.py:948:17: E128 continuation line under-indented for >>>>>> visual indent >>>>>> ./ipapython/ipautil.py:956:1: E302 expected 2 blank lines, found 1 >>>>>> ./ipapython/ipautil.py:997:80: E501 line too long (83 > 79 characters) >>>>>> ./ipapython/ipautil.py:998:80: E501 line too long (83 > 79 characters) >>>>>> ./ipaserver/install/bindinstance.py:49:9: E128 continuation line >>>>>> under-indented for visual indent >>>>>> ./ipaserver/install/bindinstance.py:292:80: E501 line too long (80 > 79 >>>>>> characters) >>>>>> ./ipaserver/install/bindinstance.py:438:19: E128 continuation line >>>>>> under-indented for visual indent >>>>>> ./ipaserver/install/server/common.py:271:63: E261 at least two spaces >>>>>> before >>>>>> inline comment >>>>>> ./ipaserver/install/server/common.py:271:80: E501 line too long (90 > 79 >>>>>> characters) >>>>>> ERROR: PEP8 --diff failed, continuing ... >>>>>> ERROR: API.txt was changed without a change in VERSION, continuing ... >>>>>> Summary of detected errors: >>>>>> ERROR: PEP8 --diff failed >>>>>> ERROR: API.txt was changed without a change in VERSION >>>>>> >>>>>> Please fix it :-) >>>>>> >>>>>> Make make this more pleasant for you, you can use (and review) my attached >>>>>> patch. It adds 'review' target to Makefile, it will do the same checks as >>>>>> I do. >>>>>> >>>>> >>>>> Unfortunately your tool does not work for me (output w/ -o xtrace attached). >>>>> Anyway I have incremented API version and fixed most of the pep8 errors >>>>> except >>>>> for E124 twice in ipalib/plugins/dns.py as it seems to be convention in the >>>>> file and E501 twice in ipapython/ipautil.py because IMO breaking string >>>>> constant into multiple lines does not help readability. >>>>> >>>>> Updated patches also attached. >>>> >>>> We are almost there, but NACK for now. >>>> >>>> 1) Following attempt to install IPA explodes. Please note that >>>> dom-245.idm.lab.eng.brq.redhat.com DNS domain is delegated to the IPA server >>>> before installation is started, so it gives 'timeout' or possibly SERVFAIL. >>>> >>>> # ipa-server-install --ds-password=root4lab --admin-password=root4lab >>>> --setup-dns --forwarder=10.38.5.26 --no-reverse --allow-zone-overlap >>>> --domain=dom-245.idm.lab.eng.brq.redhat.com >>>> --realm=DOM-245.IDM.LAB.ENG.BRQ.REDHAT.COM >>>> [...] >>>> >>>> Configuring DNS (named) >>>> [1/11]: generating rndc key file >>>> [2/11]: adding DNS container >>>> [3/11]: setting up our zone >>>> [error] InvocationError: DNS check for domain >>>> dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation timed out after >>>> 30.000453949 seconds. Please make sure that the domain is properly delegated >>>> to this IPA server. >>>> ipa.ipapython.install.cli.install_tool(Server): ERROR DNS check for domain >>>> dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation timed out after >>>> 30.000453949 seconds. Please make sure that the domain is properly delegated >>>> to this IPA server. >>>> ipa.ipapython.install.cli.install_tool(Server): ERROR The >>>> ipa-server-install command failed. See /var/log/ipaserver-install.log for >>>> more >>>> information >>>> >>>> 2015-12-09T17:15:37Z DEBUG File >>>> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in >>>> execute >>>> return_value = self.run() >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line >>>> 318, >>>> in run >>>> cfgr.run() >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 310, >>>> in run >>>> self.execute() >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 332, >>>> in execute >>>> for nothing in self._executor(): >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 372, >>>> in __runner >>>> self._handle_exception(exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 394, >>>> in _handle_exception >>>> six.reraise(*exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 362, >>>> in __runner >>>> step() >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 359, >>>> in >>>> step = lambda: next(self.__gen) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>> line 81, >>>> in run_generator_with_yield_from >>>> six.reraise(*exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>> line 59, >>>> in run_generator_with_yield_from >>>> value = gen.send(prev_value) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 571, >>>> in _configure >>>> next(executor) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 372, >>>> in __runner >>>> self._handle_exception(exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 449, >>>> in _handle_exception >>>> self.__parent._handle_exception(exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 394, >>>> in _handle_exception >>>> six.reraise(*exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 446, >>>> in _handle_exception >>>> super(ComponentBase, self)._handle_exception(exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 394, >>>> in _handle_exception >>>> six.reraise(*exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 362, >>>> in __runner >>>> step() >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>> line 359, >>>> in >>>> step = lambda: next(self.__gen) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>> line 81, >>>> in run_generator_with_yield_from >>>> six.reraise(*exc_info) >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>> line 59, >>>> in run_generator_with_yield_from >>>> value = gen.send(prev_value) >>>> >>>> File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line >>>> 63, in _install >>>> for nothing in self._installer(self.parent): >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>> line 1471, in main >>>> install(self) >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>> line 265, in decorated >>>> func(installer) >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>> line 958, in install >>>> dns.install(False, False, options) >>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/dns.py", line >>>> 322, >>>> in install >>>> bind.create_instance() >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>> line 680, in create_instance >>>> self.start_creation() >>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", >>>> line >>>> 447, in start_creation >>>> run_step(full_msg, method) >>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", >>>> line >>>> 437, in run_step >>>> method() >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>> line 805, in __setup_zone >>>> ns_hostname=self.api.env.host, force=True, api=self.api) >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>> line 331, in add_zone >>>> force=force) >>>> File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 447, in >>>> __call__ >>>> ret = self.run(*args, **options) >>>> File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 764, >>>> in run >>>> return self.execute(*args, **options) >>>> File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>> 2781, in >>>> execute >>>> result = super(dnszone_add, self).execute(*keys, **options) >>>> File "/usr/lib/python2.7/site-packages/ipalib/plugins/baseldap.py", line >>>> 1233, in execute >>>> *keys, **options) >>>> File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>> 2747, in >>>> pre_callback >>>> ldap, dn, entry_attrs, attrs_list, *keys, **options) >>>> File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>> 2153, in >>>> pre_callback >>>> raise errors.InvocationError(e.message) >>>> >>>> >>>> 2) Please print 'Checking DNS domain , please wait ...' when validating >>>> domain parameter in installer. The timeout is 30 seconds and users may be >>>> nervous when the installed blocks for 30 seconds without a visible reason. >>>> >>>> Precedent for this is in check_forwarders() in >>>> ipaserver/install/bindinstance.py . Encapsulating check_zone_overlap() in >>>> auxiliary method may be an option. >>>> >>>> >>>> 3) Timeout is a fatal error instead of warning. We have been discussing this >>>> and it should really be just a warning. >>>> >>>> >>>> 4) Nitpicks are attached in .patch file. >>>> >>>> >>>> 5) ipa dnszone-add checks work as expected, good job! >>>> >>>> >>>> Thank you for patience! >>>> >>> Updated patches attached. Added patch introducing --auto-reverse option that >>> should generate needed reverse zones automatically. >>> >> NACK: >> >> $ ipa-server-install --setup-dns >> >> Do you want to configure the reverse zone? [yes]: >> ipa.ipapython.install.cli.install_tool(Server): ERROR 'CheckedIPAddress' >> object has no attribute 'split' >> > Updated patches attached. > NACK 1) It breaks replica installation if replica is second+ DNS server. # ipa-replica-install --setup-dns Password for admin at ABC.IDM.LAB.ENG.BRQ.REDHAT.COM: Checking DNS domain abc.idm.lab.eng.brq.redhat.com., please wait ... Your system may be partly configured. Run /usr/sbin/ipa-server-install --uninstall to clean up. ipa.ipapython.install.cli.install_tool(Replica): ERROR DNS zone abc.idm.lab.eng.brq.redhat.com. already exists in DNS and is handled by server(s): vm-058-155.abc.idm.lab.eng.brq.redhat.com. Maybe the validator should have condition like "if not replica or not dns_is_configured()" or so ... 2) Check for automatic empty zones does not work: # ipa dnsforwardzone-add 10.in-addr.arpa. --forwarder=10.34.78.1 Server will check DNS forwarder(s). This may take some time, please wait ... ipa: ERROR: DNS zone 10.in-addr.arpa. already exists in DNS and is handled by server(s): 10.IN-ADDR.ARPA. Here you have to compare names as actual DNS names and not as strings. Have a nice weekend! -- Petr^2 Spacek From ofayans at redhat.com Fri Dec 11 14:08:17 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Fri, 11 Dec 2015 15:08:17 +0100 Subject: [Freeipa-devel] [PATCH 0377] CI: fix test_vault installation on domain level 1 In-Reply-To: <566AD373.3060505@redhat.com> References: <566AD373.3060505@redhat.com> Message-ID: <566AD8D1.4030503@redhat.com> ACK On 12/11/2015 02:45 PM, Martin Basti wrote: > Patch attached. -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From mbasti at redhat.com Fri Dec 11 14:11:50 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 11 Dec 2015 15:11:50 +0100 Subject: [Freeipa-devel] [PATCH 0377] CI: fix test_vault installation on domain level 1 In-Reply-To: <566AD8D1.4030503@redhat.com> References: <566AD373.3060505@redhat.com> <566AD8D1.4030503@redhat.com> Message-ID: <566AD9A6.8090900@redhat.com> On 11.12.2015 15:08, Oleg Fayans wrote: > ACK > > On 12/11/2015 02:45 PM, Martin Basti wrote: >> Patch attached. Pushed to master: 1e0f1f5197ecf8f20ae4cba29af2a2162096c4f6 From jcholast at redhat.com Fri Dec 11 14:19:23 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 11 Dec 2015 15:19:23 +0100 Subject: [Freeipa-devel] [PATCHES 516-517] spec file: put Python modules into standalone packages In-Reply-To: <566AC810.7040104@redhat.com> References: <5665A403.9090604@redhat.com> <56688636.40803@redhat.com> <566954BA.6010806@redhat.com> <56697563.4040406@redhat.com> <566AC810.7040104@redhat.com> Message-ID: <566ADB6B.7050604@redhat.com> On 11.12.2015 13:56, Petr Vobornik wrote: > On 12/10/2015 01:51 PM, Jan Cholasta wrote: >> On 10.12.2015 11:32, Jan Cholasta wrote: >>> On 9.12.2015 20:51, Petr Vobornik wrote: >>>> On 12/07/2015 04:21 PM, Jan Cholasta wrote: >>>>> Hi, >>>>> >>>>> the attached patches partially fix >>>>> . This is done to allow >>>>> the addition of Python 3 packages, see >>>>> . >>>>> >>>>> >>>>> >>>>> >>>>> See commit messages for more information. >>>>> >>>>> In order to test: >>>>> 1. make rpms >>>>> 2. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> 3. Test with both dnf and yum-deprecated. >>>>> >>>>> Beware that when you run "yum-deprecated clean all", it does not >>>>> remove >>>>> cache for the on-disk repository created in step 2, you have to remove >>>>> the /var/cache/yum/$basearch/$releasever/$reponame directory manually. >>>>> >>>>> Honza >>>>> >>>> >>>> Shouldn't freeipa-server-dns and freeipa-server-trust-add depend on >>>> freeipa-server? They do not in this patch. IMO they should. >>> >>> Fixed. >>> >>>> >>>> following updates work (all on f23, update from 4.2.3): >>>> dnf update >>>> dnf update freeipa-* >>>> yum-depracated update freeipa-* >>>> >>>> for both client or server with all packages. >>>> >>>> but when I tried to install only client "dnf install freeipa-client" >>>> and >>>> then following failed: >>>> dnf update freeipa-client >>>> >>>> The difference was: >>>> Installing: >>>> freeipa-client-common noarch >>>> freeipa-common noarch >>>> python2-ipaclient noarch >>>> python2-ipalib x86_64 >>>> Upgrading: >>>> freeipa-client >>>> >>>> Works: >>>> Installing: >>>> freeipa-client-common noarch >>>> freeipa-common noarch >>>> freeipa-python-compat noarch >>>> replacing freeipa-python.x86_64 4.2.3-1.1.fc23 >>>> python2-ipaclient noarch >>>> python2-ipalib x86_64 >>>> Upgrading: >>>> freeipa-client >>>> >>>> >>>> not sure if it is a problem, otherwise the patch looks OK. >>> >>> Hmm, Fedora packaging guidelines are silent about this. >>> >>> When you run "dnf update freeipa-client freeipa-python" to force >>> freeipa-python update it works fine. >>> >>> I removed Provides added Conflicts on freeipa-python which seem to have >>> fixed it. >> >> Actually I think it would be better to keep the Provides and do >> Conflicts < 4.2.91 - fixed. >> >> Also updated %alt_name dependencies to be in sync with %name >> dependencies and added arch-specific python-ipalib Provides to >> python2-ipalib. >> >> Updated patches attached. >> > > ACK from me. > > What surprised me a bit is that I don't see any other responses here. > > If somebody didn't ready the mail, here is the new package list: > > freeipa-admintools > freeipa-client > freeipa-client-common > freeipa-common > freeipa-debuginfo > freeipa-python-compat > freeipa-server > freeipa-server-common > freeipa-server-dns > freeipa-server-trust-ad > python2-ipaclient > python2-ipalib > python2-ipaserver > python2-ipatests Thanks for the review. Pushed to master: e9baafb08f04f91168469d3d8f36fc792d7d62dc -- Jan Cholasta From jcholast at redhat.com Fri Dec 11 14:40:57 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 11 Dec 2015 15:40:57 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <566A755D.6000701@redhat.com> References: <5667EE2C.2050100@redhat.com> <56683238.3050700@redhat.com> <56684B03.2020905@redhat.com> <56693C49.6030508@redhat.com> <566992B1.6030702@redhat.com> <566A6875.1070300@redhat.com> <566A755D.6000701@redhat.com> Message-ID: <566AE079.2030501@redhat.com> On 11.12.2015 08:03, Jan Cholasta wrote: > On 11.12.2015 07:08, Jan Cholasta wrote: >> On 10.12.2015 15:56, Martin Babinsky wrote: >>> On 12/10/2015 09:48 AM, Jan Cholasta wrote: >>>> On 9.12.2015 16:38, Jan Cholasta wrote: >>>>> On 9.12.2015 14:52, Jan Cholasta wrote: >>>>>> On 9.12.2015 10:02, Jan Cholasta wrote: >>>>>>> Hi, >>>>>>> >>>>>>> the attached patches fix >>>>>>> . >>>>>> >>>>>> Note that this needs selinux-policy fix to work, so put SELinux into >>>>>> permissive mode for testing: >>>>>> . >>>>> >>>>> Updated patches attached. >>>> >>>> I screwed up a change in patch 524 and accidentally included a chunk of >>>> code in patch 525 that doesn't belong in it. >>>> >>>> Updated patches attached. >>>> >>>> >>>> >>> >>> Patches work as expected and I was not able to find any functional >>> problem. >>> >>> I have a question about the naming of the oddjob helper script: the one >>> related to trusts is named 'com.redhat.idm.trust-fetch-domains', and the >>> conncheck runner is named 'org.freeipa.server.conncheck'. I don't want >>> to start another bikeshedding conversation but shouldn't we named them >>> in a consistent fashion (either rename the first one in separate patch >>> or rename the new helper to com.redhat.idm.server.conncheck)? >>> >>> I understand that as an upstream, we should go with the 'org.freeipa.*' >>> convention, but having two helpers with different prefixes makes me sad. >> >> If you look at the larger picture, org.freeipa is the consistent name. >> It makes me sad as well, but mistakes should be corrected. This is >> similar to how we use PEP8 in new code, but do not fix it in old code >> just for the sake of fixing it. >> >>> >>> That is a nitpick though, it does not affect the overall functionality >>> of the patches so ACK. >> >> Thanks for the review. The current patch 523 breaks the trusts oddjob >> with SELinux in enforcing mode, I will send an update which corrects >> that, until bug 1289930 is fixed. > > Updated patches attached. Rebased on top of current master. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-523.4-build-put-oddjob-scripts-into-separate-directory.patch Type: text/x-patch Size: 1625 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-524.4-replica-install-add-remote-connection-check-over-API.patch Type: text/x-patch Size: 28609 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-525.4-replica-promotion-use-host-credentials-for-connectio.patch Type: text/x-patch Size: 2344 bytes Desc: not available URL: From jcholast at redhat.com Fri Dec 11 14:50:27 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 11 Dec 2015 15:50:27 +0100 Subject: [Freeipa-devel] [PATCH 0069] Add 'review' target for make In-Reply-To: <5669B09B.9000001@redhat.com> References: <56683B03.40703@redhat.com> <5669B09B.9000001@redhat.com> Message-ID: <566AE2B3.3010201@redhat.com> Hi, On 10.12.2015 18:04, Petr Spacek wrote: > On 9.12.2015 15:30, Petr Spacek wrote: >> Hello, >> >> this patch automates some of sanity checks proposed by Petr Vobornik. >> >> 'make review' should be used in root of clean Git tree which has patches under >> review applied. >> >> Magic in review.sh attempts to detect nearest remote branch which can be used >> as diff base for review. Please see review.sh for further details. > > And here is the patch! :-) Nice, but I would rather see this in ipatool (). Or is there any obvious benefit in having this in freeipa itself that I'm missing? Honza -- Jan Cholasta From pvoborni at redhat.com Fri Dec 11 16:21:55 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 11 Dec 2015 17:21:55 +0100 Subject: [Freeipa-devel] [PATCH 0391] replicainstall: Add check for domain if server is specified In-Reply-To: <1449579620.17418.28.camel@redhat.com> References: <5661D8A0.7030303@redhat.com> <5661E661.5090601@redhat.com> <5661E810.3060503@redhat.com> <566586F4.30201@redhat.com> <5665A939.1060807@redhat.com> <56667F4C.9070305@redhat.com> <56668560.1040508@redhat.com> <5666868D.6010702@redhat.com> <5666CE45.4080301@redhat.com> <1449579620.17418.28.camel@redhat.com> Message-ID: <566AF823.9070906@redhat.com> On 12/08/2015 02:00 PM, Simo Sorce wrote: > On Tue, 2015-12-08 at 13:34 +0100, Martin Kosek wrote: >> On 12/08/2015 08:28 AM, Jan Cholasta wrote: >>> On 8.12.2015 08:23, Martin Kosek wrote: >>>> On 12/08/2015 07:57 AM, Jan Cholasta wrote: >>>>> On 7.12.2015 16:43, Martin Kosek wrote: >>>>>> On 12/07/2015 02:17 PM, Tomas Babej wrote: >>>>>>> >>>>>>> >>>>>>> On 12/04/2015 08:22 PM, Rob Crittenden wrote: >>>>>>>> Martin Kosek wrote: >>>>>>>>> On 12/04/2015 07:17 PM, Tomas Babej wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Avoids failing in the later stages during the ipa-client-install >>>>>>>>>> command. >>>>>>>>>> >>>>>>>>>> Tomas >>>>>>>>> >>>>>>>>> Is this change needed? Wouldn't it be better to update >>>>>>>>> ipa-client-install or ipa-replica-install to not require the --domain >>>>>>>>> option? I would hope that --domain can be figured out during >>>>>>>>> installation and not passed to ipa-replica-install manually by the admin. >>>>>>>>> >>>>>>>>> I just think that calling >>>>>>>>> # ipa-replica-install --server=master.example.com >>>>>>>>> is better than >>>>>>>>> # ipa-replica-install --server=master.example.com --domain example.com >>>>>>>>> if possible. >>>>>>>> >>>>>>>> IIRC this is for service discovery when using a specific server and not >>>>>>>> LDAP. This is the domain used to search for the kerberos realm, for >>>>>>>> example. >>>>>>>> >>>>>>>> That isn't to say this isn't discoverable but it would require another >>>>>>>> function in discovery to query what the IPA domain is from the given >>>>>>>> master but it gets tricky if anonymous search is disabled, for example. >>>>>>>> >>>>>>>> rob >>>>>>>> >>>>>>> >>>>>>> Needed or not, this is the behaviour that ipa-client-install has now. >>>>>>> Adding a domain detection method would be a RFE for ipa-client-install >>>>>>> (and imho not something we should be adding at this point). >>>>>>> >>>>>>> This patch only focuses on making the ipa-replica-install work more >>>>>>> smoothly. >>>>>> >>>>>> I am just thinking that client promotion (ipa-replica-install) and >>>>>> ipa-client-install are a bit different use cases. While ipa-client-install >>>>>> should be typically run in auto-discovery and you thus do not use --server >>>>>> option much, while with ipa-replica-install, you want to make sure you have >>>>>> the >>>>>> expected topology and should use --server all the time without gambling on it. >>>>>> >>>>>> But I do not think it has to be there since 4.3 GA, can you please file a >>>>>> ticket for this gap? >>>>> >>>>> I would rather do it now, because the change from optional to mandatory is >>>>> backward incompatible. (We don't want to break users' scripts, right?) >>>> >>>> I think it is the other way around - with the change I was suggesting >>>> (autodetecting --domain option instead of always requesting it, as in Tomas' >>>> patch which we can merge if my proposal is not doable for 4.3 GA). >>>> >>> >>> "with ipa-replica-install, you want to make sure you have the expected topology >>> and should use --server all the time" sounds like you want to make --server >>> mandatory for ipa-replica-install, which should be done either before 4.3 GA or >>> never. >> >> Ah, no, this is not what I meant. I was only discussing the --domain option in >> this mail the the typical use cases for --server option in ipa-client-install >> and ipa-replica-install. >> >> If we can trust ipa-replica-install to do a good job in picking a server to >> replica from, the --server can stay optional. Although I am on fence here, >> being more explicit when creating topology may be helpful. CCing Simo, in case >> he has some opinions on this. > > Leave it optional, our first order of business is making things simple, > then adding optional knobs to let the admin with knowledge to tweak > things. > > Simo. > ACK for original patch Pushed to master: c3c8651ac1bac794e32b3c01f7e4f6b487dcef08 -- Petr Vobornik From mbasti at redhat.com Fri Dec 11 16:24:08 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 11 Dec 2015 17:24:08 +0100 Subject: [Freeipa-devel] [PATCHES 529-530] ca install: use host credentials in domain level 1 In-Reply-To: <566AD706.90800@redhat.com> References: <56693CF6.8030104@redhat.com> <566AD706.90800@redhat.com> Message-ID: <566AF8A8.4050107@redhat.com> On 11.12.2015 15:00, Jan Cholasta wrote: > On 10.12.2015 09:51, Jan Cholasta wrote: >> Hi, >> >> the attached patches fix . >> >> My patches 523-525 are required for this: >> . >> >> >> >> Honza > > Rebased patches attached. > Patch works for me, but can you provide explanations (and update commit message) why the ACI change is needed: * why it is moved three ACIs from 'cn="$SUFFIX",cn=mapping tree,cn=config' to 'cn=mapping tree,cn=config' * why you removed completely 'dn: cn=o\3Dipaca,cn=mapping tree,cn=config' Martin From ofayans at redhat.com Fri Dec 11 16:28:10 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Fri, 11 Dec 2015 17:28:10 +0100 Subject: [Freeipa-devel] [PATCH 0018] Fixed install_ca and install_kra failures at domain level 0 Message-ID: <566AF99A.4060604@redhat.com> -- Oleg Fayans Quality Engineer FreeIPA team RedHat. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0018-Fixed-install_ca-and-install_kra-domainlevel-0.patch Type: text/x-patch Size: 2589 bytes Desc: not available URL: From mbasti at redhat.com Fri Dec 11 16:37:53 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 11 Dec 2015 17:37:53 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <566AE079.2030501@redhat.com> References: <5667EE2C.2050100@redhat.com> <56683238.3050700@redhat.com> <56684B03.2020905@redhat.com> <56693C49.6030508@redhat.com> <566992B1.6030702@redhat.com> <566A6875.1070300@redhat.com> <566A755D.6000701@redhat.com> <566AE079.2030501@redhat.com> Message-ID: <566AFBE1.2070406@redhat.com> On 11.12.2015 15:40, Jan Cholasta wrote: > On 11.12.2015 08:03, Jan Cholasta wrote: >> On 11.12.2015 07:08, Jan Cholasta wrote: >>> On 10.12.2015 15:56, Martin Babinsky wrote: >>>> On 12/10/2015 09:48 AM, Jan Cholasta wrote: >>>>> On 9.12.2015 16:38, Jan Cholasta wrote: >>>>>> On 9.12.2015 14:52, Jan Cholasta wrote: >>>>>>> On 9.12.2015 10:02, Jan Cholasta wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> the attached patches fix >>>>>>>> . >>>>>>> >>>>>>> Note that this needs selinux-policy fix to work, so put SELinux >>>>>>> into >>>>>>> permissive mode for testing: >>>>>>> . >>>>>> >>>>>> Updated patches attached. >>>>> >>>>> I screwed up a change in patch 524 and accidentally included a >>>>> chunk of >>>>> code in patch 525 that doesn't belong in it. >>>>> >>>>> Updated patches attached. >>>>> >>>>> >>>>> >>>> >>>> Patches work as expected and I was not able to find any functional >>>> problem. >>>> >>>> I have a question about the naming of the oddjob helper script: the >>>> one >>>> related to trusts is named 'com.redhat.idm.trust-fetch-domains', >>>> and the >>>> conncheck runner is named 'org.freeipa.server.conncheck'. I don't want >>>> to start another bikeshedding conversation but shouldn't we named them >>>> in a consistent fashion (either rename the first one in separate patch >>>> or rename the new helper to com.redhat.idm.server.conncheck)? >>>> >>>> I understand that as an upstream, we should go with the >>>> 'org.freeipa.*' >>>> convention, but having two helpers with different prefixes makes me >>>> sad. >>> >>> If you look at the larger picture, org.freeipa is the consistent name. >>> It makes me sad as well, but mistakes should be corrected. This is >>> similar to how we use PEP8 in new code, but do not fix it in old code >>> just for the sake of fixing it. >>> >>>> >>>> That is a nitpick though, it does not affect the overall functionality >>>> of the patches so ACK. >>> >>> Thanks for the review. The current patch 523 breaks the trusts oddjob >>> with SELinux in enforcing mode, I will send an update which corrects >>> that, until bug 1289930 is fixed. >> >> Updated patches attached. > > Rebased on top of current master. > > > Just question, should be any kinited user allowed to run conncheck via rpc? Martin^2 -------------- next part -------------- An HTML attachment was scrubbed... URL: From pvoborni at redhat.com Fri Dec 11 16:44:25 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 11 Dec 2015 17:44:25 +0100 Subject: [Freeipa-devel] [PATCH 0018] Fixed install_ca and install_kra failures at domain level 0 In-Reply-To: <566AF99A.4060604@redhat.com> References: <566AF99A.4060604@redhat.com> Message-ID: <566AFD69.90706@redhat.com> On 12/11/2015 05:28 PM, Oleg Fayans wrote: HI Oleg, could you prefix the commit message and mail subject with "tests: " or something similar to make clear that this is a fix in tests and not actual CA or KRA installation. -- Petr Vobornik From mbasti at redhat.com Fri Dec 11 16:58:54 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 11 Dec 2015 17:58:54 +0100 Subject: [Freeipa-devel] [PATCH 0018] Fixed install_ca and install_kra failures at domain level 0 In-Reply-To: <566AF99A.4060604@redhat.com> References: <566AF99A.4060604@redhat.com> Message-ID: <566B00CE.4040408@redhat.com> On 11.12.2015 17:28, Oleg Fayans wrote: > + myre = re.compile(".*Backed up to (?P.*?)\n.*") IMO this regexp is not good. 1) please name it better than "myre" 2) initial '.*' is not needed because regexp does not start with '^' and you use search() later 3) trailing '.*' is not needed as well, because it does not end with '$' 4) You can use re.MULTILINE that will parse string per lines path_re = re.compile("^Backed up to (?P.*)$", re.MULTILINE) 5) + matched = myre.search(result.stdout_text + result.stderr_text) Why do you need search in both stderr and stdout? Martin^2 From mbasti at redhat.com Fri Dec 11 17:05:18 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 11 Dec 2015 18:05:18 +0100 Subject: [Freeipa-devel] [PATCH 0395] replicainstall: Make sure the enrollment state is preserved In-Reply-To: <566AC990.1070605@redhat.com> References: <56697BA3.6030209@redhat.com> <56697CA6.80500@redhat.com> <566AC990.1070605@redhat.com> Message-ID: <566B024E.2050400@redhat.com> On 11.12.2015 14:03, Tomas Babej wrote: > > On 12/10/2015 02:22 PM, Tomas Babej wrote: >> >> On 12/10/2015 02:18 PM, Tomas Babej wrote: >>> Hi, >>> >>> During the promote_check phase, the subsequent checks after the machine >>> is enrolled may cause the installation to abort, hence leaving it >>> enrolled even though it might not have been prior to the execution of >>> the ipa-replica-install command. >>> >>> Make sure that ipa-client-install --uninstall is called on the machine >>> that has not been enrolled before in case of failure during the >>> promote_check phase. >>> >>> https://fedorahosted.org/freeipa/ticket/5529 >>> >> Self-NACK, installer is redundant for uninstall_client. >> >> Updated patch attached. >> >> Tomas >> >> >> > Attaching rebased version. > > ACK -------------- next part -------------- An HTML attachment was scrubbed... URL: From tbabej at redhat.com Fri Dec 11 17:08:05 2015 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 11 Dec 2015 18:08:05 +0100 Subject: [Freeipa-devel] [PATCH 0395] replicainstall: Make sure the enrollment state is preserved In-Reply-To: <566B024E.2050400@redhat.com> References: <56697BA3.6030209@redhat.com> <56697CA6.80500@redhat.com> <566AC990.1070605@redhat.com> <566B024E.2050400@redhat.com> Message-ID: <566B02F5.5070604@redhat.com> On 12/11/2015 06:05 PM, Martin Basti wrote: > > > On 11.12.2015 14:03, Tomas Babej wrote: >> >> On 12/10/2015 02:22 PM, Tomas Babej wrote: >>> >>> On 12/10/2015 02:18 PM, Tomas Babej wrote: >>>> Hi, >>>> >>>> During the promote_check phase, the subsequent checks after the machine >>>> is enrolled may cause the installation to abort, hence leaving it >>>> enrolled even though it might not have been prior to the execution of >>>> the ipa-replica-install command. >>>> >>>> Make sure that ipa-client-install --uninstall is called on the machine >>>> that has not been enrolled before in case of failure during the >>>> promote_check phase. >>>> >>>> https://fedorahosted.org/freeipa/ticket/5529 >>>> >>> Self-NACK, installer is redundant for uninstall_client. >>> >>> Updated patch attached. >>> >>> Tomas >>> >>> >>> >> Attaching rebased version. >> >> > ACK Pushed to master: 90f7fa074ac10f907c7a300305e17e6de17bd29a From tbabej at redhat.com Fri Dec 11 17:44:56 2015 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 11 Dec 2015 18:44:56 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <566AE079.2030501@redhat.com> References: <5667EE2C.2050100@redhat.com> <56683238.3050700@redhat.com> <56684B03.2020905@redhat.com> <56693C49.6030508@redhat.com> <566992B1.6030702@redhat.com> <566A6875.1070300@redhat.com> <566A755D.6000701@redhat.com> <566AE079.2030501@redhat.com> Message-ID: <566B0B98.2090008@redhat.com> On 12/11/2015 03:40 PM, Jan Cholasta wrote: > On 11.12.2015 08:03, Jan Cholasta wrote: >> On 11.12.2015 07:08, Jan Cholasta wrote: >>> On 10.12.2015 15:56, Martin Babinsky wrote: >>>> On 12/10/2015 09:48 AM, Jan Cholasta wrote: >>>>> On 9.12.2015 16:38, Jan Cholasta wrote: >>>>>> On 9.12.2015 14:52, Jan Cholasta wrote: >>>>>>> On 9.12.2015 10:02, Jan Cholasta wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> the attached patches fix >>>>>>>> . >>>>>>> >>>>>>> Note that this needs selinux-policy fix to work, so put SELinux into >>>>>>> permissive mode for testing: >>>>>>> . >>>>>> >>>>>> Updated patches attached. >>>>> >>>>> I screwed up a change in patch 524 and accidentally included a >>>>> chunk of >>>>> code in patch 525 that doesn't belong in it. >>>>> >>>>> Updated patches attached. >>>>> >>>>> >>>>> >>>> >>>> Patches work as expected and I was not able to find any functional >>>> problem. >>>> >>>> I have a question about the naming of the oddjob helper script: the one >>>> related to trusts is named 'com.redhat.idm.trust-fetch-domains', and >>>> the >>>> conncheck runner is named 'org.freeipa.server.conncheck'. I don't want >>>> to start another bikeshedding conversation but shouldn't we named them >>>> in a consistent fashion (either rename the first one in separate patch >>>> or rename the new helper to com.redhat.idm.server.conncheck)? >>>> >>>> I understand that as an upstream, we should go with the 'org.freeipa.*' >>>> convention, but having two helpers with different prefixes makes me >>>> sad. >>> >>> If you look at the larger picture, org.freeipa is the consistent name. >>> It makes me sad as well, but mistakes should be corrected. This is >>> similar to how we use PEP8 in new code, but do not fix it in old code >>> just for the sake of fixing it. >>> >>>> >>>> That is a nitpick though, it does not affect the overall functionality >>>> of the patches so ACK. >>> >>> Thanks for the review. The current patch 523 breaks the trusts oddjob >>> with SELinux in enforcing mode, I will send an update which corrects >>> that, until bug 1289930 is fixed. >> >> Updated patches attached. > > Rebased on top of current master. > > > ACK from me too, Pushed to master: 14a44ea47bf9a617019ebc91fbe272215c428d82 From mbasti at redhat.com Fri Dec 11 17:45:53 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 11 Dec 2015 18:45:53 +0100 Subject: [Freeipa-devel] [PATCH 0065] ipa-replica-install prints incorrect error message when replica is already installed In-Reply-To: References: <5666F2B8.1090803@redhat.com> <5666F3BB.8030006@redhat.com> <5666F497.3040500@redhat.com> <5666F539.1000904@redhat.com> <5666F6F0.2050005@redhat.com> <5667C566.5000906@redhat.com> <5667F667.9040603@redhat.com> Message-ID: <566B0BD1.2010802@redhat.com> ACK Pushed to master: 12e7f71600e62eab9d48a13fba37d2f182c8bdee On 09.12.2015 14:44, Gabe Alford wrote: > Fixed. Updated patch attached. > > On Wed, Dec 9, 2015 at 2:37 AM, Martin Basti > wrote: > > NACK > > Patch contains syntax error, missing brace > > ipaserver/install/server/replicainstall.py:850: > [E0001(syntax-error), ] invalid syntax) > > Martin > > > On 09.12.2015 07:08, Jan Cholasta wrote: > > LGTM > > On 8.12.2015 17:04, Gabe Alford wrote: > > Updated patch attached. > > On Tue, Dec 8, 2015 at 8:27 AM, Martin Basti > > >> wrote: > > > > On 08.12.2015 16:26, Gabe Alford wrote: > > Just to confirm: > > if server is installed: > Let's stop here and not do anything else > > if domain level 0: > check if client installed and stop here > > Right? > > yes > > > > > On Tue, Dec 8, 2015 at 8:20 AM, Jan Cholasta > > >> wrote: > > On 8.12.2015 16:17, Martin Basti wrote: > > > > On 08.12.2015 16:14, Jan Cholasta wrote: > > On 8.12.2015 16:09, Martin Basti wrote: > > > > On 01.12.2015 14:57, Gabe Alford > wrote: > > Sorry guys, I forgot to add a > meaningful > subject to this message. > Ignore the previous thread start. > > ---------- Forwarded message > ---------- > From: *Gabe Alford* > > > > > >>> > Date: Mon, Nov 30, 2015 at 7:31 PM > Subject: [PATCH 0065] > To: freeipa-devel > > > > > > >>> > > > Hello, > > Patch fix for the following > tickets: > > https://fedorahosted.org/freeipa/ticket/5022 > https://fedorahosted.org/freeipa/ticket/5320 > > Thanks, > > Gabe > > > > ACK > > > NACK, you can't install a server over > an already > installed client, > thus the original check is correct. > > Ahh domain level 0, right, but this check > can be added > before the client > check. > > > Yes. > > With domain level 1, this check should > stay there IMO. > > > Yes. It should say "IPA server is already > configured" rather > than "IPA replica is already configured", though. > > -- > Jan Cholasta > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tbabej at redhat.com Fri Dec 11 17:49:12 2015 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 11 Dec 2015 18:49:12 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <566AFBE1.2070406@redhat.com> References: <5667EE2C.2050100@redhat.com> <56683238.3050700@redhat.com> <56684B03.2020905@redhat.com> <56693C49.6030508@redhat.com> <566992B1.6030702@redhat.com> <566A6875.1070300@redhat.com> <566A755D.6000701@redhat.com> <566AE079.2030501@redhat.com> <566AFBE1.2070406@redhat.com> Message-ID: <566B0C98.8090108@redhat.com> On 12/11/2015 05:37 PM, Martin Basti wrote: > > > On 11.12.2015 15:40, Jan Cholasta wrote: >> On 11.12.2015 08:03, Jan Cholasta wrote: >>> On 11.12.2015 07:08, Jan Cholasta wrote: >>>> On 10.12.2015 15:56, Martin Babinsky wrote: >>>>> On 12/10/2015 09:48 AM, Jan Cholasta wrote: >>>>>> On 9.12.2015 16:38, Jan Cholasta wrote: >>>>>>> On 9.12.2015 14:52, Jan Cholasta wrote: >>>>>>>> On 9.12.2015 10:02, Jan Cholasta wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> the attached patches fix >>>>>>>>> . >>>>>>>> >>>>>>>> Note that this needs selinux-policy fix to work, so put SELinux >>>>>>>> into >>>>>>>> permissive mode for testing: >>>>>>>> . >>>>>>> >>>>>>> Updated patches attached. >>>>>> >>>>>> I screwed up a change in patch 524 and accidentally included a >>>>>> chunk of >>>>>> code in patch 525 that doesn't belong in it. >>>>>> >>>>>> Updated patches attached. >>>>>> >>>>>> >>>>>> >>>>> >>>>> Patches work as expected and I was not able to find any functional >>>>> problem. >>>>> >>>>> I have a question about the naming of the oddjob helper script: the >>>>> one >>>>> related to trusts is named 'com.redhat.idm.trust-fetch-domains', >>>>> and the >>>>> conncheck runner is named 'org.freeipa.server.conncheck'. I don't want >>>>> to start another bikeshedding conversation but shouldn't we named them >>>>> in a consistent fashion (either rename the first one in separate patch >>>>> or rename the new helper to com.redhat.idm.server.conncheck)? >>>>> >>>>> I understand that as an upstream, we should go with the >>>>> 'org.freeipa.*' >>>>> convention, but having two helpers with different prefixes makes me >>>>> sad. >>>> >>>> If you look at the larger picture, org.freeipa is the consistent name. >>>> It makes me sad as well, but mistakes should be corrected. This is >>>> similar to how we use PEP8 in new code, but do not fix it in old code >>>> just for the sake of fixing it. >>>> >>>>> >>>>> That is a nitpick though, it does not affect the overall functionality >>>>> of the patches so ACK. >>>> >>>> Thanks for the review. The current patch 523 breaks the trusts oddjob >>>> with SELinux in enforcing mode, I will send an update which corrects >>>> that, until bug 1289930 is fixed. >>> >>> Updated patches attached. >> >> Rebased on top of current master. >> >> >> > Just question, should be any kinited user allowed to run conncheck via rpc? > > Martin^2 I guess there's is little harm, any kinited user that was allowed to access the machine could perform the conncheck even without these patches: # ipa-replica-conncheck --master master.ipa.test -p random at IPA.TEST -w ratarata -a -r IPA.TEST Check connection from replica to remote master 'master.ipa.test': Directory Service: Unsecure port (389): OK Directory Service: Secure port (636): OK Kerberos KDC: TCP (88): OK Kerberos Kpasswd: TCP (464): OK HTTP Server: Unsecure port (80): OK HTTP Server: Secure port (443): OK The following list of ports use UDP protocol and would need to be checked manually: Kerberos KDC: UDP (88): SKIPPED Kerberos Kpasswd: UDP (464): SKIPPED Connection from replica to master is OK. Start listening on required ports for remote master check Get credentials to log in to remote master Check SSH connection to remote master Execute check on remote master Check connection from master to remote replica 'replica.ipa.test': Directory Service: Unsecure port (389): OK Directory Service: Secure port (636): OK Kerberos KDC: TCP (88): OK Kerberos KDC: UDP (88): OK Kerberos Kpasswd: TCP (464): OK Kerberos Kpasswd: UDP (464): OK HTTP Server: Unsecure port (80): OK HTTP Server: Secure port (443): OK Connection from master to replica is OK. From mbasti at redhat.com Fri Dec 11 17:49:14 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 11 Dec 2015 18:49:14 +0100 Subject: [Freeipa-devel] [PATCH 0066] Migrate wget references to curl In-Reply-To: References: <5661E6B3.3070103@redhat.com> Message-ID: <566B0C9A.9040502@redhat.com> ACK Pushed to master: 5c9b9089b7b0d40b7e924177f99c2568aaa1b5b2 On 04.12.2015 22:55, Gabe Alford wrote: > My bad. Copy and paste error. Updated patch attached. > > Thanks, > > Gabe > > On Fri, Dec 4, 2015 at 12:17 PM, Martin Basti > wrote: > > > > On 01.12.2015 15:00, Gabe Alford wrote: >> Hello, >> >> Fix for https://fedorahosted.org/freeipa/ticket/5458 >> >> Thanks, >> >> Gabe >> >> > Hello, > > I haven't looked closer, but your patch is causing this: > > Configuring certificate server (pki-tomcatd). Estimated time: 3 > minutes 30 seconds > [1/27]: creating certificate server user > [2/27]: configuring certificate server instance > [3/27]: stopping certificate server instance to update CS.cfg > [4/27]: backing up CS.cfg > [5/27]: disabling nonces > [6/27]: set up CRL publishing > [7/27]: enable PKIX certificate path discovery and validation > [8/27]: starting certificate server instance > ipa.ipaserver.install.cainstance.CAInstance: CRITICAL Failed to > restart the Dogtag instance.See the installation log for details. > [9/27]: creating RA agent certificate database > [10/27]: importing CA chain to RA certificate database > [11/27]: fixing RA database permissions > [12/27]: setting up signing cert profile > [13/27]: setting audit signing renewal to 2 years > [14/27]: restarting certificate server > ipa.ipaserver.install.cainstance.CAInstance: CRITICAL Failed to > restart the Dogtag instance.See the installation log for details. > [15/27]: requesting RA certificate from CA > [16/27]: issuing RA agent certificate > [17/27]: adding RA agent as a trusted user > [18/27]: authorizing RA to modify profiles > [19/27]: configure certmonger for renewals > [20/27]: configure certificate renewals > [21/27]: configure RA certificate renewal > [22/27]: configure Server-Cert certificate renewal > [23/27]: Configure HTTP to proxy connections > [24/27]: restarting certificate server > > ipa.ipaserver.install.cainstance.CAInstance: CRITICAL Failed to > restart the Dogtag instance.See the installation log for details. > [25/27]: migrating certificate profiles to LDAP > [26/27]: importing IPA certificate profiles > [27/27]: adding default CA ACL > > > CA is operational and ready, but IPA installer is not able to > detect it correctly > > 2015-12-04T19:08:54Z DEBUG stderr=curl: option --connect-timeout > 30: is unknown > curl: try 'curl --help' or 'curl --manual' for more information > > Martin^2 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tbabej at redhat.com Fri Dec 11 18:01:56 2015 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 11 Dec 2015 19:01:56 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <566A8B22.5020101@redhat.com> References: <56687174.8000305@redhat.com> <20151210081357.GB21550@mail.corp.redhat.com> <56695291.5090705@redhat.com> <566990E3.2080903@redhat.com> <5669A3D3.7050608@redhat.com> <566A8B22.5020101@redhat.com> Message-ID: <566B0F94.5000007@redhat.com> On 12/11/2015 09:36 AM, Martin Kosek wrote: > On 12/10/2015 05:09 PM, Martin Basti wrote: >> >> >> On 10.12.2015 15:49, Tomas Babej wrote: >>> >>> On 12/10/2015 11:23 AM, Martin Basti wrote: >>>> >>>> On 10.12.2015 09:13, Lukas Slebodnik wrote: >>>>> On (09/12/15 19:22), Martin Basti wrote: >>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>> >>>>>> Patch attached. >>>>> >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 00:00:00 >>>>> 2001 >>>>>> From: Martin Basti >>>>>> Date: Wed, 9 Dec 2015 18:53:35 +0100 >>>>>> Subject: [PATCH] Fix version comparison >>>>>> >>>>>> Use RPM library to compare vendor versions of IPA for redhat platform >>>>>> >>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>> --- >>>>>> freeipa.spec.in | 2 ++ >>>>>> ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ >>>>>> 2 files changed, 21 insertions(+) >>>>>> >>>>>> diff --git a/freeipa.spec.in b/freeipa.spec.in >>>>>> index >>>>>> 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 >>>>>> >>>>>> >>>>>> 100644 >>>>>> --- a/freeipa.spec.in >>>>>> +++ b/freeipa.spec.in >>>>>> @@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} >>>>>> Requires: gzip >>>>>> Requires: python-gssapi >= 1.1.0 >>>>>> Requires: custodia >>>>>> +Requires: rpm-python >>>>>> +Requires: rpmdevtools >>>>> Could you explain why do you need the 2nd package? >>>>> It does not contains any python modules >>>>> and I cannot see usage of any binary in this patch >>>>> >>>>> LS >>>> Thanks for this catch, it is actually located in yum package, I rather >>>> copy stringToVersion function from there to IPA, to avoid dependency >>>> hell >>>> >>>> Updated patch attached. >>>> >>>> >>> Looking good. The __cmp__ function, however, is not available in Python >>> 3. As we will eventually support python3 in RHEL as well, maybe we >>> should make sure even platform-dependent parts are python3 compatible? >>> For the future's sake. >>> >>> Tomas >>> >> Thanks, >> >> python 3 compatible patch attached. > > I wonder why we have to add so much code here and reimplement RPM > version checking if it is already provided by rpmdevtools: > > ~~~ > $ /usr/bin/rpmdev-vercmp ipa-4.2.0-15.el7 4.2.0-15.el7_2.3; echo $? > WARNING: hyphen in release1: 4.2.0-15.el7 > > rpmdev-vercmp > rpmdev-vercmp > rpmdev-vercmp # with no arguments, prompt > > Exit status is 0 if the EVR's are equal, 11 if EVR1 is newer, and 12 if > EVR2 > is newer. Other exit statuses indicate problems. > > ipa-4.2.0-15.el7 < 4.2.0-15.el7_2.3 > 12 > ~~~ > > which could be directly called from __eq__ or __lt__, since we are in > platform specific code anyway already. > > Martin Imho we should generally prefer reaching out to a Python library rather launching a subprocess to compare the versions, it's unnecessary overhead. That said, the main argument for the usage of rpm-python over rpmdevtools I see is that rpm-python is very likely to be present on the system (i.e. it is yum's own dependency), while rpmdevtools will not be present by default. >From the standpoint of trying to minimize the size of freeipa installation (i.e recent wget -> curl migration), we should keep the spirit here and do not introduce a dependency for a collection of developer tools. /2 cents Tomas From mbasti at redhat.com Fri Dec 11 18:19:19 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 11 Dec 2015 19:19:19 +0100 Subject: [Freeipa-devel] [PATCH 0374-0375] Fix permissions on newly created directories In-Reply-To: <566989B2.2080502@redhat.com> References: <566989B2.2080502@redhat.com> Message-ID: <566B13A7.4090107@redhat.com> On 10.12.2015 15:18, Martin Basti wrote: > Hello, > > patch 0374 fixes the ticket, but I found more issues with directory > permission, I fixed them in 0375 > > https://fedorahosted.org/freeipa/ticket/5520 > > Patches attached. Patches attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0374-DNS-fix-file-permissions.patch Type: text/x-patch Size: 1514 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0375-Explicitly-call-chmod-on-newly-created-directories.patch Type: text/x-patch Size: 4931 bytes Desc: not available URL: From mbasti at redhat.com Fri Dec 11 18:23:05 2015 From: mbasti at redhat.com (Martin Basti) Date: Fri, 11 Dec 2015 19:23:05 +0100 Subject: [Freeipa-devel] [PATCH 0111] prevent crashes of server uninstall check caused by failed, 5 LDAP connections In-Reply-To: <5665A028.3010905@redhat.com> References: <5661EA3A.3070504@redhat.com> <5661EE41.1090900@redhat.com> <5665A028.3010905@redhat.com> Message-ID: <566B1489.80200@redhat.com> On 07.12.2015 16:05, Martin Babinsky wrote: > On 12/04/2015 08:49 PM, Rob Crittenden wrote: >> Martin Babinsky wrote: >>> https://fedorahosted.org/freeipa/ticket/5409 >> >> Should it also warn about the potential loss of the DNSSEC master? >> >> rob >> > > Probably, but that warrants a separate ticket IMHO. > > IIRC these checks are a part of replica deletion code at domain level > 1 in ipa-replica-manage, so it should be possible to transplant them > on server uninstaller. > https://fedorahosted.org/freeipa/ticket/5544 ACK Rebased. Pushed to master: 4cc206b0f82dd68d615f0aebba5b03acf127f53a From fskola at redhat.com Sun Dec 13 11:53:27 2015 From: fskola at redhat.com (Filip Skola) Date: Sun, 13 Dec 2015 06:53:27 -0500 (EST) Subject: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin In-Reply-To: <273613290.27075436.1449834011983.JavaMail.zimbra@redhat.com> References: <20151106113201.5e6a7042@dhcp-24-123.brq.redhat.com> <20151110121350.06295a68@dhcp-24-103.brq.redhat.com> <56448E17.2010409@redhat.com> <20151203173843.37179230@vor2.netbox.priv> <869710129.24860544.1449498571599.JavaMail.zimbra@redhat.com> <20151210112952.3ddb7136@vor2.netbox.priv> <964966782.26674717.1449760307850.JavaMail.zimbra@redhat.com> <273613290.27075436.1449834011983.JavaMail.zimbra@redhat.com> Message-ID: <184590988.47122995.1450007607203.JavaMail.zimbra@redhat.com> Hi, I don't know what is causing the \r\n issue. I use vim and than send each email with claws-mail. Didn't spot this issue when trying emailing the patch to my other address. I'm trying to send it from zimbra now, let me know if that helped pls. Fix for the stageuser plugin issues caused by this patch should have been included in the last update; I think the remaining issue is not caused by UserTracker changes. Please correct me, if I'm wrong. > There is some issue with "test_rename_to_too_long_login" test. It fails but > actually this is false positive because it is possible to create login upto > 255 characters. I don't know why test mentions 32 characters without any > other modified setup. > NACK for now. > - alich - This has been changed. This test still fails, though. Filip > > > ----- Original Message ----- > > From: "Ale? Mare?ek" > > To: "Filip ?kola" > > Cc: freeipa-devel at redhat.com, "Milan Kub?k" > > Sent: Thursday, December 10, 2015 4:11:47 PM > > Subject: Re: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin > > > > Ah, sorry, haven't realized there had been devel list attached. > > Ok, there is some problem with \r\n in the patch. > > Filip, please take a look at it... > > Thanks... > > - alich - > > > > ----- Original Message ----- > > > From: "Filip ?kola" > > > To: "Ale? Mare?ek" > > > Cc: freeipa-devel at redhat.com, "Milan Kub?k" > > > Sent: Thursday, December 10, 2015 11:29:52 AM > > > Subject: Re: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin > > > > > > Hi, > > > > > > this if fixed. Also issues with test_stageuser_plugin caused by > > > UserTracker changes should be fixed here. > > > > > > Filip > > > > > > > > > On Mon, 7 Dec 2015 09:29:31 -0500 (EST) > > > Ale? Mare?ek wrote: > > > > > > > NACK. > > > > > > > > $ ./make-lint > > > > ************* Module ipatests.test_xmlrpc.test_user_plugin > > > > ipatests/test_xmlrpc/test_user_plugin.py:42: > > > > [E0611(no-name-in-module), ] No name 'ldaptracker' in module > > > > 'ipatests.test_xmlrpc') > > > > > > > > $ grep ldaptracker ipatests/test_xmlrpc/test_user_plugin.py > > > > from ipatests.test_xmlrpc.ldaptracker import Tracker > > > > $ ls ipatests/test_xmlrpc/ldaptracker* > > > > ls: cannot access ipatests/test_xmlrpc/ldaptracker*: No such file or > > > > directory > > > > > > > > > > > > ----- Original Message ----- > > > > > From: "Filip ?kola" > > > > > To: "Milan Kub?k" > > > > > Cc: freeipa-devel at redhat.com > > > > > Sent: Thursday, December 3, 2015 5:38:43 PM > > > > > Subject: Re: [Freeipa-devel] [PATCH] 0001 Refactor test_user_plugin > > > > > > > > > > Hi, > > > > > > > > > > sending corrected version. > > > > > > > > > > F. > > > > > > > > > > On Thu, 12 Nov 2015 14:03:19 +0100 > > > > > Milan Kub?k wrote: > > > > > > > > > > > On 11/10/2015 12:13 PM, Filip ?kola wrote: > > > > > > > Hi, > > > > > > > > > > > > > > fixed. > > > > > > > > > > > > > > F. > > > > > > > > > > > > > > On Tue, 10 Nov 2015 10:52:45 +0100 > > > > > > > Milan Kub?k wrote: > > > > > > > > > > > > > >> On 11/09/2015 04:35 PM, Filip ?kola wrote: > > > > > > >>> Another patch was applied in the meantime. > > > > > > >>> > > > > > > >>> Attaching an updated version. > > > > > > >>> > > > > > > >>> F. > > > > > > >>> > > > > > > >>> On Mon, 9 Nov 2015 13:35:02 +0100 > > > > > > >>> Milan Kub?k wrote: > > > > > > >>> > > > > > > >>>> On 11/06/2015 11:32 AM, Filip ?kola wrote: > > > > > > >>>> Hi, > > > > > > >>>> the patch doesn't apply. > > > > > > >>>> > > > > > > >> Please fix this. > > > > > > >> > > > > > > >> ipatests/test_xmlrpc/test_user_plugin.py:1419: > > > > > > >> [E0602(undefined-variable), > > > > > > >> TestDeniedBindWithExpiredPrincipal.teardown_class] Undefined > > > > > > >> variable 'user1') > > > > > > >> > > > > > > >> Also, use the version numbers for your changed patches. > > > > > > >> > > > > > > > > > > > > > > > > > > > > Thanks for the patch. Several issues: > > > > > > > > > > > > 1. Use dict.items instead of dict.iteritems, for python3 > > > > > > compatibility > > > > > > > > > > > > 2. What is the purpose of TestPrepare class? The 'purge' methods > > > > > > do not call any ipa commands. > > > > > > Tracker.make_fixture should be used to make the Tracked resources > > > > > > clean themselves up when they're out of scope. > > > > > > > > > > > > 3. Why reference the resources by hardcoded name if they have a > > > > > > fixture representation? > > > > > > > > > > > > 4. Rewrite {create,delete}_test_group to a fixture. You may want > > > > > > to use different scope (or not). > > > > > > > > > > > > 5. In `def atest_rename_to_invalid_login(self, user):` - use > > > > > > pytest.skipif decorator and provide a reason if you must, > > > > > > do not obfuscate method name in order not to run it. > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Manage your subscription for the Freeipa-devel mailing list: > > > > > https://www.redhat.com/mailman/listinfo/freeipa-devel > > > > > Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code > > > > > > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-fskola-0001-5-Refactor-test_user_plugin.patch Type: text/x-patch Size: 98562 bytes Desc: not available URL: From ftweedal at redhat.com Mon Dec 14 05:56:51 2015 From: ftweedal at redhat.com (Fraser Tweedale) Date: Mon, 14 Dec 2015 15:56:51 +1000 Subject: [Freeipa-devel] [PATCH] 0047 dogtaginstance: remove unused function 'check_inst' Message-ID: <20151214055650.GD23644@dhcp-40-8.bne.redhat.com> Just some drive-by cleanup of an unused function. Cheers, Fraser -------------- next part -------------- From 6eb963aac30376a1d86bbdc4b9ce299cbec5220a Mon Sep 17 00:00:00 2001 From: Fraser Tweedale Date: Mon, 14 Dec 2015 16:52:40 +1100 Subject: [PATCH] dogtaginstance: remove unused function 'check_inst' --- ipaplatform/base/paths.py | 1 - ipaserver/install/dogtaginstance.py | 17 ----------------- 2 files changed, 18 deletions(-) diff --git a/ipaplatform/base/paths.py b/ipaplatform/base/paths.py index 762a38136e6c612767705389ee667b6f2ddab397..5513d77c831f8f93ba5878fd7b1d0f22a3640651 100644 --- a/ipaplatform/base/paths.py +++ b/ipaplatform/base/paths.py @@ -236,7 +236,6 @@ class BasePathNamespace(object): SCHEMA_COMPAT_ULDIF = "/usr/share/ipa/schema_compat.uldif" IPA_JS_PLUGINS_DIR = "/usr/share/ipa/ui/js/plugins" UPDATES_DIR = "/usr/share/ipa/updates/" - PKI_CONF_SERVER_XML_TEMPLATE = "/usr/share/pki/%s/conf/server.xml" CACHE_IPA_SESSIONS = "/var/cache/ipa/sessions" VAR_KERBEROS_KRB5KDC_DIR = "/var/kerberos/krb5kdc/" VAR_KRB5KDC_K5_REALM = "/var/kerberos/krb5kdc/.k5." diff --git a/ipaserver/install/dogtaginstance.py b/ipaserver/install/dogtaginstance.py index aad6fbbe5b00aa9d352d87b66ee3e7f91bf1a64e..193423d7e09cec17a82d4f5da2ed6c43accf1c0c 100644 --- a/ipaserver/install/dogtaginstance.py +++ b/ipaserver/install/dogtaginstance.py @@ -47,23 +47,6 @@ from ipapython.ipa_log_manager import log_mgr PKI_USER = "pkiuser" -def check_inst(subsystem): - """ - Validate that the appropriate dogtag/RHCS packages have been installed. - """ - - # Check for a couple of binaries we need - if not os.path.exists(paths.PKISPAWN): - return False - if not os.path.exists(paths.PKIDESTROY): - return False - - if not os.path.exists(paths.PKI_CONF_SERVER_XML_TEMPLATE % subsystem): - return False - - return True - - def get_security_domain(): """ Get the security domain from the REST interface on the local Dogtag CA -- 2.4.3 From jcholast at redhat.com Mon Dec 14 06:21:46 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 14 Dec 2015 07:21:46 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <566B0F94.5000007@redhat.com> References: <56687174.8000305@redhat.com> <20151210081357.GB21550@mail.corp.redhat.com> <56695291.5090705@redhat.com> <566990E3.2080903@redhat.com> <5669A3D3.7050608@redhat.com> <566A8B22.5020101@redhat.com> <566B0F94.5000007@redhat.com> Message-ID: <566E5FFA.5050705@redhat.com> On 11.12.2015 19:01, Tomas Babej wrote: > > > On 12/11/2015 09:36 AM, Martin Kosek wrote: >> On 12/10/2015 05:09 PM, Martin Basti wrote: >>> >>> >>> On 10.12.2015 15:49, Tomas Babej wrote: >>>> >>>> On 12/10/2015 11:23 AM, Martin Basti wrote: >>>>> >>>>> On 10.12.2015 09:13, Lukas Slebodnik wrote: >>>>>> On (09/12/15 19:22), Martin Basti wrote: >>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>> >>>>>>> Patch attached. >>>>>> >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 00:00:00 >>>>>> 2001 >>>>>>> From: Martin Basti >>>>>>> Date: Wed, 9 Dec 2015 18:53:35 +0100 >>>>>>> Subject: [PATCH] Fix version comparison >>>>>>> >>>>>>> Use RPM library to compare vendor versions of IPA for redhat platform >>>>>>> >>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>> --- >>>>>>> freeipa.spec.in | 2 ++ >>>>>>> ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ >>>>>>> 2 files changed, 21 insertions(+) >>>>>>> >>>>>>> diff --git a/freeipa.spec.in b/freeipa.spec.in >>>>>>> index >>>>>>> 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 >>>>>>> >>>>>>> >>>>>>> 100644 >>>>>>> --- a/freeipa.spec.in >>>>>>> +++ b/freeipa.spec.in >>>>>>> @@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} >>>>>>> Requires: gzip >>>>>>> Requires: python-gssapi >= 1.1.0 >>>>>>> Requires: custodia >>>>>>> +Requires: rpm-python >>>>>>> +Requires: rpmdevtools >>>>>> Could you explain why do you need the 2nd package? >>>>>> It does not contains any python modules >>>>>> and I cannot see usage of any binary in this patch >>>>>> >>>>>> LS >>>>> Thanks for this catch, it is actually located in yum package, I rather >>>>> copy stringToVersion function from there to IPA, to avoid dependency >>>>> hell >>>>> >>>>> Updated patch attached. >>>>> >>>>> >>>> Looking good. The __cmp__ function, however, is not available in Python >>>> 3. As we will eventually support python3 in RHEL as well, maybe we >>>> should make sure even platform-dependent parts are python3 compatible? >>>> For the future's sake. >>>> >>>> Tomas >>>> >>> Thanks, >>> >>> python 3 compatible patch attached. >> >> I wonder why we have to add so much code here and reimplement RPM >> version checking if it is already provided by rpmdevtools: >> >> ~~~ >> $ /usr/bin/rpmdev-vercmp ipa-4.2.0-15.el7 4.2.0-15.el7_2.3; echo $? >> WARNING: hyphen in release1: 4.2.0-15.el7 >> >> rpmdev-vercmp >> rpmdev-vercmp >> rpmdev-vercmp # with no arguments, prompt >> >> Exit status is 0 if the EVR's are equal, 11 if EVR1 is newer, and 12 if >> EVR2 >> is newer. Other exit statuses indicate problems. >> >> ipa-4.2.0-15.el7 < 4.2.0-15.el7_2.3 >> 12 >> ~~~ >> >> which could be directly called from __eq__ or __lt__, since we are in >> platform specific code anyway already. >> >> Martin > > Imho we should generally prefer reaching out to a Python library rather > launching a subprocess to compare the versions, it's unnecessary overhead. > > That said, the main argument for the usage of rpm-python over > rpmdevtools I see is that rpm-python is very likely to be present on the > system (i.e. it is yum's own dependency), while rpmdevtools will not be > present by default. > > From the standpoint of trying to minimize the size of freeipa > installation (i.e recent wget -> curl migration), we should keep the > spirit here and do not introduce a dependency for a collection of > developer tools. > > /2 cents +1, also a single function is hardly "much code". -- Jan Cholasta From jcholast at redhat.com Mon Dec 14 06:23:18 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 14 Dec 2015 07:23:18 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <566B0C98.8090108@redhat.com> References: <5667EE2C.2050100@redhat.com> <56683238.3050700@redhat.com> <56684B03.2020905@redhat.com> <56693C49.6030508@redhat.com> <566992B1.6030702@redhat.com> <566A6875.1070300@redhat.com> <566A755D.6000701@redhat.com> <566AE079.2030501@redhat.com> <566AFBE1.2070406@redhat.com> <566B0C98.8090108@redhat.com> Message-ID: <566E6056.2030007@redhat.com> On 11.12.2015 18:49, Tomas Babej wrote: > > > On 12/11/2015 05:37 PM, Martin Basti wrote: >> >> >> On 11.12.2015 15:40, Jan Cholasta wrote: >>> On 11.12.2015 08:03, Jan Cholasta wrote: >>>> On 11.12.2015 07:08, Jan Cholasta wrote: >>>>> On 10.12.2015 15:56, Martin Babinsky wrote: >>>>>> On 12/10/2015 09:48 AM, Jan Cholasta wrote: >>>>>>> On 9.12.2015 16:38, Jan Cholasta wrote: >>>>>>>> On 9.12.2015 14:52, Jan Cholasta wrote: >>>>>>>>> On 9.12.2015 10:02, Jan Cholasta wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> the attached patches fix >>>>>>>>>> . >>>>>>>>> >>>>>>>>> Note that this needs selinux-policy fix to work, so put SELinux >>>>>>>>> into >>>>>>>>> permissive mode for testing: >>>>>>>>> . >>>>>>>> >>>>>>>> Updated patches attached. >>>>>>> >>>>>>> I screwed up a change in patch 524 and accidentally included a >>>>>>> chunk of >>>>>>> code in patch 525 that doesn't belong in it. >>>>>>> >>>>>>> Updated patches attached. >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> Patches work as expected and I was not able to find any functional >>>>>> problem. >>>>>> >>>>>> I have a question about the naming of the oddjob helper script: the >>>>>> one >>>>>> related to trusts is named 'com.redhat.idm.trust-fetch-domains', >>>>>> and the >>>>>> conncheck runner is named 'org.freeipa.server.conncheck'. I don't want >>>>>> to start another bikeshedding conversation but shouldn't we named them >>>>>> in a consistent fashion (either rename the first one in separate patch >>>>>> or rename the new helper to com.redhat.idm.server.conncheck)? >>>>>> >>>>>> I understand that as an upstream, we should go with the >>>>>> 'org.freeipa.*' >>>>>> convention, but having two helpers with different prefixes makes me >>>>>> sad. >>>>> >>>>> If you look at the larger picture, org.freeipa is the consistent name. >>>>> It makes me sad as well, but mistakes should be corrected. This is >>>>> similar to how we use PEP8 in new code, but do not fix it in old code >>>>> just for the sake of fixing it. >>>>> >>>>>> >>>>>> That is a nitpick though, it does not affect the overall functionality >>>>>> of the patches so ACK. >>>>> >>>>> Thanks for the review. The current patch 523 breaks the trusts oddjob >>>>> with SELinux in enforcing mode, I will send an update which corrects >>>>> that, until bug 1289930 is fixed. >>>> >>>> Updated patches attached. >>> >>> Rebased on top of current master. >>> >>> >>> >> Just question, should be any kinited user allowed to run conncheck via rpc? >> >> Martin^2 > > I guess there's is little harm, any kinited user that was allowed to > access the machine could perform the conncheck even without these patches: In the RPC check, the user must have the Replication Administrators privilege, which by default only admins have. > > # ipa-replica-conncheck --master master.ipa.test -p random at IPA.TEST -w > ratarata -a -r IPA.TEST > Check connection from replica to remote master 'master.ipa.test': > Directory Service: Unsecure port (389): OK > Directory Service: Secure port (636): OK > Kerberos KDC: TCP (88): OK > Kerberos Kpasswd: TCP (464): OK > HTTP Server: Unsecure port (80): OK > HTTP Server: Secure port (443): OK > > The following list of ports use UDP protocol and would need to be > checked manually: > Kerberos KDC: UDP (88): SKIPPED > Kerberos Kpasswd: UDP (464): SKIPPED > > Connection from replica to master is OK. > Start listening on required ports for remote master check > Get credentials to log in to remote master > Check SSH connection to remote master > Execute check on remote master > Check connection from master to remote replica 'replica.ipa.test': > Directory Service: Unsecure port (389): OK > Directory Service: Secure port (636): OK > Kerberos KDC: TCP (88): OK > Kerberos KDC: UDP (88): OK > Kerberos Kpasswd: TCP (464): OK > Kerberos Kpasswd: UDP (464): OK > HTTP Server: Unsecure port (80): OK > HTTP Server: Secure port (443): OK > > Connection from master to replica is OK. > -- Jan Cholasta From jcholast at redhat.com Mon Dec 14 06:53:47 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 14 Dec 2015 07:53:47 +0100 Subject: [Freeipa-devel] [PATCHES 529-530] ca install: use host credentials in domain level 1 In-Reply-To: <566AF8A8.4050107@redhat.com> References: <56693CF6.8030104@redhat.com> <566AD706.90800@redhat.com> <566AF8A8.4050107@redhat.com> Message-ID: <566E677B.5@redhat.com> On 11.12.2015 17:24, Martin Basti wrote: > > > On 11.12.2015 15:00, Jan Cholasta wrote: >> On 10.12.2015 09:51, Jan Cholasta wrote: >>> Hi, >>> >>> the attached patches fix . >>> >>> My patches 523-525 are required for this: >>> . >>> >>> >>> >>> Honza >> >> Rebased patches attached. >> > Patch works for me, but can you provide explanations (and update commit > message) why the ACI change is needed: > > * why it is moved three ACIs from 'cn="$SUFFIX",cn=mapping > tree,cn=config' to 'cn=mapping tree,cn=config' So that they apply to all replication agreements. > * why you removed completely 'dn: cn=o\3Dipaca,cn=mapping tree,cn=config' I didn't, they were moved to cn=mapping tree,cn=config as well. Updated patches attached. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-529.2-aci-merge-domain-and-CA-suffix-replication-agreement.patch Type: text/x-patch Size: 5967 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-530.2-ca-install-use-host-credentials-in-domain-level-1.patch Type: text/x-patch Size: 4418 bytes Desc: not available URL: From jcholast at redhat.com Mon Dec 14 07:18:57 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 14 Dec 2015 08:18:57 +0100 Subject: [Freeipa-devel] [PATCH] 0751 spec: Split out python-ipap11helper and, python-default_encoding_utf8 In-Reply-To: <56619535.1010700@redhat.com> References: <56606E90.3000909@redhat.com> <56619535.1010700@redhat.com> Message-ID: <566E6D61.80002@redhat.com> On 4.12.2015 14:29, Jan Cholasta wrote: > Hi, > > On 3.12.2015 17:32, Petr Viktorin wrote: >> Hello, >> This specfile patch makes python-ipalib noarch, by splitting out the >> arch-dependent stuff. > > I'm not sure if this should be done at all. Both py_default_encoding and > (especially) _ipap11helper are internal submodules of ipapython and I > don't think they should be packaged separately. > > If this is just about packaging arch-specific code separately from > noarch code, I'm not sure either - all other packages with both Python > and C modules I have seen (e.g. python-ldap, PyYAML) simply do not care > and put everything in a single arch-specific package. IMHO we should > follow. This would mean putting everything into the architecture-specific prefix. Currently the python files in python2-ipalib are installed into the architecture-agnostic prefix, which is wrong, because you can't install both python2-ipalib.i686 and python2-ipalib.x86_64 at the same time. > >> >> It depends on Honza Cholasta's patches 516 and 517. FYI the patches were pushed on Friday. >> >> Part of https://fedorahosted.org/freeipa/ticket/3197 >> > > Honza > -- Jan Cholasta From mkosek at redhat.com Mon Dec 14 08:24:38 2015 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 14 Dec 2015 09:24:38 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <566E5FFA.5050705@redhat.com> References: <56687174.8000305@redhat.com> <20151210081357.GB21550@mail.corp.redhat.com> <56695291.5090705@redhat.com> <566990E3.2080903@redhat.com> <5669A3D3.7050608@redhat.com> <566A8B22.5020101@redhat.com> <566B0F94.5000007@redhat.com> <566E5FFA.5050705@redhat.com> Message-ID: <566E7CC6.1010201@redhat.com> On 12/14/2015 07:21 AM, Jan Cholasta wrote: > On 11.12.2015 19:01, Tomas Babej wrote: >> >> >> On 12/11/2015 09:36 AM, Martin Kosek wrote: >>> On 12/10/2015 05:09 PM, Martin Basti wrote: >>>> >>>> >>>> On 10.12.2015 15:49, Tomas Babej wrote: >>>>> >>>>> On 12/10/2015 11:23 AM, Martin Basti wrote: >>>>>> >>>>>> On 10.12.2015 09:13, Lukas Slebodnik wrote: >>>>>>> On (09/12/15 19:22), Martin Basti wrote: >>>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>>> >>>>>>>> Patch attached. >>>>>>> >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 00:00:00 >>>>>>> 2001 >>>>>>>> From: Martin Basti >>>>>>>> Date: Wed, 9 Dec 2015 18:53:35 +0100 >>>>>>>> Subject: [PATCH] Fix version comparison >>>>>>>> >>>>>>>> Use RPM library to compare vendor versions of IPA for redhat platform >>>>>>>> >>>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>>> --- >>>>>>>> freeipa.spec.in | 2 ++ >>>>>>>> ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ >>>>>>>> 2 files changed, 21 insertions(+) >>>>>>>> >>>>>>>> diff --git a/freeipa.spec.in b/freeipa.spec.in >>>>>>>> index >>>>>>>> 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 100644 >>>>>>>> --- a/freeipa.spec.in >>>>>>>> +++ b/freeipa.spec.in >>>>>>>> @@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} >>>>>>>> Requires: gzip >>>>>>>> Requires: python-gssapi >= 1.1.0 >>>>>>>> Requires: custodia >>>>>>>> +Requires: rpm-python >>>>>>>> +Requires: rpmdevtools >>>>>>> Could you explain why do you need the 2nd package? >>>>>>> It does not contains any python modules >>>>>>> and I cannot see usage of any binary in this patch >>>>>>> >>>>>>> LS >>>>>> Thanks for this catch, it is actually located in yum package, I rather >>>>>> copy stringToVersion function from there to IPA, to avoid dependency >>>>>> hell >>>>>> >>>>>> Updated patch attached. >>>>>> >>>>>> >>>>> Looking good. The __cmp__ function, however, is not available in Python >>>>> 3. As we will eventually support python3 in RHEL as well, maybe we >>>>> should make sure even platform-dependent parts are python3 compatible? >>>>> For the future's sake. >>>>> >>>>> Tomas >>>>> >>>> Thanks, >>>> >>>> python 3 compatible patch attached. >>> >>> I wonder why we have to add so much code here and reimplement RPM >>> version checking if it is already provided by rpmdevtools: >>> >>> ~~~ >>> $ /usr/bin/rpmdev-vercmp ipa-4.2.0-15.el7 4.2.0-15.el7_2.3; echo $? >>> WARNING: hyphen in release1: 4.2.0-15.el7 >>> >>> rpmdev-vercmp >>> rpmdev-vercmp >>> rpmdev-vercmp # with no arguments, prompt >>> >>> Exit status is 0 if the EVR's are equal, 11 if EVR1 is newer, and 12 if >>> EVR2 >>> is newer. Other exit statuses indicate problems. >>> >>> ipa-4.2.0-15.el7 < 4.2.0-15.el7_2.3 >>> 12 >>> ~~~ >>> >>> which could be directly called from __eq__ or __lt__, since we are in >>> platform specific code anyway already. >>> >>> Martin >> >> Imho we should generally prefer reaching out to a Python library rather >> launching a subprocess to compare the versions, it's unnecessary overhead. I would not be too worried about miliseconds longer execution on a function called during RPM upgrade. >> That said, the main argument for the usage of rpm-python over >> rpmdevtools I see is that rpm-python is very likely to be present on the >> system (i.e. it is yum's own dependency), while rpmdevtools will not be >> present by default. >> >> From the standpoint of trying to minimize the size of freeipa >> installation (i.e recent wget -> curl migration), we should keep the >> spirit here and do not introduce a dependency for a collection of >> developer tools. >> >> /2 cents > > +1, also a single function is hardly "much code". Ok then. If you all want to add yet another N-V-R parsing function in the FreeIPA code, I can live with it (with raised eyebrows though). From pspacek at redhat.com Mon Dec 14 08:43:55 2015 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 14 Dec 2015 09:43:55 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <566E7CC6.1010201@redhat.com> References: <56687174.8000305@redhat.com> <20151210081357.GB21550@mail.corp.redhat.com> <56695291.5090705@redhat.com> <566990E3.2080903@redhat.com> <5669A3D3.7050608@redhat.com> <566A8B22.5020101@redhat.com> <566B0F94.5000007@redhat.com> <566E5FFA.5050705@redhat.com> <566E7CC6.1010201@redhat.com> Message-ID: <566E814B.3020905@redhat.com> On 14.12.2015 09:24, Martin Kosek wrote: > On 12/14/2015 07:21 AM, Jan Cholasta wrote: >> On 11.12.2015 19:01, Tomas Babej wrote: >>> >>> >>> On 12/11/2015 09:36 AM, Martin Kosek wrote: >>>> On 12/10/2015 05:09 PM, Martin Basti wrote: >>>>> >>>>> >>>>> On 10.12.2015 15:49, Tomas Babej wrote: >>>>>> >>>>>> On 12/10/2015 11:23 AM, Martin Basti wrote: >>>>>>> >>>>>>> On 10.12.2015 09:13, Lukas Slebodnik wrote: >>>>>>>> On (09/12/15 19:22), Martin Basti wrote: >>>>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>>>> >>>>>>>>> Patch attached. >>>>>>>> >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 00:00:00 >>>>>>>> 2001 >>>>>>>>> From: Martin Basti >>>>>>>>> Date: Wed, 9 Dec 2015 18:53:35 +0100 >>>>>>>>> Subject: [PATCH] Fix version comparison >>>>>>>>> >>>>>>>>> Use RPM library to compare vendor versions of IPA for redhat platform >>>>>>>>> >>>>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>>>> --- >>>>>>>>> freeipa.spec.in | 2 ++ >>>>>>>>> ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ >>>>>>>>> 2 files changed, 21 insertions(+) >>>>>>>>> >>>>>>>>> diff --git a/freeipa.spec.in b/freeipa.spec.in >>>>>>>>> index >>>>>>>>> 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> 100644 >>>>>>>>> --- a/freeipa.spec.in >>>>>>>>> +++ b/freeipa.spec.in >>>>>>>>> @@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} >>>>>>>>> Requires: gzip >>>>>>>>> Requires: python-gssapi >= 1.1.0 >>>>>>>>> Requires: custodia >>>>>>>>> +Requires: rpm-python >>>>>>>>> +Requires: rpmdevtools >>>>>>>> Could you explain why do you need the 2nd package? >>>>>>>> It does not contains any python modules >>>>>>>> and I cannot see usage of any binary in this patch >>>>>>>> >>>>>>>> LS >>>>>>> Thanks for this catch, it is actually located in yum package, I rather >>>>>>> copy stringToVersion function from there to IPA, to avoid dependency >>>>>>> hell >>>>>>> >>>>>>> Updated patch attached. >>>>>>> >>>>>>> >>>>>> Looking good. The __cmp__ function, however, is not available in Python >>>>>> 3. As we will eventually support python3 in RHEL as well, maybe we >>>>>> should make sure even platform-dependent parts are python3 compatible? >>>>>> For the future's sake. >>>>>> >>>>>> Tomas >>>>>> >>>>> Thanks, >>>>> >>>>> python 3 compatible patch attached. >>>> >>>> I wonder why we have to add so much code here and reimplement RPM >>>> version checking if it is already provided by rpmdevtools: >>>> >>>> ~~~ >>>> $ /usr/bin/rpmdev-vercmp ipa-4.2.0-15.el7 4.2.0-15.el7_2.3; echo $? >>>> WARNING: hyphen in release1: 4.2.0-15.el7 >>>> >>>> rpmdev-vercmp >>>> rpmdev-vercmp >>>> rpmdev-vercmp # with no arguments, prompt >>>> >>>> Exit status is 0 if the EVR's are equal, 11 if EVR1 is newer, and 12 if >>>> EVR2 >>>> is newer. Other exit statuses indicate problems. >>>> >>>> ipa-4.2.0-15.el7 < 4.2.0-15.el7_2.3 >>>> 12 >>>> ~~~ >>>> >>>> which could be directly called from __eq__ or __lt__, since we are in >>>> platform specific code anyway already. >>>> >>>> Martin >>> >>> Imho we should generally prefer reaching out to a Python library rather >>> launching a subprocess to compare the versions, it's unnecessary overhead. > > I would not be too worried about miliseconds longer execution on a function > called during RPM upgrade. > >>> That said, the main argument for the usage of rpm-python over >>> rpmdevtools I see is that rpm-python is very likely to be present on the >>> system (i.e. it is yum's own dependency), while rpmdevtools will not be >>> present by default. >>> >>> From the standpoint of trying to minimize the size of freeipa >>> installation (i.e recent wget -> curl migration), we should keep the >>> spirit here and do not introduce a dependency for a collection of >>> developer tools. >>> >>> /2 cents >> >> +1, also a single function is hardly "much code". > > Ok then. If you all want to add yet another N-V-R parsing function in the > FreeIPA code, I can live with it (with raised eyebrows though). +1 We already tried and failed in custom parsing (presumably caused by request to not add any new dependency). I agree with Martin that copy&pasting code does not look like a proper fix. -- Petr^2 Spacek From mbabinsk at redhat.com Mon Dec 14 08:59:38 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Mon, 14 Dec 2015 09:59:38 +0100 Subject: [Freeipa-devel] [PATCH 0114] harden domain level 1 topology connectivity checks In-Reply-To: <566706D9.2080006@redhat.com> References: <566706D9.2080006@redhat.com> Message-ID: <566E84FA.2070008@redhat.com> On 12/08/2015 05:35 PM, Martin Babinsky wrote: > A sort of auxilliary patch which makes topology checks more resistant to > https://fedorahosted.org/freeipa/ticket/5526 > > If required I will open a separate ticket for it though. > > > Bump for review. -- Martin^3 Babinsky From mbasti at redhat.com Mon Dec 14 09:15:54 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 14 Dec 2015 10:15:54 +0100 Subject: [Freeipa-devel] [PATCHES 529-530] ca install: use host credentials in domain level 1 In-Reply-To: <566E677B.5@redhat.com> References: <56693CF6.8030104@redhat.com> <566AD706.90800@redhat.com> <566AF8A8.4050107@redhat.com> <566E677B.5@redhat.com> Message-ID: <566E88CA.7060800@redhat.com> On 14.12.2015 07:53, Jan Cholasta wrote: > On 11.12.2015 17:24, Martin Basti wrote: >> >> >> On 11.12.2015 15:00, Jan Cholasta wrote: >>> On 10.12.2015 09:51, Jan Cholasta wrote: >>>> Hi, >>>> >>>> the attached patches fix >>>> . >>>> >>>> My patches 523-525 are required for this: >>>> . >>>> >>>> >>>> >>>> >>>> Honza >>> >>> Rebased patches attached. >>> >> Patch works for me, but can you provide explanations (and update commit >> message) why the ACI change is needed: >> >> * why it is moved three ACIs from 'cn="$SUFFIX",cn=mapping >> tree,cn=config' to 'cn=mapping tree,cn=config' > > So that they apply to all replication agreements. > >> * why you removed completely 'dn: cn=o\3Dipaca,cn=mapping >> tree,cn=config' > > I didn't, they were moved to cn=mapping tree,cn=config as well. > > Updated patches attached. > ACK From mbasti at redhat.com Mon Dec 14 09:21:35 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 14 Dec 2015 10:21:35 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <566E7CC6.1010201@redhat.com> References: <56687174.8000305@redhat.com> <20151210081357.GB21550@mail.corp.redhat.com> <56695291.5090705@redhat.com> <566990E3.2080903@redhat.com> <5669A3D3.7050608@redhat.com> <566A8B22.5020101@redhat.com> <566B0F94.5000007@redhat.com> <566E5FFA.5050705@redhat.com> <566E7CC6.1010201@redhat.com> Message-ID: <566E8A1F.70702@redhat.com> On 14.12.2015 09:24, Martin Kosek wrote: > On 12/14/2015 07:21 AM, Jan Cholasta wrote: >> On 11.12.2015 19:01, Tomas Babej wrote: >>> >>> On 12/11/2015 09:36 AM, Martin Kosek wrote: >>>> On 12/10/2015 05:09 PM, Martin Basti wrote: >>>>> >>>>> On 10.12.2015 15:49, Tomas Babej wrote: >>>>>> On 12/10/2015 11:23 AM, Martin Basti wrote: >>>>>>> On 10.12.2015 09:13, Lukas Slebodnik wrote: >>>>>>>> On (09/12/15 19:22), Martin Basti wrote: >>>>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>>>> >>>>>>>>> Patch attached. >>>>>>>> >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 00:00:00 >>>>>>>> 2001 >>>>>>>>> From: Martin Basti >>>>>>>>> Date: Wed, 9 Dec 2015 18:53:35 +0100 >>>>>>>>> Subject: [PATCH] Fix version comparison >>>>>>>>> >>>>>>>>> Use RPM library to compare vendor versions of IPA for redhat platform >>>>>>>>> >>>>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>>>> --- >>>>>>>>> freeipa.spec.in | 2 ++ >>>>>>>>> ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ >>>>>>>>> 2 files changed, 21 insertions(+) >>>>>>>>> >>>>>>>>> diff --git a/freeipa.spec.in b/freeipa.spec.in >>>>>>>>> index >>>>>>>>> 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> 100644 >>>>>>>>> --- a/freeipa.spec.in >>>>>>>>> +++ b/freeipa.spec.in >>>>>>>>> @@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} >>>>>>>>> Requires: gzip >>>>>>>>> Requires: python-gssapi >= 1.1.0 >>>>>>>>> Requires: custodia >>>>>>>>> +Requires: rpm-python >>>>>>>>> +Requires: rpmdevtools >>>>>>>> Could you explain why do you need the 2nd package? >>>>>>>> It does not contains any python modules >>>>>>>> and I cannot see usage of any binary in this patch >>>>>>>> >>>>>>>> LS >>>>>>> Thanks for this catch, it is actually located in yum package, I rather >>>>>>> copy stringToVersion function from there to IPA, to avoid dependency >>>>>>> hell >>>>>>> >>>>>>> Updated patch attached. >>>>>>> >>>>>>> >>>>>> Looking good. The __cmp__ function, however, is not available in Python >>>>>> 3. As we will eventually support python3 in RHEL as well, maybe we >>>>>> should make sure even platform-dependent parts are python3 compatible? >>>>>> For the future's sake. >>>>>> >>>>>> Tomas >>>>>> >>>>> Thanks, >>>>> >>>>> python 3 compatible patch attached. >>>> I wonder why we have to add so much code here and reimplement RPM >>>> version checking if it is already provided by rpmdevtools: >>>> >>>> ~~~ >>>> $ /usr/bin/rpmdev-vercmp ipa-4.2.0-15.el7 4.2.0-15.el7_2.3; echo $? >>>> WARNING: hyphen in release1: 4.2.0-15.el7 >>>> >>>> rpmdev-vercmp >>>> rpmdev-vercmp >>>> rpmdev-vercmp # with no arguments, prompt >>>> >>>> Exit status is 0 if the EVR's are equal, 11 if EVR1 is newer, and 12 if >>>> EVR2 >>>> is newer. Other exit statuses indicate problems. >>>> >>>> ipa-4.2.0-15.el7 < 4.2.0-15.el7_2.3 >>>> 12 >>>> ~~~ >>>> >>>> which could be directly called from __eq__ or __lt__, since we are in >>>> platform specific code anyway already. >>>> >>>> Martin >>> Imho we should generally prefer reaching out to a Python library rather >>> launching a subprocess to compare the versions, it's unnecessary overhead. > I would not be too worried about miliseconds longer execution on a function > called during RPM upgrade. > >>> That said, the main argument for the usage of rpm-python over >>> rpmdevtools I see is that rpm-python is very likely to be present on the >>> system (i.e. it is yum's own dependency), while rpmdevtools will not be >>> present by default. >>> >>> From the standpoint of trying to minimize the size of freeipa >>> installation (i.e recent wget -> curl migration), we should keep the >>> spirit here and do not introduce a dependency for a collection of >>> developer tools. >>> >>> /2 cents >> +1, also a single function is hardly "much code". > Ok then. If you all want to add yet another N-V-R parsing function in the > FreeIPA code, I can live with it (with raised eyebrows though). Rebased patch attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0373.4-Fix-version-comparison.patch Type: text/x-patch Size: 3644 bytes Desc: not available URL: From mbasti at redhat.com Mon Dec 14 09:27:15 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 14 Dec 2015 10:27:15 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <566E6056.2030007@redhat.com> References: <5667EE2C.2050100@redhat.com> <56683238.3050700@redhat.com> <56684B03.2020905@redhat.com> <56693C49.6030508@redhat.com> <566992B1.6030702@redhat.com> <566A6875.1070300@redhat.com> <566A755D.6000701@redhat.com> <566AE079.2030501@redhat.com> <566AFBE1.2070406@redhat.com> <566B0C98.8090108@redhat.com> <566E6056.2030007@redhat.com> Message-ID: <566E8B73.3050300@redhat.com> On 14.12.2015 07:23, Jan Cholasta wrote: > On 11.12.2015 18:49, Tomas Babej wrote: >> >> >> On 12/11/2015 05:37 PM, Martin Basti wrote: >>> >>> >>> On 11.12.2015 15:40, Jan Cholasta wrote: >>>> On 11.12.2015 08:03, Jan Cholasta wrote: >>>>> On 11.12.2015 07:08, Jan Cholasta wrote: >>>>>> On 10.12.2015 15:56, Martin Babinsky wrote: >>>>>>> On 12/10/2015 09:48 AM, Jan Cholasta wrote: >>>>>>>> On 9.12.2015 16:38, Jan Cholasta wrote: >>>>>>>>> On 9.12.2015 14:52, Jan Cholasta wrote: >>>>>>>>>> On 9.12.2015 10:02, Jan Cholasta wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> the attached patches fix >>>>>>>>>>> . >>>>>>>>>> >>>>>>>>>> Note that this needs selinux-policy fix to work, so put SELinux >>>>>>>>>> into >>>>>>>>>> permissive mode for testing: >>>>>>>>>> . >>>>>>>>> >>>>>>>>> Updated patches attached. >>>>>>>> >>>>>>>> I screwed up a change in patch 524 and accidentally included a >>>>>>>> chunk of >>>>>>>> code in patch 525 that doesn't belong in it. >>>>>>>> >>>>>>>> Updated patches attached. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> Patches work as expected and I was not able to find any functional >>>>>>> problem. >>>>>>> >>>>>>> I have a question about the naming of the oddjob helper script: the >>>>>>> one >>>>>>> related to trusts is named 'com.redhat.idm.trust-fetch-domains', >>>>>>> and the >>>>>>> conncheck runner is named 'org.freeipa.server.conncheck'. I >>>>>>> don't want >>>>>>> to start another bikeshedding conversation but shouldn't we >>>>>>> named them >>>>>>> in a consistent fashion (either rename the first one in separate >>>>>>> patch >>>>>>> or rename the new helper to com.redhat.idm.server.conncheck)? >>>>>>> >>>>>>> I understand that as an upstream, we should go with the >>>>>>> 'org.freeipa.*' >>>>>>> convention, but having two helpers with different prefixes makes me >>>>>>> sad. >>>>>> >>>>>> If you look at the larger picture, org.freeipa is the consistent >>>>>> name. >>>>>> It makes me sad as well, but mistakes should be corrected. This is >>>>>> similar to how we use PEP8 in new code, but do not fix it in old >>>>>> code >>>>>> just for the sake of fixing it. >>>>>> >>>>>>> >>>>>>> That is a nitpick though, it does not affect the overall >>>>>>> functionality >>>>>>> of the patches so ACK. >>>>>> >>>>>> Thanks for the review. The current patch 523 breaks the trusts >>>>>> oddjob >>>>>> with SELinux in enforcing mode, I will send an update which corrects >>>>>> that, until bug 1289930 is fixed. >>>>> >>>>> Updated patches attached. >>>> >>>> Rebased on top of current master. >>>> >>>> >>>> >>> Just question, should be any kinited user allowed to run conncheck >>> via rpc? >>> >>> Martin^2 >> >> I guess there's is little harm, any kinited user that was allowed to >> access the machine could perform the conncheck even without these >> patches: > > In the RPC check, the user must have the Replication Administrators > privilege, which by default only admins have. I tried to install replica with a regular user and conncheck passed. Martin^2 > >> >> # ipa-replica-conncheck --master master.ipa.test -p random at IPA.TEST -w >> ratarata -a -r IPA.TEST >> Check connection from replica to remote master 'master.ipa.test': >> Directory Service: Unsecure port (389): OK >> Directory Service: Secure port (636): OK >> Kerberos KDC: TCP (88): OK >> Kerberos Kpasswd: TCP (464): OK >> HTTP Server: Unsecure port (80): OK >> HTTP Server: Secure port (443): OK >> >> The following list of ports use UDP protocol and would need to be >> checked manually: >> Kerberos KDC: UDP (88): SKIPPED >> Kerberos Kpasswd: UDP (464): SKIPPED >> >> Connection from replica to master is OK. >> Start listening on required ports for remote master check >> Get credentials to log in to remote master >> Check SSH connection to remote master >> Execute check on remote master >> Check connection from master to remote replica 'replica.ipa.test': >> Directory Service: Unsecure port (389): OK >> Directory Service: Secure port (636): OK >> Kerberos KDC: TCP (88): OK >> Kerberos KDC: UDP (88): OK >> Kerberos Kpasswd: TCP (464): OK >> Kerberos Kpasswd: UDP (464): OK >> HTTP Server: Unsecure port (80): OK >> HTTP Server: Secure port (443): OK >> >> Connection from master to replica is OK. >> > > From jcholast at redhat.com Mon Dec 14 09:29:07 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 14 Dec 2015 10:29:07 +0100 Subject: [Freeipa-devel] [PATCH] 0748 Handle encoding for ipautil.run In-Reply-To: <56680AD9.7060703@redhat.com> References: <5638B8F9.6010703@redhat.com> <564D0FD9.4070505@redhat.com> <5652B5AC.2040508@redhat.com> <5652E174.50301@redhat.com> <56548E73.1090105@redhat.com> <565587AE.2070303@redhat.com> <5655CA08.8090701@redhat.com> <565C01FD.7050906@redhat.com> <565D83CC.8020802@redhat.com> <565EA4FA.3020307@redhat.com> <565EF7B9.6020507@redhat.com> <566011A5.4040200@redhat.com> <566054D7.9020008@redhat.com> <566145FE.3090005@redhat.com> <566162B2.9010809@redhat.com> <566175D0.1000105@redhat.com> <56617C48.4030005@redhat.com> <56617DBB.9000109@redhat.com> <56680AD9.7060703@redhat.com> Message-ID: <566E8BE3.7030007@redhat.com> On 9.12.2015 12:04, Petr Viktorin wrote: > On 12/04/2015 12:49 PM, Jan Cholasta wrote: >> On 4.12.2015 12:43, Petr Viktorin wrote: >>> On 12/04/2015 12:15 PM, Jan Cholasta wrote: >>>> On 4.12.2015 10:53, Petr Viktorin wrote: >>>>> On 12/04/2015 08:51 AM, Jan Cholasta wrote: >>>>>> On 3.12.2015 15:42, Petr Viktorin wrote: >>>>>>> On 12/03/2015 10:55 AM, Jan Cholasta wrote: >>> [...] >>>>>>>> If we do encode/decode in run(), I think there should be a way to >>>>>>>> override the default encoding. My idea was to either accept/return >>>>>>>> only >>>>>>>> bytes and let the caller handle encoding themselves, or to handle >>>>>>>> encoding in run(), but be able to override the defaults. >>>>>>>> >>>>>>>> Given we handle encoding in run(), I would imagine it like this: >>>>>>>> >>>>>>>> def run(args, stdin=None, raiseonerr=True, nolog=(), >>>>>>>> env=None, >>>>>>>> capture_stdout=False, skip_output=False, cwd=None, >>>>>>>> runas=None, timeout=None, suplementary_groups=[], >>>>>>>> capture_stderr=False, encode_stdout=False, >>>>>>>> encode_stderr=False, encoding=None): >>>>>>>> >>>>>>>> i.e. nothing is captured by default, when stdout or stderr are >>>>>>>> captured >>>>>>>> they are returned as bytes by default, when stdout or stderr are >>>>>>>> returned as text they are decoded using the locale encoding by >>>>>>>> default, >>>>>>>> or the encoding can be explicitly specified. >>>>>>>> >>>>>>>> Do you think this is feasible? >>>>>>> >>>>>>> Feasible, yes. >>>>>>> In the majority of cases where the output is needed, we want text >>>>>>> encoded with locale.getpreferredencoding(), so I'd rather make >>>>>>> that the >>>>>>> default rather than bytes. >>>>>>> >>>>>>> Or we could make "encode_stdout" imply "capture_stdout", so you >>>>>>> wouldn't >>>>>>> have to always specify both. (It would be better named >>>>>>> "encoded_stdout" >>>>>>> in that case.) >>>>>> >>>>>> I think it should be OK to decode by default. (IMO it would be best to >>>>>> reverse the logic and name it "raw_stdout", with False default. Do we >>>>>> need "raw_stderr" at all? I don't think we do.) >>>>> >>>>> Actually, now that I think about it: if the result was a namedtuple >>>>> subclass, we can always set extra raw_stdout/raw_stderr attributes on >>>>> it. (Unless skip_output=True, of course.) >>>>> >>>>> result = run(['generate_binary_data']) >>>>> use(result.raw_stdout) >>>>> >>>>> # and for backcompat, >>>>> rc, _out, _err = result >>>> >>>> Good idea. >>>> >>>> Perhaps we should use the same names as CalledProcessError for the >>>> attributes, i.e. "returncode", "output", and "error_output", >>>> "raw_output", "raw_error_output" for the new attributes? >> >> (Actually, since "returncode" is not "return_code", it should probably >> be "returncode", "output", "erroroutput", "raw_output", "raw_erroroutput".) >> >>> >>> OK. It's also good to use different names than Popen's stdout/stderr, >>> which are file-like objects rather than strings. But then, >>> capture_stdout/capture_stderr should be >>> capture_output/capture_error_output. > > Here is the new patch. > I also added output_log and error_log to the result to abstract the > following common pattern: > > result = run([...]) > if result.returncode != 0: > stderr = result.error_output > if six.PY3: > error = stderr.encode(locale.getpreferredencoding(), 'replace') > self.log.critical('...: %s', stderr) 1) certutil -L -r outputs raw DER cert, so this will explode in Python 3 when pem == False: else: args.append('-r') try: - cert, err, returncode = self.run_certutil(args) + result = self.run_certutil(args, capture_output=True) except ipautil.CalledProcessError: raise RuntimeError("Failed to get %s" % nickname) - return cert + return result.output def has_nickname(self, nickname): try: 2) There are some PEP8 transgressions: $ git show -U0 | pep8 --diff ./ipaplatform/base/services.py:297:22: E127 continuation line over-indented for visual indent ./ipapython/ipautil.py:279:1: E302 expected 2 blank lines, found 1 ./ipapython/ipautil.py:280:40: E128 continuation line under-indented for visual indent ./ipapython/ipautil.py:283:1: E302 expected 2 blank lines, found 1 ./ipapython/ipautil.py:330:80: E501 line too long (80 > 79 characters) ./ipapython/ipautil.py:437:43: E127 continuation line over-indented for visual indent ./ipapython/ipautil.py:442:43: E127 continuation line over-indented for visual indent ./ipapython/kernel_keyring.py:57:11: E225 missing whitespace around operator ./ipaserver/install/cainstance.py:626:80: E501 line too long (80 > 79 characters) ./ipaserver/install/ipa_backup.py:498:17: E128 continuation line under-indented for visual indent ./ipaserver/install/ipa_backup.py:521:21: E122 continuation line missing indentation or outdented ./ipaserver/install/ipa_backup.py:530:17: E122 continuation line missing indentation or outdented ./ipaserver/install/ipa_backup.py:605:52: E225 missing whitespace around operator ./ipaserver/install/ipa_backup.py:606:17: E122 continuation line missing indentation or outdented ./ipaserver/install/krbinstance.py:303:80: E501 line too long (84 > 79 characters) ./ipatests/test_ipapython/test_keyring.py:97:33: E222 multiple spaces after operator 3) The patch needs a rebase. I took the liberty of fixing all of the above, see the attached patch. ACK from me. If you are OK with the changes, I will push the patch. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0748.5-Refactor-ipautil.run.patch Type: text/x-patch Size: 69718 bytes Desc: not available URL: From jcholast at redhat.com Mon Dec 14 09:30:54 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 14 Dec 2015 10:30:54 +0100 Subject: [Freeipa-devel] [PATCHES 523-525] replica install: add remote connection check over API In-Reply-To: <566E8B73.3050300@redhat.com> References: <5667EE2C.2050100@redhat.com> <56683238.3050700@redhat.com> <56684B03.2020905@redhat.com> <56693C49.6030508@redhat.com> <566992B1.6030702@redhat.com> <566A6875.1070300@redhat.com> <566A755D.6000701@redhat.com> <566AE079.2030501@redhat.com> <566AFBE1.2070406@redhat.com> <566B0C98.8090108@redhat.com> <566E6056.2030007@redhat.com> <566E8B73.3050300@redhat.com> Message-ID: <566E8C4E.6050406@redhat.com> On 14.12.2015 10:27, Martin Basti wrote: > > > On 14.12.2015 07:23, Jan Cholasta wrote: >> On 11.12.2015 18:49, Tomas Babej wrote: >>> >>> >>> On 12/11/2015 05:37 PM, Martin Basti wrote: >>>> >>>> >>>> On 11.12.2015 15:40, Jan Cholasta wrote: >>>>> On 11.12.2015 08:03, Jan Cholasta wrote: >>>>>> On 11.12.2015 07:08, Jan Cholasta wrote: >>>>>>> On 10.12.2015 15:56, Martin Babinsky wrote: >>>>>>>> On 12/10/2015 09:48 AM, Jan Cholasta wrote: >>>>>>>>> On 9.12.2015 16:38, Jan Cholasta wrote: >>>>>>>>>> On 9.12.2015 14:52, Jan Cholasta wrote: >>>>>>>>>>> On 9.12.2015 10:02, Jan Cholasta wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> the attached patches fix >>>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>>> Note that this needs selinux-policy fix to work, so put SELinux >>>>>>>>>>> into >>>>>>>>>>> permissive mode for testing: >>>>>>>>>>> . >>>>>>>>>> >>>>>>>>>> Updated patches attached. >>>>>>>>> >>>>>>>>> I screwed up a change in patch 524 and accidentally included a >>>>>>>>> chunk of >>>>>>>>> code in patch 525 that doesn't belong in it. >>>>>>>>> >>>>>>>>> Updated patches attached. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> Patches work as expected and I was not able to find any functional >>>>>>>> problem. >>>>>>>> >>>>>>>> I have a question about the naming of the oddjob helper script: the >>>>>>>> one >>>>>>>> related to trusts is named 'com.redhat.idm.trust-fetch-domains', >>>>>>>> and the >>>>>>>> conncheck runner is named 'org.freeipa.server.conncheck'. I >>>>>>>> don't want >>>>>>>> to start another bikeshedding conversation but shouldn't we >>>>>>>> named them >>>>>>>> in a consistent fashion (either rename the first one in separate >>>>>>>> patch >>>>>>>> or rename the new helper to com.redhat.idm.server.conncheck)? >>>>>>>> >>>>>>>> I understand that as an upstream, we should go with the >>>>>>>> 'org.freeipa.*' >>>>>>>> convention, but having two helpers with different prefixes makes me >>>>>>>> sad. >>>>>>> >>>>>>> If you look at the larger picture, org.freeipa is the consistent >>>>>>> name. >>>>>>> It makes me sad as well, but mistakes should be corrected. This is >>>>>>> similar to how we use PEP8 in new code, but do not fix it in old >>>>>>> code >>>>>>> just for the sake of fixing it. >>>>>>> >>>>>>>> >>>>>>>> That is a nitpick though, it does not affect the overall >>>>>>>> functionality >>>>>>>> of the patches so ACK. >>>>>>> >>>>>>> Thanks for the review. The current patch 523 breaks the trusts >>>>>>> oddjob >>>>>>> with SELinux in enforcing mode, I will send an update which corrects >>>>>>> that, until bug 1289930 is fixed. >>>>>> >>>>>> Updated patches attached. >>>>> >>>>> Rebased on top of current master. >>>>> >>>>> >>>>> >>>> Just question, should be any kinited user allowed to run conncheck >>>> via rpc? >>>> >>>> Martin^2 >>> >>> I guess there's is little harm, any kinited user that was allowed to >>> access the machine could perform the conncheck even without these >>> patches: >> >> In the RPC check, the user must have the Replication Administrators >> privilege, which by default only admins have. > > I tried to install replica with a regular user and conncheck passed. > Martin^2 See patch 525. -- Jan Cholasta From pviktori at redhat.com Mon Dec 14 09:49:49 2015 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 14 Dec 2015 10:49:49 +0100 Subject: [Freeipa-devel] [PATCH] 0748 Handle encoding for ipautil.run In-Reply-To: <566E8BE3.7030007@redhat.com> References: <5638B8F9.6010703@redhat.com> <564D0FD9.4070505@redhat.com> <5652B5AC.2040508@redhat.com> <5652E174.50301@redhat.com> <56548E73.1090105@redhat.com> <565587AE.2070303@redhat.com> <5655CA08.8090701@redhat.com> <565C01FD.7050906@redhat.com> <565D83CC.8020802@redhat.com> <565EA4FA.3020307@redhat.com> <565EF7B9.6020507@redhat.com> <566011A5.4040200@redhat.com> <566054D7.9020008@redhat.com> <566145FE.3090005@redhat.com> <566162B2.9010809@redhat.com> <566175D0.1000105@redhat.com> <56617C48.4030005@redhat.com> <56617DBB.9000109@redhat.com> <56680AD9.7060703@redhat.com> <566E8BE3.7030007@redhat.com> Message-ID: <566E90BD.5000207@redhat.com> On 12/14/2015 10:29 AM, Jan Cholasta wrote: > On 9.12.2015 12:04, Petr Viktorin wrote: >> On 12/04/2015 12:49 PM, Jan Cholasta wrote: >>> On 4.12.2015 12:43, Petr Viktorin wrote: >>>> On 12/04/2015 12:15 PM, Jan Cholasta wrote: >>>>> On 4.12.2015 10:53, Petr Viktorin wrote: >>>>>> On 12/04/2015 08:51 AM, Jan Cholasta wrote: >>>>>>> On 3.12.2015 15:42, Petr Viktorin wrote: >>>>>>>> On 12/03/2015 10:55 AM, Jan Cholasta wrote: >>>> [...] >>>>>>>>> If we do encode/decode in run(), I think there should be a way to >>>>>>>>> override the default encoding. My idea was to either accept/return >>>>>>>>> only >>>>>>>>> bytes and let the caller handle encoding themselves, or to handle >>>>>>>>> encoding in run(), but be able to override the defaults. >>>>>>>>> >>>>>>>>> Given we handle encoding in run(), I would imagine it like this: >>>>>>>>> >>>>>>>>> def run(args, stdin=None, raiseonerr=True, nolog=(), >>>>>>>>> env=None, >>>>>>>>> capture_stdout=False, skip_output=False, cwd=None, >>>>>>>>> runas=None, timeout=None, suplementary_groups=[], >>>>>>>>> capture_stderr=False, encode_stdout=False, >>>>>>>>> encode_stderr=False, encoding=None): >>>>>>>>> >>>>>>>>> i.e. nothing is captured by default, when stdout or stderr are >>>>>>>>> captured >>>>>>>>> they are returned as bytes by default, when stdout or stderr are >>>>>>>>> returned as text they are decoded using the locale encoding by >>>>>>>>> default, >>>>>>>>> or the encoding can be explicitly specified. >>>>>>>>> >>>>>>>>> Do you think this is feasible? >>>>>>>> >>>>>>>> Feasible, yes. >>>>>>>> In the majority of cases where the output is needed, we want text >>>>>>>> encoded with locale.getpreferredencoding(), so I'd rather make >>>>>>>> that the >>>>>>>> default rather than bytes. >>>>>>>> >>>>>>>> Or we could make "encode_stdout" imply "capture_stdout", so you >>>>>>>> wouldn't >>>>>>>> have to always specify both. (It would be better named >>>>>>>> "encoded_stdout" >>>>>>>> in that case.) >>>>>>> >>>>>>> I think it should be OK to decode by default. (IMO it would be >>>>>>> best to >>>>>>> reverse the logic and name it "raw_stdout", with False default. >>>>>>> Do we >>>>>>> need "raw_stderr" at all? I don't think we do.) >>>>>> >>>>>> Actually, now that I think about it: if the result was a namedtuple >>>>>> subclass, we can always set extra raw_stdout/raw_stderr attributes on >>>>>> it. (Unless skip_output=True, of course.) >>>>>> >>>>>> result = run(['generate_binary_data']) >>>>>> use(result.raw_stdout) >>>>>> >>>>>> # and for backcompat, >>>>>> rc, _out, _err = result >>>>> >>>>> Good idea. >>>>> >>>>> Perhaps we should use the same names as CalledProcessError for the >>>>> attributes, i.e. "returncode", "output", and "error_output", >>>>> "raw_output", "raw_error_output" for the new attributes? >>> >>> (Actually, since "returncode" is not "return_code", it should probably >>> be "returncode", "output", "erroroutput", "raw_output", >>> "raw_erroroutput".) >>> >>>> >>>> OK. It's also good to use different names than Popen's stdout/stderr, >>>> which are file-like objects rather than strings. But then, >>>> capture_stdout/capture_stderr should be >>>> capture_output/capture_error_output. >> >> Here is the new patch. >> I also added output_log and error_log to the result to abstract the >> following common pattern: >> >> result = run([...]) >> if result.returncode != 0: >> stderr = result.error_output >> if six.PY3: >> error = stderr.encode(locale.getpreferredencoding(), >> 'replace') >> self.log.critical('...: %s', stderr) > > 1) certutil -L -r outputs raw DER cert, so this will explode in Python 3 > when pem == False: > > else: > args.append('-r') > try: > - cert, err, returncode = self.run_certutil(args) > + result = self.run_certutil(args, capture_output=True) > except ipautil.CalledProcessError: > raise RuntimeError("Failed to get %s" % nickname) > - return cert > + return result.output > > def has_nickname(self, nickname): > try: > > > 2) There are some PEP8 transgressions: > > $ git show -U0 | pep8 --diff > ./ipaplatform/base/services.py:297:22: E127 continuation line > over-indented for visual indent > ./ipapython/ipautil.py:279:1: E302 expected 2 blank lines, found 1 > ./ipapython/ipautil.py:280:40: E128 continuation line under-indented for > visual indent > ./ipapython/ipautil.py:283:1: E302 expected 2 blank lines, found 1 > ./ipapython/ipautil.py:330:80: E501 line too long (80 > 79 characters) > ./ipapython/ipautil.py:437:43: E127 continuation line over-indented for > visual indent > ./ipapython/ipautil.py:442:43: E127 continuation line over-indented for > visual indent > ./ipapython/kernel_keyring.py:57:11: E225 missing whitespace around > operator > ./ipaserver/install/cainstance.py:626:80: E501 line too long (80 > 79 > characters) > ./ipaserver/install/ipa_backup.py:498:17: E128 continuation line > under-indented for visual indent > ./ipaserver/install/ipa_backup.py:521:21: E122 continuation line missing > indentation or outdented > ./ipaserver/install/ipa_backup.py:530:17: E122 continuation line missing > indentation or outdented > ./ipaserver/install/ipa_backup.py:605:52: E225 missing whitespace around > operator > ./ipaserver/install/ipa_backup.py:606:17: E122 continuation line missing > indentation or outdented > ./ipaserver/install/krbinstance.py:303:80: E501 line too long (84 > 79 > characters) > ./ipatests/test_ipapython/test_keyring.py:97:33: E222 multiple spaces > after operator > > > 3) The patch needs a rebase. > > > I took the liberty of fixing all of the above, see the attached patch. > > ACK from me. If you are OK with the changes, I will push the patch. I am, ACK. Thank you! -- Petr Viktorin From jcholast at redhat.com Mon Dec 14 09:54:51 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 14 Dec 2015 10:54:51 +0100 Subject: [Freeipa-devel] [PATCH] 0748 Handle encoding for ipautil.run In-Reply-To: <566E90BD.5000207@redhat.com> References: <5638B8F9.6010703@redhat.com> <564D0FD9.4070505@redhat.com> <5652B5AC.2040508@redhat.com> <5652E174.50301@redhat.com> <56548E73.1090105@redhat.com> <565587AE.2070303@redhat.com> <5655CA08.8090701@redhat.com> <565C01FD.7050906@redhat.com> <565D83CC.8020802@redhat.com> <565EA4FA.3020307@redhat.com> <565EF7B9.6020507@redhat.com> <566011A5.4040200@redhat.com> <566054D7.9020008@redhat.com> <566145FE.3090005@redhat.com> <566162B2.9010809@redhat.com> <566175D0.1000105@redhat.com> <56617C48.4030005@redhat.com> <56617DBB.9000109@redhat.com> <56680AD9.7060703@redhat.com> <566E8BE3.7030007@redhat.com> <566E90BD.5000207@redhat.com> Message-ID: <566E91EB.30504@redhat.com> On 14.12.2015 10:49, Petr Viktorin wrote: > On 12/14/2015 10:29 AM, Jan Cholasta wrote: >> On 9.12.2015 12:04, Petr Viktorin wrote: >>> On 12/04/2015 12:49 PM, Jan Cholasta wrote: >>>> On 4.12.2015 12:43, Petr Viktorin wrote: >>>>> On 12/04/2015 12:15 PM, Jan Cholasta wrote: >>>>>> On 4.12.2015 10:53, Petr Viktorin wrote: >>>>>>> On 12/04/2015 08:51 AM, Jan Cholasta wrote: >>>>>>>> On 3.12.2015 15:42, Petr Viktorin wrote: >>>>>>>>> On 12/03/2015 10:55 AM, Jan Cholasta wrote: >>>>> [...] >>>>>>>>>> If we do encode/decode in run(), I think there should be a way to >>>>>>>>>> override the default encoding. My idea was to either accept/return >>>>>>>>>> only >>>>>>>>>> bytes and let the caller handle encoding themselves, or to handle >>>>>>>>>> encoding in run(), but be able to override the defaults. >>>>>>>>>> >>>>>>>>>> Given we handle encoding in run(), I would imagine it like this: >>>>>>>>>> >>>>>>>>>> def run(args, stdin=None, raiseonerr=True, nolog=(), >>>>>>>>>> env=None, >>>>>>>>>> capture_stdout=False, skip_output=False, cwd=None, >>>>>>>>>> runas=None, timeout=None, suplementary_groups=[], >>>>>>>>>> capture_stderr=False, encode_stdout=False, >>>>>>>>>> encode_stderr=False, encoding=None): >>>>>>>>>> >>>>>>>>>> i.e. nothing is captured by default, when stdout or stderr are >>>>>>>>>> captured >>>>>>>>>> they are returned as bytes by default, when stdout or stderr are >>>>>>>>>> returned as text they are decoded using the locale encoding by >>>>>>>>>> default, >>>>>>>>>> or the encoding can be explicitly specified. >>>>>>>>>> >>>>>>>>>> Do you think this is feasible? >>>>>>>>> >>>>>>>>> Feasible, yes. >>>>>>>>> In the majority of cases where the output is needed, we want text >>>>>>>>> encoded with locale.getpreferredencoding(), so I'd rather make >>>>>>>>> that the >>>>>>>>> default rather than bytes. >>>>>>>>> >>>>>>>>> Or we could make "encode_stdout" imply "capture_stdout", so you >>>>>>>>> wouldn't >>>>>>>>> have to always specify both. (It would be better named >>>>>>>>> "encoded_stdout" >>>>>>>>> in that case.) >>>>>>>> >>>>>>>> I think it should be OK to decode by default. (IMO it would be >>>>>>>> best to >>>>>>>> reverse the logic and name it "raw_stdout", with False default. >>>>>>>> Do we >>>>>>>> need "raw_stderr" at all? I don't think we do.) >>>>>>> >>>>>>> Actually, now that I think about it: if the result was a namedtuple >>>>>>> subclass, we can always set extra raw_stdout/raw_stderr attributes on >>>>>>> it. (Unless skip_output=True, of course.) >>>>>>> >>>>>>> result = run(['generate_binary_data']) >>>>>>> use(result.raw_stdout) >>>>>>> >>>>>>> # and for backcompat, >>>>>>> rc, _out, _err = result >>>>>> >>>>>> Good idea. >>>>>> >>>>>> Perhaps we should use the same names as CalledProcessError for the >>>>>> attributes, i.e. "returncode", "output", and "error_output", >>>>>> "raw_output", "raw_error_output" for the new attributes? >>>> >>>> (Actually, since "returncode" is not "return_code", it should probably >>>> be "returncode", "output", "erroroutput", "raw_output", >>>> "raw_erroroutput".) >>>> >>>>> >>>>> OK. It's also good to use different names than Popen's stdout/stderr, >>>>> which are file-like objects rather than strings. But then, >>>>> capture_stdout/capture_stderr should be >>>>> capture_output/capture_error_output. >>> >>> Here is the new patch. >>> I also added output_log and error_log to the result to abstract the >>> following common pattern: >>> >>> result = run([...]) >>> if result.returncode != 0: >>> stderr = result.error_output >>> if six.PY3: >>> error = stderr.encode(locale.getpreferredencoding(), >>> 'replace') >>> self.log.critical('...: %s', stderr) >> >> 1) certutil -L -r outputs raw DER cert, so this will explode in Python 3 >> when pem == False: >> >> else: >> args.append('-r') >> try: >> - cert, err, returncode = self.run_certutil(args) >> + result = self.run_certutil(args, capture_output=True) >> except ipautil.CalledProcessError: >> raise RuntimeError("Failed to get %s" % nickname) >> - return cert >> + return result.output >> >> def has_nickname(self, nickname): >> try: >> >> >> 2) There are some PEP8 transgressions: >> >> $ git show -U0 | pep8 --diff >> ./ipaplatform/base/services.py:297:22: E127 continuation line >> over-indented for visual indent >> ./ipapython/ipautil.py:279:1: E302 expected 2 blank lines, found 1 >> ./ipapython/ipautil.py:280:40: E128 continuation line under-indented for >> visual indent >> ./ipapython/ipautil.py:283:1: E302 expected 2 blank lines, found 1 >> ./ipapython/ipautil.py:330:80: E501 line too long (80 > 79 characters) >> ./ipapython/ipautil.py:437:43: E127 continuation line over-indented for >> visual indent >> ./ipapython/ipautil.py:442:43: E127 continuation line over-indented for >> visual indent >> ./ipapython/kernel_keyring.py:57:11: E225 missing whitespace around >> operator >> ./ipaserver/install/cainstance.py:626:80: E501 line too long (80 > 79 >> characters) >> ./ipaserver/install/ipa_backup.py:498:17: E128 continuation line >> under-indented for visual indent >> ./ipaserver/install/ipa_backup.py:521:21: E122 continuation line missing >> indentation or outdented >> ./ipaserver/install/ipa_backup.py:530:17: E122 continuation line missing >> indentation or outdented >> ./ipaserver/install/ipa_backup.py:605:52: E225 missing whitespace around >> operator >> ./ipaserver/install/ipa_backup.py:606:17: E122 continuation line missing >> indentation or outdented >> ./ipaserver/install/krbinstance.py:303:80: E501 line too long (84 > 79 >> characters) >> ./ipatests/test_ipapython/test_keyring.py:97:33: E222 multiple spaces >> after operator >> >> >> 3) The patch needs a rebase. >> >> >> I took the liberty of fixing all of the above, see the attached patch. >> >> ACK from me. If you are OK with the changes, I will push the patch. > > I am, ACK. Thank you! Pushed to master: 099cf98307d4b2f0ace5d5e28754f264808bf59d -- Jan Cholasta From dkupka at redhat.com Mon Dec 14 10:00:08 2015 From: dkupka at redhat.com (David Kupka) Date: Mon, 14 Dec 2015 11:00:08 +0100 Subject: [Freeipa-devel] [PATCH 0376] KRA: add RA cert during replica promotion In-Reply-To: <5669C702.4080204@redhat.com> References: <5669C702.4080204@redhat.com> Message-ID: <566E9328.8080005@redhat.com> On 10/12/15 19:40, Martin Basti wrote: > https://fedorahosted.org/freeipa/ticket/5512 > > patch attached. > > Hi, thanks for the patch. It works but only when WAIT_AFTER_ARCHIVE is raised. Patch attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0073.0-test-Temporarily-increase-timeout-in-vault-test.patch Type: text/x-patch Size: 944 bytes Desc: not available URL: From dkupka at redhat.com Mon Dec 14 10:46:11 2015 From: dkupka at redhat.com (David Kupka) Date: Mon, 14 Dec 2015 11:46:11 +0100 Subject: [Freeipa-devel] [PATCH 0376] KRA: add RA cert during replica promotion In-Reply-To: <566E9328.8080005@redhat.com> References: <5669C702.4080204@redhat.com> <566E9328.8080005@redhat.com> Message-ID: <566E9DF3.2010805@redhat.com> On 14/12/15 11:00, David Kupka wrote: > On 10/12/15 19:40, Martin Basti wrote: >> https://fedorahosted.org/freeipa/ticket/5512 >> >> patch attached. >> >> > Hi, > thanks for the patch. It works but only when WAIT_AFTER_ARCHIVE is raised. > Patch attached. IOW, your patch works for me, ACK. To let tests pass (and eventually fall on other issue when it appears) please include my patch. -- David Kupka From mbasti at redhat.com Mon Dec 14 10:53:07 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 14 Dec 2015 11:53:07 +0100 Subject: [Freeipa-devel] [PATCH 0376] KRA: add RA cert during replica promotion In-Reply-To: <566E9DF3.2010805@redhat.com> References: <5669C702.4080204@redhat.com> <566E9328.8080005@redhat.com> <566E9DF3.2010805@redhat.com> Message-ID: <566E9F93.6030902@redhat.com> On 14.12.2015 11:46, David Kupka wrote: > On 14/12/15 11:00, David Kupka wrote: >> On 10/12/15 19:40, Martin Basti wrote: >>> https://fedorahosted.org/freeipa/ticket/5512 >>> >>> patch attached. >>> >>> >> Hi, >> thanks for the patch. It works but only when WAIT_AFTER_ARCHIVE is >> raised. >> Patch attached. > IOW, your patch works for me, ACK. To let tests pass (and eventually > fall on other issue when it appears) please include my patch. > Pushed to master: bf9a34f4cfc2c514ff53efea4ba56e2c0cb3033f I also pushed your workaround for tests. Pushed to: master: 8112ac69ccf56dd98c5eb6e77ea131b4665bd1cf ipa-4-2: ac0999ede0d665dee75446f660301fd53e69177b Martin^2 From mbasti at redhat.com Mon Dec 14 11:24:31 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 14 Dec 2015 12:24:31 +0100 Subject: [Freeipa-devel] [PATCH 0378] Tests: fix always true assertion Message-ID: <566EA6EF.4080701@redhat.com> Fixes: /usr/lib/python2.7/site-packages/ipatests/test_cmdline/test_ipagetkeytab.py:116: SyntaxWarning: assertion is always true, perhaps remove parentheses? Patch attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0378-Tests-test_ipagetkeytab-fix-assert-that-is-always-tr.patch Type: text/x-patch Size: 1382 bytes Desc: not available URL: From tbabej at redhat.com Mon Dec 14 11:34:09 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 14 Dec 2015 12:34:09 +0100 Subject: [Freeipa-devel] [PATCH 0070] Makefile: disable parallel build In-Reply-To: <566A8AD6.7070102@redhat.com> References: <566A8AD6.7070102@redhat.com> Message-ID: <566EA931.8000103@redhat.com> On 12/11/2015 09:35 AM, Petr Spacek wrote: > Hello, > > Makefile: disable parallel build > > IPA build system cannot cope with parallel build anyway, so this patch > disables parallel build explicitly so it does not blow up when user > has -j specified in default MAKEOPTS. > > > ACK. Pushed to: master: e650e5eda161a912d22ee2e6ebaf94df3a07d7ff ipa-4-2: 58453bc25092d93f79aca8fa2bd3cc3a10bc6810 From jcholast at redhat.com Mon Dec 14 11:52:25 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 14 Dec 2015 12:52:25 +0100 Subject: [Freeipa-devel] [PATCHES 531-532] server install: redirect ipa-client-install output to standard output Message-ID: <566EAD79.7070700@redhat.com> Hi, the attached patches fix . Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-531-ipautil-allow-redirecting-command-output-to-standard.patch Type: text/x-patch Size: 2510 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-532-server-install-redirect-ipa-client-install-output-to.patch Type: text/x-patch Size: 3443 bytes Desc: not available URL: From tbabej at redhat.com Mon Dec 14 11:54:47 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 14 Dec 2015 12:54:47 +0100 Subject: [Freeipa-devel] [PATCHES 531-532] server install: redirect ipa-client-install output to standard output In-Reply-To: <566EAD79.7070700@redhat.com> References: <566EAD79.7070700@redhat.com> Message-ID: <566EAE07.10300@redhat.com> On 12/14/2015 12:52 PM, Jan Cholasta wrote: > Hi, > > the attached patches fix . > > Honza > > > Shouldn't skip_output be also marked as incompatible with redirect_output? Tomas From jcholast at redhat.com Mon Dec 14 11:57:35 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 14 Dec 2015 12:57:35 +0100 Subject: [Freeipa-devel] [PATCHES 531-532] server install: redirect ipa-client-install output to standard output In-Reply-To: <566EAE07.10300@redhat.com> References: <566EAD79.7070700@redhat.com> <566EAE07.10300@redhat.com> Message-ID: <566EAEAF.2030000@redhat.com> On 14.12.2015 12:54, Tomas Babej wrote: > > > On 12/14/2015 12:52 PM, Jan Cholasta wrote: >> Hi, >> >> the attached patches fix . >> >> Honza >> >> >> > > Shouldn't skip_output be also marked as incompatible with redirect_output? Yes, it should. -- Jan Cholasta From dkupka at redhat.com Mon Dec 14 12:26:34 2015 From: dkupka at redhat.com (David Kupka) Date: Mon, 14 Dec 2015 13:26:34 +0100 Subject: [Freeipa-devel] [PATCH 0074] spec file: Add dbus-python to BuildRequires Message-ID: <566EB57A.3000000@redhat.com> During work on ticket #5497 [0] the need for dbus-python in build time was introduced but it was not added in spec file. [0] https://fedorahosted.org/freeipa/ticket/5497 -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0074.0-spec-file-Add-dbus-python-to-BuildRequires.patch Type: text/x-patch Size: 768 bytes Desc: not available URL: From jcholast at redhat.com Mon Dec 14 12:39:28 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 14 Dec 2015 13:39:28 +0100 Subject: [Freeipa-devel] [PATCH 0074] spec file: Add dbus-python to BuildRequires In-Reply-To: <566EB57A.3000000@redhat.com> References: <566EB57A.3000000@redhat.com> Message-ID: <566EB880.8030108@redhat.com> On 14.12.2015 13:26, David Kupka wrote: > During work on ticket #5497 [0] the need for dbus-python in build time > was introduced but it was not added in spec file. > > [0] https://fedorahosted.org/freeipa/ticket/5497 Thanks. Pushed to master under the one-liner rule: 8b1002a18c42e8be45f0e03e7d3526179ef5b648 Honza -- Jan Cholasta From jcholast at redhat.com Mon Dec 14 12:41:35 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 14 Dec 2015 13:41:35 +0100 Subject: [Freeipa-devel] [PATCH 533] replica promotion: notify user about ignoring client enrollment options Message-ID: <566EB8FF.7050100@redhat.com> Hi, the attached patch fixes . Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-533-replica-promotion-notify-user-about-ignoring-client-.patch Type: text/x-patch Size: 1264 bytes Desc: not available URL: From jcholast at redhat.com Mon Dec 14 13:02:20 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 14 Dec 2015 14:02:20 +0100 Subject: [Freeipa-devel] [PATCH 533] replica promotion: notify user about ignoring client enrollment options In-Reply-To: <566EB8FF.7050100@redhat.com> References: <566EB8FF.7050100@redhat.com> Message-ID: <566EBDDC.1030100@redhat.com> On 14.12.2015 13:41, Jan Cholasta wrote: > Hi, > > the attached patch fixes . Self-NACK, updated patch attached. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-533.1-replica-promotion-notify-user-about-ignoring-client-.patch Type: text/x-patch Size: 1356 bytes Desc: not available URL: From ofayans at redhat.com Mon Dec 14 13:08:35 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Mon, 14 Dec 2015 14:08:35 +0100 Subject: [Freeipa-devel] [PATCH 0018] Fixed install_ca and install_kra failures at domain level 0 In-Reply-To: <566B00CE.4040408@redhat.com> References: <566AF99A.4060604@redhat.com> <566B00CE.4040408@redhat.com> Message-ID: <566EBF53.3030608@redhat.com> Hi Martin, On 12/11/2015 05:58 PM, Martin Basti wrote: > > > On 11.12.2015 17:28, Oleg Fayans wrote: >> + myre = re.compile(".*Backed up to (?P.*?)\n.*") > > IMO this regexp is not good. > > 1) > please name it better than "myre" Done > > 2) > initial '.*' is not needed because regexp does not start with '^' and > you use search() later > > 3) > > trailing '.*' is not needed as well, because it does not end with '$' > > 4) > You can use re.MULTILINE that will parse string per lines > > path_re = re.compile("^Backed up to (?P.*)$", re.MULTILINE) Used it, thanks! > > 5) > + matched = myre.search(result.stdout_text + result.stderr_text) > Why do you need search in both stderr and stdout? Because of this bug: https://fedorahosted.org/freeipa/ticket/5484 > > Martin^2 > > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0018.1-Fixed-install_ca-and-install_kra-domainlevel-0.patch Type: text/x-patch Size: 2604 bytes Desc: not available URL: From tbabej at redhat.com Mon Dec 14 13:19:51 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 14 Dec 2015 14:19:51 +0100 Subject: [Freeipa-devel] [PATCH 533] replica promotion: notify user about ignoring client enrollment options In-Reply-To: <566EBDDC.1030100@redhat.com> References: <566EB8FF.7050100@redhat.com> <566EBDDC.1030100@redhat.com> Message-ID: <566EC1F7.3060508@redhat.com> On 12/14/2015 02:02 PM, Jan Cholasta wrote: > On 14.12.2015 13:41, Jan Cholasta wrote: >> Hi, >> >> the attached patch fixes . > > Self-NACK, updated patch attached. > > > ACK, works fine. From tbabej at redhat.com Mon Dec 14 13:20:50 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 14 Dec 2015 14:20:50 +0100 Subject: [Freeipa-devel] [PATCHES 531-532] server install: redirect ipa-client-install output to standard output In-Reply-To: <566EAEAF.2030000@redhat.com> References: <566EAD79.7070700@redhat.com> <566EAE07.10300@redhat.com> <566EAEAF.2030000@redhat.com> Message-ID: <566EC232.3040708@redhat.com> On 12/14/2015 12:57 PM, Jan Cholasta wrote: > On 14.12.2015 12:54, Tomas Babej wrote: >> >> >> On 12/14/2015 12:52 PM, Jan Cholasta wrote: >>> Hi, >>> >>> the attached patches fix . >>> >>> Honza >>> >>> >>> >> >> Shouldn't skip_output be also marked as incompatible with >> redirect_output? > > Yes, it should. > Otherwise the patch works fine, so conditional ACK here given that the additional sanity check is implemented. Tomas From tbabej at redhat.com Mon Dec 14 13:23:16 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 14 Dec 2015 14:23:16 +0100 Subject: [Freeipa-devel] [PATCH 0378] Tests: fix always true assertion In-Reply-To: <566EA6EF.4080701@redhat.com> References: <566EA6EF.4080701@redhat.com> Message-ID: <566EC2C4.2080903@redhat.com> On 12/14/2015 12:24 PM, Martin Basti wrote: > Fixes: > /usr/lib/python2.7/site-packages/ipatests/test_cmdline/test_ipagetkeytab.py:116: > SyntaxWarning: assertion is always true, perhaps remove parentheses? > > Patch attached. > > Nice catch. ACK. Pushed to master: e1cb802d15d07f80b4812c51e25eb4f7ea48d7c5 From tbabej at redhat.com Mon Dec 14 13:24:07 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 14 Dec 2015 14:24:07 +0100 Subject: [Freeipa-devel] [PATCH 533] replica promotion: notify user about ignoring client enrollment options In-Reply-To: <566EC1F7.3060508@redhat.com> References: <566EB8FF.7050100@redhat.com> <566EBDDC.1030100@redhat.com> <566EC1F7.3060508@redhat.com> Message-ID: <566EC2F7.1000506@redhat.com> On 12/14/2015 02:19 PM, Tomas Babej wrote: > > > On 12/14/2015 02:02 PM, Jan Cholasta wrote: >> On 14.12.2015 13:41, Jan Cholasta wrote: >>> Hi, >>> >>> the attached patch fixes . >> >> Self-NACK, updated patch attached. >> >> >> > > ACK, works fine. > Pushed to master: d68613194b4a26239ebb689d7270cf6c2035afbd From tbabej at redhat.com Mon Dec 14 13:29:48 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 14 Dec 2015 14:29:48 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <566E8A1F.70702@redhat.com> References: <56687174.8000305@redhat.com> <20151210081357.GB21550@mail.corp.redhat.com> <56695291.5090705@redhat.com> <566990E3.2080903@redhat.com> <5669A3D3.7050608@redhat.com> <566A8B22.5020101@redhat.com> <566B0F94.5000007@redhat.com> <566E5FFA.5050705@redhat.com> <566E7CC6.1010201@redhat.com> <566E8A1F.70702@redhat.com> Message-ID: <566EC44C.8010905@redhat.com> On 12/14/2015 10:21 AM, Martin Basti wrote: > > > On 14.12.2015 09:24, Martin Kosek wrote: >> On 12/14/2015 07:21 AM, Jan Cholasta wrote: >>> On 11.12.2015 19:01, Tomas Babej wrote: >>>> >>>> On 12/11/2015 09:36 AM, Martin Kosek wrote: >>>>> On 12/10/2015 05:09 PM, Martin Basti wrote: >>>>>> >>>>>> On 10.12.2015 15:49, Tomas Babej wrote: >>>>>>> On 12/10/2015 11:23 AM, Martin Basti wrote: >>>>>>>> On 10.12.2015 09:13, Lukas Slebodnik wrote: >>>>>>>>> On (09/12/15 19:22), Martin Basti wrote: >>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>>>>> >>>>>>>>>> Patch attached. >>>>>>>>> >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 00:00:00 >>>>>>>>> 2001 >>>>>>>>>> From: Martin Basti >>>>>>>>>> Date: Wed, 9 Dec 2015 18:53:35 +0100 >>>>>>>>>> Subject: [PATCH] Fix version comparison >>>>>>>>>> >>>>>>>>>> Use RPM library to compare vendor versions of IPA for redhat >>>>>>>>>> platform >>>>>>>>>> >>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>>>>> --- >>>>>>>>>> freeipa.spec.in | 2 ++ >>>>>>>>>> ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ >>>>>>>>>> 2 files changed, 21 insertions(+) >>>>>>>>>> >>>>>>>>>> diff --git a/freeipa.spec.in b/freeipa.spec.in >>>>>>>>>> index >>>>>>>>>> 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 100644 >>>>>>>>>> --- a/freeipa.spec.in >>>>>>>>>> +++ b/freeipa.spec.in >>>>>>>>>> @@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} >>>>>>>>>> Requires: gzip >>>>>>>>>> Requires: python-gssapi >= 1.1.0 >>>>>>>>>> Requires: custodia >>>>>>>>>> +Requires: rpm-python >>>>>>>>>> +Requires: rpmdevtools >>>>>>>>> Could you explain why do you need the 2nd package? >>>>>>>>> It does not contains any python modules >>>>>>>>> and I cannot see usage of any binary in this patch >>>>>>>>> >>>>>>>>> LS >>>>>>>> Thanks for this catch, it is actually located in yum package, I >>>>>>>> rather >>>>>>>> copy stringToVersion function from there to IPA, to avoid >>>>>>>> dependency >>>>>>>> hell >>>>>>>> >>>>>>>> Updated patch attached. >>>>>>>> >>>>>>>> >>>>>>> Looking good. The __cmp__ function, however, is not available in >>>>>>> Python >>>>>>> 3. As we will eventually support python3 in RHEL as well, maybe we >>>>>>> should make sure even platform-dependent parts are python3 >>>>>>> compatible? >>>>>>> For the future's sake. >>>>>>> >>>>>>> Tomas >>>>>>> >>>>>> Thanks, >>>>>> >>>>>> python 3 compatible patch attached. >>>>> I wonder why we have to add so much code here and reimplement RPM >>>>> version checking if it is already provided by rpmdevtools: >>>>> >>>>> ~~~ >>>>> $ /usr/bin/rpmdev-vercmp ipa-4.2.0-15.el7 4.2.0-15.el7_2.3; echo $? >>>>> WARNING: hyphen in release1: 4.2.0-15.el7 >>>>> >>>>> rpmdev-vercmp >>>>> rpmdev-vercmp >>>>> rpmdev-vercmp # with no arguments, prompt >>>>> >>>>> Exit status is 0 if the EVR's are equal, 11 if EVR1 is newer, and >>>>> 12 if >>>>> EVR2 >>>>> is newer. Other exit statuses indicate problems. >>>>> >>>>> ipa-4.2.0-15.el7 < 4.2.0-15.el7_2.3 >>>>> 12 >>>>> ~~~ >>>>> >>>>> which could be directly called from __eq__ or __lt__, since we are in >>>>> platform specific code anyway already. >>>>> >>>>> Martin >>>> Imho we should generally prefer reaching out to a Python library rather >>>> launching a subprocess to compare the versions, it's unnecessary >>>> overhead. >> I would not be too worried about miliseconds longer execution on a >> function >> called during RPM upgrade. >> >>>> That said, the main argument for the usage of rpm-python over >>>> rpmdevtools I see is that rpm-python is very likely to be present on >>>> the >>>> system (i.e. it is yum's own dependency), while rpmdevtools will not be >>>> present by default. >>>> >>>> From the standpoint of trying to minimize the size of freeipa >>>> installation (i.e recent wget -> curl migration), we should keep the >>>> spirit here and do not introduce a dependency for a collection of >>>> developer tools. >>>> >>>> /2 cents >>> +1, also a single function is hardly "much code". >> Ok then. If you all want to add yet another N-V-R parsing function in the >> FreeIPA code, I can live with it (with raised eyebrows though). > > Rebased patch attached. I tested the patch, and it works fine - so conditional ACK from me for the current iteration of the patch, given developer consensus which was not reached yet. There's a split of opinions (external binary camp vs. copy&paste camp), so we need to decide if we both camps are OK with proceeding. Tomas From tbabej at redhat.com Mon Dec 14 13:32:57 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 14 Dec 2015 14:32:57 +0100 Subject: [Freeipa-devel] [PATCH] 0047 dogtaginstance: remove unused function 'check_inst' In-Reply-To: <20151214055650.GD23644@dhcp-40-8.bne.redhat.com> References: <20151214055650.GD23644@dhcp-40-8.bne.redhat.com> Message-ID: <566EC509.7020106@redhat.com> On 12/14/2015 06:56 AM, Fraser Tweedale wrote: > Just some drive-by cleanup of an unused function. > > Cheers, > Fraser > ACK, thanks for the cleanup! Pushed to master: 38861428e76c19107a03f07530e3724aee60a270 From jcholast at redhat.com Mon Dec 14 13:41:34 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 14 Dec 2015 14:41:34 +0100 Subject: [Freeipa-devel] [PATCHES 531-532] server install: redirect ipa-client-install output to standard output In-Reply-To: <566EC232.3040708@redhat.com> References: <566EAD79.7070700@redhat.com> <566EAE07.10300@redhat.com> <566EAEAF.2030000@redhat.com> <566EC232.3040708@redhat.com> Message-ID: <566EC70E.9060205@redhat.com> On 14.12.2015 14:20, Tomas Babej wrote: > > > On 12/14/2015 12:57 PM, Jan Cholasta wrote: >> On 14.12.2015 12:54, Tomas Babej wrote: >>> >>> >>> On 12/14/2015 12:52 PM, Jan Cholasta wrote: >>>> Hi, >>>> >>>> the attached patches fix . >>>> >>>> Honza >>>> >>>> >>>> >>> >>> Shouldn't skip_output be also marked as incompatible with >>> redirect_output? >> >> Yes, it should. >> > > Otherwise the patch works fine, so conditional ACK here given that the > additional sanity check is implemented. Updated patches attched. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-531.1-ipautil-allow-redirecting-command-output-to-standard.patch Type: text/x-patch Size: 2634 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-532.1-server-install-redirect-ipa-client-install-output-to.patch Type: text/x-patch Size: 3515 bytes Desc: not available URL: From mbasti at redhat.com Mon Dec 14 13:41:16 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 14 Dec 2015 14:41:16 +0100 Subject: [Freeipa-devel] [PATCHES 529-530] ca install: use host credentials in domain level 1 In-Reply-To: <566E88CA.7060800@redhat.com> References: <56693CF6.8030104@redhat.com> <566AD706.90800@redhat.com> <566AF8A8.4050107@redhat.com> <566E677B.5@redhat.com> <566E88CA.7060800@redhat.com> Message-ID: <566EC6FC.3040509@redhat.com> On 14.12.2015 10:15, Martin Basti wrote: > > > On 14.12.2015 07:53, Jan Cholasta wrote: >> On 11.12.2015 17:24, Martin Basti wrote: >>> >>> >>> On 11.12.2015 15:00, Jan Cholasta wrote: >>>> On 10.12.2015 09:51, Jan Cholasta wrote: >>>>> Hi, >>>>> >>>>> the attached patches fix >>>>> . >>>>> >>>>> My patches 523-525 are required for this: >>>>> . >>>>> >>>>> >>>>> >>>>> >>>>> Honza >>>> >>>> Rebased patches attached. >>>> >>> Patch works for me, but can you provide explanations (and update commit >>> message) why the ACI change is needed: >>> >>> * why it is moved three ACIs from 'cn="$SUFFIX",cn=mapping >>> tree,cn=config' to 'cn=mapping tree,cn=config' >> >> So that they apply to all replication agreements. >> >>> * why you removed completely 'dn: cn=o\3Dipaca,cn=mapping >>> tree,cn=config' >> >> I didn't, they were moved to cn=mapping tree,cn=config as well. >> >> Updated patches attached. >> > ACK > Pushed to master: b248dfda3980244070f85a1968e76d37ad50de9c From tbabej at redhat.com Mon Dec 14 13:47:22 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 14 Dec 2015 14:47:22 +0100 Subject: [Freeipa-devel] [PATCHES 531-532] server install: redirect ipa-client-install output to standard output In-Reply-To: <566EC70E.9060205@redhat.com> References: <566EAD79.7070700@redhat.com> <566EAE07.10300@redhat.com> <566EAEAF.2030000@redhat.com> <566EC232.3040708@redhat.com> <566EC70E.9060205@redhat.com> Message-ID: <566EC86A.8080101@redhat.com> On 12/14/2015 02:41 PM, Jan Cholasta wrote: > On 14.12.2015 14:20, Tomas Babej wrote: >> >> >> On 12/14/2015 12:57 PM, Jan Cholasta wrote: >>> On 14.12.2015 12:54, Tomas Babej wrote: >>>> >>>> >>>> On 12/14/2015 12:52 PM, Jan Cholasta wrote: >>>>> Hi, >>>>> >>>>> the attached patches fix >>>>> . >>>>> >>>>> Honza >>>>> >>>>> >>>>> >>>> >>>> Shouldn't skip_output be also marked as incompatible with >>>> redirect_output? >>> >>> Yes, it should. >>> >> >> Otherwise the patch works fine, so conditional ACK here given that the >> additional sanity check is implemented. > > Updated patches attched. > Thanks, I just removed the empty space in the 'print ()' call before pushing. ACK, Pushed to master: c856401478ce2f4fdd9cd7192afd18704f78e2e6 Tomas From dkupka at redhat.com Mon Dec 14 13:51:24 2015 From: dkupka at redhat.com (David Kupka) Date: Mon, 14 Dec 2015 14:51:24 +0100 Subject: [Freeipa-devel] [PATCH 0365] Remove unused KRA code from ipa-server-install In-Reply-To: <565C6BBC.4050902@redhat.com> References: <565C6BBC.4050902@redhat.com> Message-ID: <566EC95C.2010208@redhat.com> On 30/11/15 16:31, Martin Basti wrote: > First instance of KRA should be installed only by ipa-kra-install > > Patch attached. > > > > Hi, patch works, but I don't like the approach. Do we really want to remove '--setup-kra' option from ipa-server-install? Why do we remove '--setup-kra' while keeping '--setup-dns'? Why do we keep installing PKI CA when it can be also installed later? I would love if 'ipa-server-install' installed only the bare minimum needed and then other features was added using ipa-{feature}-install but also I don't mind one almighty installer that can install all possible combinations of features. But this is neither of it. This just brings another inconsistency into FreeIPA behavior. -- David Kupka From dkupka at redhat.com Mon Dec 14 13:52:51 2015 From: dkupka at redhat.com (David Kupka) Date: Mon, 14 Dec 2015 14:52:51 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <566AD6FF.7020409@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> <5665843C.2090507@redhat.com> <56658C89.8050103@redhat.com> <56668D30.8060308@redhat.com> <5666BCBC.9040005@redhat.com> <5666E822.90103@redhat.com> <56682093.3010505@redhat.com> <56686AFE.4000401@redhat.com> <5669A8D8.5030405@redhat.com> <5669B20E.8000803@redhat.com> <566AB505.3060508@redhat.com> <566AD6FF.7020409@redhat.com> Message-ID: <566EC9B3.6030008@redhat.com> On 11/12/15 15:00, Petr Spacek wrote: > On 11.12.2015 12:35, David Kupka wrote: >> On 10/12/15 18:10, Petr Spacek wrote: >>> On 10.12.2015 17:31, David Kupka wrote: >>>> On 09/12/15 18:55, Petr Spacek wrote: >>>>> On 9.12.2015 13:37, David Kupka wrote: >>>>>> On 08/12/15 15:24, Petr Spacek wrote: >>>>>>> On 8.12.2015 12:19, David Kupka wrote: >>>>>>>> On 08/12/15 08:56, Petr Spacek wrote: >>>>>>>>> On 7.12.2015 14:41, David Kupka wrote: >>>>>>>>>> +def is_host_resolvable(fqdn): >>>>>>>>>> + if not isinstance(fqdn, DNSName): >>>>>>>>>> + fqdn = DNSName(fqdn) >>>>>>>>>> + for rdtype in (rdatatype.A, rdatatype.AAAA): >>>>>>>>>> + try: >>>>>>>>>> + resolver.query(fqdn.make_absolute(), rdtype) >>>>>>>>>> + except DNSException: >>>>>>>>>> + continue >>>>>>>>>> + else: >>>>>>>>>> + return True >>>>>>>>>> + >>>>>>>>>> + return False >>>>>>>>>> >>>>>>>>> >>>>>>>>> NACK, you are re-introducing duplicate function which was removed in >>>>>>>>> 498471e4aed1367b72cd74d15811d0584a6ee268. >>>>>>>>> >>>>>>>>> Please amend the patch ASAP to use new verify_host_resolvable() function >>>>>>>>> so I >>>>>>>>> can test it and get it into 4.3. >>>>>>>>> >>>>>>>> Updated patches attached. >>>>>>> >>>>>>> Hmm, my review script screams: >>>>>>> >>>>>>> Detected base branch: origin/master >>>>>>> Checks will be made against following base commit: 848912a add missing >>>>>>> /ipaplatform/constants.py to .gitignore >>>>>>> Writing API to API.txt >>>>>>> ./ipalib/plugins/dns.py:2127:9: E124 closing bracket does not match visual >>>>>>> indentation >>>>>>> ./ipalib/plugins/dns.py:2711:13: E128 continuation line under-indented for >>>>>>> visual indent >>>>>>> ./ipalib/plugins/dns.py:2713:9: E124 closing bracket does not match visual >>>>>>> indentation >>>>>>> ./ipalib/plugins/dns.py:2716:13: E128 continuation line under-indented for >>>>>>> visual indent >>>>>>> ./ipapython/ipautil.py:928:1: E302 expected 2 blank lines, found 1 >>>>>>> ./ipapython/ipautil.py:948:17: E128 continuation line under-indented for >>>>>>> visual indent >>>>>>> ./ipapython/ipautil.py:956:1: E302 expected 2 blank lines, found 1 >>>>>>> ./ipapython/ipautil.py:997:80: E501 line too long (83 > 79 characters) >>>>>>> ./ipapython/ipautil.py:998:80: E501 line too long (83 > 79 characters) >>>>>>> ./ipaserver/install/bindinstance.py:49:9: E128 continuation line >>>>>>> under-indented for visual indent >>>>>>> ./ipaserver/install/bindinstance.py:292:80: E501 line too long (80 > 79 >>>>>>> characters) >>>>>>> ./ipaserver/install/bindinstance.py:438:19: E128 continuation line >>>>>>> under-indented for visual indent >>>>>>> ./ipaserver/install/server/common.py:271:63: E261 at least two spaces >>>>>>> before >>>>>>> inline comment >>>>>>> ./ipaserver/install/server/common.py:271:80: E501 line too long (90 > 79 >>>>>>> characters) >>>>>>> ERROR: PEP8 --diff failed, continuing ... >>>>>>> ERROR: API.txt was changed without a change in VERSION, continuing ... >>>>>>> Summary of detected errors: >>>>>>> ERROR: PEP8 --diff failed >>>>>>> ERROR: API.txt was changed without a change in VERSION >>>>>>> >>>>>>> Please fix it :-) >>>>>>> >>>>>>> Make make this more pleasant for you, you can use (and review) my attached >>>>>>> patch. It adds 'review' target to Makefile, it will do the same checks as >>>>>>> I do. >>>>>>> >>>>>> >>>>>> Unfortunately your tool does not work for me (output w/ -o xtrace attached). >>>>>> Anyway I have incremented API version and fixed most of the pep8 errors >>>>>> except >>>>>> for E124 twice in ipalib/plugins/dns.py as it seems to be convention in the >>>>>> file and E501 twice in ipapython/ipautil.py because IMO breaking string >>>>>> constant into multiple lines does not help readability. >>>>>> >>>>>> Updated patches also attached. >>>>> >>>>> We are almost there, but NACK for now. >>>>> >>>>> 1) Following attempt to install IPA explodes. Please note that >>>>> dom-245.idm.lab.eng.brq.redhat.com DNS domain is delegated to the IPA server >>>>> before installation is started, so it gives 'timeout' or possibly SERVFAIL. >>>>> >>>>> # ipa-server-install --ds-password=root4lab --admin-password=root4lab >>>>> --setup-dns --forwarder=10.38.5.26 --no-reverse --allow-zone-overlap >>>>> --domain=dom-245.idm.lab.eng.brq.redhat.com >>>>> --realm=DOM-245.IDM.LAB.ENG.BRQ.REDHAT.COM >>>>> [...] >>>>> >>>>> Configuring DNS (named) >>>>> [1/11]: generating rndc key file >>>>> [2/11]: adding DNS container >>>>> [3/11]: setting up our zone >>>>> [error] InvocationError: DNS check for domain >>>>> dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation timed out after >>>>> 30.000453949 seconds. Please make sure that the domain is properly delegated >>>>> to this IPA server. >>>>> ipa.ipapython.install.cli.install_tool(Server): ERROR DNS check for domain >>>>> dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation timed out after >>>>> 30.000453949 seconds. Please make sure that the domain is properly delegated >>>>> to this IPA server. >>>>> ipa.ipapython.install.cli.install_tool(Server): ERROR The >>>>> ipa-server-install command failed. See /var/log/ipaserver-install.log for >>>>> more >>>>> information >>>>> >>>>> 2015-12-09T17:15:37Z DEBUG File >>>>> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in >>>>> execute >>>>> return_value = self.run() >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line >>>>> 318, >>>>> in run >>>>> cfgr.run() >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 310, >>>>> in run >>>>> self.execute() >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 332, >>>>> in execute >>>>> for nothing in self._executor(): >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 372, >>>>> in __runner >>>>> self._handle_exception(exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 394, >>>>> in _handle_exception >>>>> six.reraise(*exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 362, >>>>> in __runner >>>>> step() >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 359, >>>>> in >>>>> step = lambda: next(self.__gen) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>> line 81, >>>>> in run_generator_with_yield_from >>>>> six.reraise(*exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>> line 59, >>>>> in run_generator_with_yield_from >>>>> value = gen.send(prev_value) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 571, >>>>> in _configure >>>>> next(executor) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 372, >>>>> in __runner >>>>> self._handle_exception(exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 449, >>>>> in _handle_exception >>>>> self.__parent._handle_exception(exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 394, >>>>> in _handle_exception >>>>> six.reraise(*exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 446, >>>>> in _handle_exception >>>>> super(ComponentBase, self)._handle_exception(exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 394, >>>>> in _handle_exception >>>>> six.reraise(*exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 362, >>>>> in __runner >>>>> step() >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>> line 359, >>>>> in >>>>> step = lambda: next(self.__gen) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>> line 81, >>>>> in run_generator_with_yield_from >>>>> six.reraise(*exc_info) >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>> line 59, >>>>> in run_generator_with_yield_from >>>>> value = gen.send(prev_value) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line >>>>> 63, in _install >>>>> for nothing in self._installer(self.parent): >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>> line 1471, in main >>>>> install(self) >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>> line 265, in decorated >>>>> func(installer) >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>> line 958, in install >>>>> dns.install(False, False, options) >>>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/dns.py", line >>>>> 322, >>>>> in install >>>>> bind.create_instance() >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>>> line 680, in create_instance >>>>> self.start_creation() >>>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", >>>>> line >>>>> 447, in start_creation >>>>> run_step(full_msg, method) >>>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", >>>>> line >>>>> 437, in run_step >>>>> method() >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>>> line 805, in __setup_zone >>>>> ns_hostname=self.api.env.host, force=True, api=self.api) >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>>> line 331, in add_zone >>>>> force=force) >>>>> File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 447, in >>>>> __call__ >>>>> ret = self.run(*args, **options) >>>>> File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 764, >>>>> in run >>>>> return self.execute(*args, **options) >>>>> File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>>> 2781, in >>>>> execute >>>>> result = super(dnszone_add, self).execute(*keys, **options) >>>>> File "/usr/lib/python2.7/site-packages/ipalib/plugins/baseldap.py", line >>>>> 1233, in execute >>>>> *keys, **options) >>>>> File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>>> 2747, in >>>>> pre_callback >>>>> ldap, dn, entry_attrs, attrs_list, *keys, **options) >>>>> File "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>>> 2153, in >>>>> pre_callback >>>>> raise errors.InvocationError(e.message) >>>>> >>>>> >>>>> 2) Please print 'Checking DNS domain , please wait ...' when validating >>>>> domain parameter in installer. The timeout is 30 seconds and users may be >>>>> nervous when the installed blocks for 30 seconds without a visible reason. >>>>> >>>>> Precedent for this is in check_forwarders() in >>>>> ipaserver/install/bindinstance.py . Encapsulating check_zone_overlap() in >>>>> auxiliary method may be an option. >>>>> >>>>> >>>>> 3) Timeout is a fatal error instead of warning. We have been discussing this >>>>> and it should really be just a warning. >>>>> >>>>> >>>>> 4) Nitpicks are attached in .patch file. >>>>> >>>>> >>>>> 5) ipa dnszone-add checks work as expected, good job! >>>>> >>>>> >>>>> Thank you for patience! >>>>> >>>> Updated patches attached. Added patch introducing --auto-reverse option that >>>> should generate needed reverse zones automatically. >>>> >>> NACK: >>> >>> $ ipa-server-install --setup-dns >>> >>> Do you want to configure the reverse zone? [yes]: >>> ipa.ipapython.install.cli.install_tool(Server): ERROR 'CheckedIPAddress' >>> object has no attribute 'split' >>> >> Updated patches attached. >> > NACK > > 1) It breaks replica installation if replica is second+ DNS server. > > # ipa-replica-install --setup-dns > Password for admin at ABC.IDM.LAB.ENG.BRQ.REDHAT.COM: > Checking DNS domain abc.idm.lab.eng.brq.redhat.com., please wait ... > Your system may be partly configured. > Run /usr/sbin/ipa-server-install --uninstall to clean up. > > ipa.ipapython.install.cli.install_tool(Replica): ERROR DNS zone > abc.idm.lab.eng.brq.redhat.com. already exists in DNS and is handled by > server(s): vm-058-155.abc.idm.lab.eng.brq.redhat.com. > > Maybe the validator should have condition like > "if not replica or not dns_is_configured()" or so ... > > > 2) Check for automatic empty zones does not work: > # ipa dnsforwardzone-add 10.in-addr.arpa. --forwarder=10.34.78.1 > Server will check DNS forwarder(s). > This may take some time, please wait ... > ipa: ERROR: DNS zone 10.in-addr.arpa. already exists in DNS and is handled by > server(s): 10.IN-ADDR.ARPA. > > Here you have to compare names as actual DNS names and not as strings. > > > Have a nice weekend! > Updated patches attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0072.2-dns-Add-auto-reverse-option.patch Type: text/x-patch Size: 6702 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0058.13-dns-do-not-add-forward-zone-if-it-is-already-resolva.patch Type: text/x-patch Size: 12507 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0064.10-dns-Check-if-domain-already-exists.patch Type: text/x-patch Size: 19635 bytes Desc: not available URL: From tbabej at redhat.com Mon Dec 14 13:58:21 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 14 Dec 2015 14:58:21 +0100 Subject: [Freeipa-devel] [PATCH 0374-0375] Fix permissions on newly created directories In-Reply-To: <566B13A7.4090107@redhat.com> References: <566989B2.2080502@redhat.com> <566B13A7.4090107@redhat.com> Message-ID: <566ECAFD.8010108@redhat.com> On 12/11/2015 07:19 PM, Martin Basti wrote: > > > On 10.12.2015 15:18, Martin Basti wrote: >> Hello, >> >> patch 0374 fixes the ticket, but I found more issues with directory >> permission, I fixed them in 0375 >> >> https://fedorahosted.org/freeipa/ticket/5520 >> >> Patches attached. > > Patches attached. ACK, works as expected. Pushed to master: 4272ba40ea909b1f783a6fada5b1eebb6efbdf93 From abokovoy at redhat.com Mon Dec 14 14:05:40 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 14 Dec 2015 16:05:40 +0200 Subject: [Freeipa-devel] [PATCH 0365] Remove unused KRA code from ipa-server-install In-Reply-To: <566EC95C.2010208@redhat.com> References: <565C6BBC.4050902@redhat.com> <566EC95C.2010208@redhat.com> Message-ID: <20151214140540.GT4620@redhat.com> On Mon, 14 Dec 2015, David Kupka wrote: >On 30/11/15 16:31, Martin Basti wrote: >>First instance of KRA should be installed only by ipa-kra-install >> >>Patch attached. >> >> >> >> >Hi, >patch works, but I don't like the approach. > >Do we really want to remove '--setup-kra' option from ipa-server-install? >Why do we remove '--setup-kra' while keeping '--setup-dns'? Why do we >keep installing PKI CA when it can be also installed later? Yes, we do want. Adding the option was an error in the first place. This patch fixes the error. >I would love if 'ipa-server-install' installed only the bare minimum >needed and then other features was added using ipa-{feature}-install >but also I don't mind one almighty installer that can install all >possible combinations of features. >But this is neither of it. This just brings another inconsistency into >FreeIPA behavior. We don't have --with-adtrust either. And we don't want it. -- / Alexander Bokovoy From dkupka at redhat.com Mon Dec 14 14:25:43 2015 From: dkupka at redhat.com (David Kupka) Date: Mon, 14 Dec 2015 15:25:43 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <566EC9B3.6030008@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> <5665843C.2090507@redhat.com> <56658C89.8050103@redhat.com> <56668D30.8060308@redhat.com> <5666BCBC.9040005@redhat.com> <5666E822.90103@redhat.com> <56682093.3010505@redhat.com> <56686AFE.4000401@redhat.com> <5669A8D8.5030405@redhat.com> <5669B20E.8000803@redhat.com> <566AB505.3060508@redhat.com> <566AD6FF.7020409@redhat.com> <566EC9B3.6030008@redhat.com> Message-ID: <566ED167.9060208@redhat.com> On 14/12/15 14:52, David Kupka wrote: > On 11/12/15 15:00, Petr Spacek wrote: >> On 11.12.2015 12:35, David Kupka wrote: >>> On 10/12/15 18:10, Petr Spacek wrote: >>>> On 10.12.2015 17:31, David Kupka wrote: >>>>> On 09/12/15 18:55, Petr Spacek wrote: >>>>>> On 9.12.2015 13:37, David Kupka wrote: >>>>>>> On 08/12/15 15:24, Petr Spacek wrote: >>>>>>>> On 8.12.2015 12:19, David Kupka wrote: >>>>>>>>> On 08/12/15 08:56, Petr Spacek wrote: >>>>>>>>>> On 7.12.2015 14:41, David Kupka wrote: >>>>>>>>>>> +def is_host_resolvable(fqdn): >>>>>>>>>>> + if not isinstance(fqdn, DNSName): >>>>>>>>>>> + fqdn = DNSName(fqdn) >>>>>>>>>>> + for rdtype in (rdatatype.A, rdatatype.AAAA): >>>>>>>>>>> + try: >>>>>>>>>>> + resolver.query(fqdn.make_absolute(), rdtype) >>>>>>>>>>> + except DNSException: >>>>>>>>>>> + continue >>>>>>>>>>> + else: >>>>>>>>>>> + return True >>>>>>>>>>> + >>>>>>>>>>> + return False >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> NACK, you are re-introducing duplicate function which was >>>>>>>>>> removed in >>>>>>>>>> 498471e4aed1367b72cd74d15811d0584a6ee268. >>>>>>>>>> >>>>>>>>>> Please amend the patch ASAP to use new >>>>>>>>>> verify_host_resolvable() function >>>>>>>>>> so I >>>>>>>>>> can test it and get it into 4.3. >>>>>>>>>> >>>>>>>>> Updated patches attached. >>>>>>>> >>>>>>>> Hmm, my review script screams: >>>>>>>> >>>>>>>> Detected base branch: origin/master >>>>>>>> Checks will be made against following base commit: 848912a add >>>>>>>> missing >>>>>>>> /ipaplatform/constants.py to .gitignore >>>>>>>> Writing API to API.txt >>>>>>>> ./ipalib/plugins/dns.py:2127:9: E124 closing bracket does not >>>>>>>> match visual >>>>>>>> indentation >>>>>>>> ./ipalib/plugins/dns.py:2711:13: E128 continuation line >>>>>>>> under-indented for >>>>>>>> visual indent >>>>>>>> ./ipalib/plugins/dns.py:2713:9: E124 closing bracket does not >>>>>>>> match visual >>>>>>>> indentation >>>>>>>> ./ipalib/plugins/dns.py:2716:13: E128 continuation line >>>>>>>> under-indented for >>>>>>>> visual indent >>>>>>>> ./ipapython/ipautil.py:928:1: E302 expected 2 blank lines, found 1 >>>>>>>> ./ipapython/ipautil.py:948:17: E128 continuation line >>>>>>>> under-indented for >>>>>>>> visual indent >>>>>>>> ./ipapython/ipautil.py:956:1: E302 expected 2 blank lines, found 1 >>>>>>>> ./ipapython/ipautil.py:997:80: E501 line too long (83 > 79 >>>>>>>> characters) >>>>>>>> ./ipapython/ipautil.py:998:80: E501 line too long (83 > 79 >>>>>>>> characters) >>>>>>>> ./ipaserver/install/bindinstance.py:49:9: E128 continuation line >>>>>>>> under-indented for visual indent >>>>>>>> ./ipaserver/install/bindinstance.py:292:80: E501 line too long >>>>>>>> (80 > 79 >>>>>>>> characters) >>>>>>>> ./ipaserver/install/bindinstance.py:438:19: E128 continuation line >>>>>>>> under-indented for visual indent >>>>>>>> ./ipaserver/install/server/common.py:271:63: E261 at least two >>>>>>>> spaces >>>>>>>> before >>>>>>>> inline comment >>>>>>>> ./ipaserver/install/server/common.py:271:80: E501 line too long >>>>>>>> (90 > 79 >>>>>>>> characters) >>>>>>>> ERROR: PEP8 --diff failed, continuing ... >>>>>>>> ERROR: API.txt was changed without a change in VERSION, >>>>>>>> continuing ... >>>>>>>> Summary of detected errors: >>>>>>>> ERROR: PEP8 --diff failed >>>>>>>> ERROR: API.txt was changed without a change in VERSION >>>>>>>> >>>>>>>> Please fix it :-) >>>>>>>> >>>>>>>> Make make this more pleasant for you, you can use (and review) >>>>>>>> my attached >>>>>>>> patch. It adds 'review' target to Makefile, it will do the same >>>>>>>> checks as >>>>>>>> I do. >>>>>>>> >>>>>>> >>>>>>> Unfortunately your tool does not work for me (output w/ -o xtrace >>>>>>> attached). >>>>>>> Anyway I have incremented API version and fixed most of the pep8 >>>>>>> errors >>>>>>> except >>>>>>> for E124 twice in ipalib/plugins/dns.py as it seems to be >>>>>>> convention in the >>>>>>> file and E501 twice in ipapython/ipautil.py because IMO breaking >>>>>>> string >>>>>>> constant into multiple lines does not help readability. >>>>>>> >>>>>>> Updated patches also attached. >>>>>> >>>>>> We are almost there, but NACK for now. >>>>>> >>>>>> 1) Following attempt to install IPA explodes. Please note that >>>>>> dom-245.idm.lab.eng.brq.redhat.com DNS domain is delegated to the >>>>>> IPA server >>>>>> before installation is started, so it gives 'timeout' or possibly >>>>>> SERVFAIL. >>>>>> >>>>>> # ipa-server-install --ds-password=root4lab --admin-password=root4lab >>>>>> --setup-dns --forwarder=10.38.5.26 --no-reverse --allow-zone-overlap >>>>>> --domain=dom-245.idm.lab.eng.brq.redhat.com >>>>>> --realm=DOM-245.IDM.LAB.ENG.BRQ.REDHAT.COM >>>>>> [...] >>>>>> >>>>>> Configuring DNS (named) >>>>>> [1/11]: generating rndc key file >>>>>> [2/11]: adding DNS container >>>>>> [3/11]: setting up our zone >>>>>> [error] InvocationError: DNS check for domain >>>>>> dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation >>>>>> timed out after >>>>>> 30.000453949 seconds. Please make sure that the domain is properly >>>>>> delegated >>>>>> to this IPA server. >>>>>> ipa.ipapython.install.cli.install_tool(Server): ERROR DNS check >>>>>> for domain >>>>>> dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation >>>>>> timed out after >>>>>> 30.000453949 seconds. Please make sure that the domain is properly >>>>>> delegated >>>>>> to this IPA server. >>>>>> ipa.ipapython.install.cli.install_tool(Server): ERROR The >>>>>> ipa-server-install command failed. See >>>>>> /var/log/ipaserver-install.log for >>>>>> more >>>>>> information >>>>>> >>>>>> 2015-12-09T17:15:37Z DEBUG File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line >>>>>> 171, in >>>>>> execute >>>>>> return_value = self.run() >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line >>>>>> 318, >>>>>> in run >>>>>> cfgr.run() >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 310, >>>>>> in run >>>>>> self.execute() >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 332, >>>>>> in execute >>>>>> for nothing in self._executor(): >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 372, >>>>>> in __runner >>>>>> self._handle_exception(exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 394, >>>>>> in _handle_exception >>>>>> six.reraise(*exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 362, >>>>>> in __runner >>>>>> step() >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 359, >>>>>> in >>>>>> step = lambda: next(self.__gen) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>> line 81, >>>>>> in run_generator_with_yield_from >>>>>> six.reraise(*exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>> line 59, >>>>>> in run_generator_with_yield_from >>>>>> value = gen.send(prev_value) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 571, >>>>>> in _configure >>>>>> next(executor) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 372, >>>>>> in __runner >>>>>> self._handle_exception(exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 449, >>>>>> in _handle_exception >>>>>> self.__parent._handle_exception(exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 394, >>>>>> in _handle_exception >>>>>> six.reraise(*exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 446, >>>>>> in _handle_exception >>>>>> super(ComponentBase, self)._handle_exception(exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 394, >>>>>> in _handle_exception >>>>>> six.reraise(*exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 362, >>>>>> in __runner >>>>>> step() >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>> line 359, >>>>>> in >>>>>> step = lambda: next(self.__gen) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>> line 81, >>>>>> in run_generator_with_yield_from >>>>>> six.reraise(*exc_info) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>> line 59, >>>>>> in run_generator_with_yield_from >>>>>> value = gen.send(prev_value) >>>>>> >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line >>>>>> 63, in _install >>>>>> for nothing in self._installer(self.parent): >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>> >>>>>> line 1471, in main >>>>>> install(self) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>> >>>>>> line 265, in decorated >>>>>> func(installer) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>> >>>>>> line 958, in install >>>>>> dns.install(False, False, options) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/dns.py", line >>>>>> 322, >>>>>> in install >>>>>> bind.create_instance() >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>>>> line 680, in create_instance >>>>>> self.start_creation() >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", >>>>>> line >>>>>> 447, in start_creation >>>>>> run_step(full_msg, method) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", >>>>>> line >>>>>> 437, in run_step >>>>>> method() >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>>>> line 805, in __setup_zone >>>>>> ns_hostname=self.api.env.host, force=True, api=self.api) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>>>> line 331, in add_zone >>>>>> force=force) >>>>>> File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", >>>>>> line 447, in >>>>>> __call__ >>>>>> ret = self.run(*args, **options) >>>>>> File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", >>>>>> line 764, >>>>>> in run >>>>>> return self.execute(*args, **options) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>>>> 2781, in >>>>>> execute >>>>>> result = super(dnszone_add, self).execute(*keys, **options) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/baseldap.py", line >>>>>> 1233, in execute >>>>>> *keys, **options) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>>>> 2747, in >>>>>> pre_callback >>>>>> ldap, dn, entry_attrs, attrs_list, *keys, **options) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>>>> 2153, in >>>>>> pre_callback >>>>>> raise errors.InvocationError(e.message) >>>>>> >>>>>> >>>>>> 2) Please print 'Checking DNS domain , please wait ...' when >>>>>> validating >>>>>> domain parameter in installer. The timeout is 30 seconds and users >>>>>> may be >>>>>> nervous when the installed blocks for 30 seconds without a visible >>>>>> reason. >>>>>> >>>>>> Precedent for this is in check_forwarders() in >>>>>> ipaserver/install/bindinstance.py . Encapsulating >>>>>> check_zone_overlap() in >>>>>> auxiliary method may be an option. >>>>>> >>>>>> >>>>>> 3) Timeout is a fatal error instead of warning. We have been >>>>>> discussing this >>>>>> and it should really be just a warning. >>>>>> >>>>>> >>>>>> 4) Nitpicks are attached in .patch file. >>>>>> >>>>>> >>>>>> 5) ipa dnszone-add checks work as expected, good job! >>>>>> >>>>>> >>>>>> Thank you for patience! >>>>>> >>>>> Updated patches attached. Added patch introducing --auto-reverse >>>>> option that >>>>> should generate needed reverse zones automatically. >>>>> >>>> NACK: >>>> >>>> $ ipa-server-install --setup-dns >>>> >>>> Do you want to configure the reverse zone? [yes]: >>>> ipa.ipapython.install.cli.install_tool(Server): ERROR >>>> 'CheckedIPAddress' >>>> object has no attribute 'split' >>>> >>> Updated patches attached. >>> >> NACK >> >> 1) It breaks replica installation if replica is second+ DNS server. >> >> # ipa-replica-install --setup-dns >> Password for admin at ABC.IDM.LAB.ENG.BRQ.REDHAT.COM: >> Checking DNS domain abc.idm.lab.eng.brq.redhat.com., please wait ... >> Your system may be partly configured. >> Run /usr/sbin/ipa-server-install --uninstall to clean up. >> >> ipa.ipapython.install.cli.install_tool(Replica): ERROR DNS zone >> abc.idm.lab.eng.brq.redhat.com. already exists in DNS and is handled by >> server(s): vm-058-155.abc.idm.lab.eng.brq.redhat.com. >> >> Maybe the validator should have condition like >> "if not replica or not dns_is_configured()" or so ... >> >> >> 2) Check for automatic empty zones does not work: >> # ipa dnsforwardzone-add 10.in-addr.arpa. --forwarder=10.34.78.1 >> Server will check DNS forwarder(s). >> This may take some time, please wait ... >> ipa: ERROR: DNS zone 10.in-addr.arpa. already exists in DNS and is >> handled by >> server(s): 10.IN-ADDR.ARPA. >> >> Here you have to compare names as actual DNS names and not as strings. >> >> >> Have a nice weekend! >> > Updated patches attached. > As always, FreeIPA's code is really live before release. Rebased patches attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0064.11-dns-Check-if-domain-already-exists.patch Type: text/x-patch Size: 19648 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0058.14-dns-do-not-add-forward-zone-if-it-is-already-resolva.patch Type: text/x-patch Size: 12494 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0072.3-dns-Add-auto-reverse-option.patch Type: text/x-patch Size: 6702 bytes Desc: not available URL: From dkupka at redhat.com Mon Dec 14 14:31:02 2015 From: dkupka at redhat.com (David Kupka) Date: Mon, 14 Dec 2015 15:31:02 +0100 Subject: [Freeipa-devel] [PATCH 0365] Remove unused KRA code from ipa-server-install In-Reply-To: <20151214140540.GT4620@redhat.com> References: <565C6BBC.4050902@redhat.com> <566EC95C.2010208@redhat.com> <20151214140540.GT4620@redhat.com> Message-ID: <566ED2A6.8000202@redhat.com> On 14/12/15 15:05, Alexander Bokovoy wrote: > On Mon, 14 Dec 2015, David Kupka wrote: >> On 30/11/15 16:31, Martin Basti wrote: >>> First instance of KRA should be installed only by ipa-kra-install >>> >>> Patch attached. >>> >>> >>> >>> >> Hi, >> patch works, but I don't like the approach. >> >> Do we really want to remove '--setup-kra' option from ipa-server-install? >> Why do we remove '--setup-kra' while keeping '--setup-dns'? Why do we >> keep installing PKI CA when it can be also installed later? > Yes, we do want. Adding the option was an error in the first place. This > patch fixes the error. > >> I would love if 'ipa-server-install' installed only the bare minimum >> needed and then other features was added using ipa-{feature}-install >> but also I don't mind one almighty installer that can install all >> possible combinations of features. >> But this is neither of it. This just brings another inconsistency into >> FreeIPA behavior. > We don't have --with-adtrust either. And we don't want it. > Ok, then are we aiming for 'ipa-server-install' that only installs the bare minimum and everything else should be installed later? Or, do we decide per feature if it's appropriate to include it in 'ipa-server-install'? What are the criteria in this case? -- David Kupka From jcholast at redhat.com Mon Dec 14 14:37:40 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 14 Dec 2015 15:37:40 +0100 Subject: [Freeipa-devel] [PATCH 534] replica promotion: let ipa-client-install validate enrollment options Message-ID: <566ED434.6050400@redhat.com> Hi, the attached patch fixes . Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-534-replica-promotion-let-ipa-client-install-validate-en.patch Type: text/x-patch Size: 1419 bytes Desc: not available URL: From tbabej at redhat.com Mon Dec 14 14:39:12 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 14 Dec 2015 15:39:12 +0100 Subject: [Freeipa-devel] [PATCH 534] replica promotion: let ipa-client-install validate enrollment options In-Reply-To: <566ED434.6050400@redhat.com> References: <566ED434.6050400@redhat.com> Message-ID: <566ED490.1090802@redhat.com> On 12/14/2015 03:37 PM, Jan Cholasta wrote: > Hi, > > the attached patch fixes . > > Honza > > > ACK, Pushed to master: 110e3dfc5401899ae0a54cc979ca0820e53cfa02 From mkubik at redhat.com Mon Dec 14 14:40:30 2015 From: mkubik at redhat.com (=?UTF-8?Q?Milan_Kub=c3=adk?=) Date: Mon, 14 Dec 2015 15:40:30 +0100 Subject: [Freeipa-devel] [PATCH 0116] CI tests: remove '-p' option from ipa-dns-install calls In-Reply-To: <56699BD1.7080804@redhat.com> References: <56699BD1.7080804@redhat.com> Message-ID: <566ED4DE.9070804@redhat.com> On 12/10/2015 04:35 PM, Martin Babinsky wrote: > See commit message. > Works for me. ACK. -- Milan Kubik From mbasti at redhat.com Mon Dec 14 14:42:57 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 14 Dec 2015 15:42:57 +0100 Subject: [Freeipa-devel] [PATCH 0116] CI tests: remove '-p' option from ipa-dns-install calls In-Reply-To: <566ED4DE.9070804@redhat.com> References: <56699BD1.7080804@redhat.com> <566ED4DE.9070804@redhat.com> Message-ID: <566ED571.3070005@redhat.com> On 14.12.2015 15:40, Milan Kub?k wrote: > On 12/10/2015 04:35 PM, Martin Babinsky wrote: >> See commit message. >> > Works for me. ACK. > > Pushed to master: c4b9b295d8184694c50c0d56051e0273445c98ec From npmccallum at redhat.com Mon Dec 14 15:22:38 2015 From: npmccallum at redhat.com (Nathaniel McCallum) Date: Mon, 14 Dec 2015 10:22:38 -0500 Subject: [Freeipa-devel] [PATCH 0088] Don't error when find_base() fails if a base is not required Message-ID: <1450106558.23069.29.camel@redhat.com> We always have to call find_base() in order to force libldap to open the socket. However, if no base is actually required then there is no reason to error out if find_base() fails. This condition can arise when anonymous binds are disabled. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-npmccallum-0088-Don-t-error-when-find_base-fails-if-a-base-is-not-re.patch Type: text/x-patch Size: 1357 bytes Desc: not available URL: From dkupka at redhat.com Mon Dec 14 15:31:51 2015 From: dkupka at redhat.com (David Kupka) Date: Mon, 14 Dec 2015 16:31:51 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <566ED167.9060208@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> <5665843C.2090507@redhat.com> <56658C89.8050103@redhat.com> <56668D30.8060308@redhat.com> <5666BCBC.9040005@redhat.com> <5666E822.90103@redhat.com> <56682093.3010505@redhat.com> <56686AFE.4000401@redhat.com> <5669A8D8.5030405@redhat.com> <5669B20E.8000803@redhat.com> <566AB505.3060508@redhat.com> <566AD6FF.7020409@redhat.com> <566EC9B3.6030008@redhat.com> <566ED167.9060208@redhat.com> Message-ID: <566EE0E7.6060206@redhat.com> On 14/12/15 15:25, David Kupka wrote: > On 14/12/15 14:52, David Kupka wrote: >> On 11/12/15 15:00, Petr Spacek wrote: >>> On 11.12.2015 12:35, David Kupka wrote: >>>> On 10/12/15 18:10, Petr Spacek wrote: >>>>> On 10.12.2015 17:31, David Kupka wrote: >>>>>> On 09/12/15 18:55, Petr Spacek wrote: >>>>>>> On 9.12.2015 13:37, David Kupka wrote: >>>>>>>> On 08/12/15 15:24, Petr Spacek wrote: >>>>>>>>> On 8.12.2015 12:19, David Kupka wrote: >>>>>>>>>> On 08/12/15 08:56, Petr Spacek wrote: >>>>>>>>>>> On 7.12.2015 14:41, David Kupka wrote: >>>>>>>>>>>> +def is_host_resolvable(fqdn): >>>>>>>>>>>> + if not isinstance(fqdn, DNSName): >>>>>>>>>>>> + fqdn = DNSName(fqdn) >>>>>>>>>>>> + for rdtype in (rdatatype.A, rdatatype.AAAA): >>>>>>>>>>>> + try: >>>>>>>>>>>> + resolver.query(fqdn.make_absolute(), rdtype) >>>>>>>>>>>> + except DNSException: >>>>>>>>>>>> + continue >>>>>>>>>>>> + else: >>>>>>>>>>>> + return True >>>>>>>>>>>> + >>>>>>>>>>>> + return False >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> NACK, you are re-introducing duplicate function which was >>>>>>>>>>> removed in >>>>>>>>>>> 498471e4aed1367b72cd74d15811d0584a6ee268. >>>>>>>>>>> >>>>>>>>>>> Please amend the patch ASAP to use new >>>>>>>>>>> verify_host_resolvable() function >>>>>>>>>>> so I >>>>>>>>>>> can test it and get it into 4.3. >>>>>>>>>>> >>>>>>>>>> Updated patches attached. >>>>>>>>> >>>>>>>>> Hmm, my review script screams: >>>>>>>>> >>>>>>>>> Detected base branch: origin/master >>>>>>>>> Checks will be made against following base commit: 848912a add >>>>>>>>> missing >>>>>>>>> /ipaplatform/constants.py to .gitignore >>>>>>>>> Writing API to API.txt >>>>>>>>> ./ipalib/plugins/dns.py:2127:9: E124 closing bracket does not >>>>>>>>> match visual >>>>>>>>> indentation >>>>>>>>> ./ipalib/plugins/dns.py:2711:13: E128 continuation line >>>>>>>>> under-indented for >>>>>>>>> visual indent >>>>>>>>> ./ipalib/plugins/dns.py:2713:9: E124 closing bracket does not >>>>>>>>> match visual >>>>>>>>> indentation >>>>>>>>> ./ipalib/plugins/dns.py:2716:13: E128 continuation line >>>>>>>>> under-indented for >>>>>>>>> visual indent >>>>>>>>> ./ipapython/ipautil.py:928:1: E302 expected 2 blank lines, found 1 >>>>>>>>> ./ipapython/ipautil.py:948:17: E128 continuation line >>>>>>>>> under-indented for >>>>>>>>> visual indent >>>>>>>>> ./ipapython/ipautil.py:956:1: E302 expected 2 blank lines, found 1 >>>>>>>>> ./ipapython/ipautil.py:997:80: E501 line too long (83 > 79 >>>>>>>>> characters) >>>>>>>>> ./ipapython/ipautil.py:998:80: E501 line too long (83 > 79 >>>>>>>>> characters) >>>>>>>>> ./ipaserver/install/bindinstance.py:49:9: E128 continuation line >>>>>>>>> under-indented for visual indent >>>>>>>>> ./ipaserver/install/bindinstance.py:292:80: E501 line too long >>>>>>>>> (80 > 79 >>>>>>>>> characters) >>>>>>>>> ./ipaserver/install/bindinstance.py:438:19: E128 continuation line >>>>>>>>> under-indented for visual indent >>>>>>>>> ./ipaserver/install/server/common.py:271:63: E261 at least two >>>>>>>>> spaces >>>>>>>>> before >>>>>>>>> inline comment >>>>>>>>> ./ipaserver/install/server/common.py:271:80: E501 line too long >>>>>>>>> (90 > 79 >>>>>>>>> characters) >>>>>>>>> ERROR: PEP8 --diff failed, continuing ... >>>>>>>>> ERROR: API.txt was changed without a change in VERSION, >>>>>>>>> continuing ... >>>>>>>>> Summary of detected errors: >>>>>>>>> ERROR: PEP8 --diff failed >>>>>>>>> ERROR: API.txt was changed without a change in VERSION >>>>>>>>> >>>>>>>>> Please fix it :-) >>>>>>>>> >>>>>>>>> Make make this more pleasant for you, you can use (and review) >>>>>>>>> my attached >>>>>>>>> patch. It adds 'review' target to Makefile, it will do the same >>>>>>>>> checks as >>>>>>>>> I do. >>>>>>>>> >>>>>>>> >>>>>>>> Unfortunately your tool does not work for me (output w/ -o xtrace >>>>>>>> attached). >>>>>>>> Anyway I have incremented API version and fixed most of the pep8 >>>>>>>> errors >>>>>>>> except >>>>>>>> for E124 twice in ipalib/plugins/dns.py as it seems to be >>>>>>>> convention in the >>>>>>>> file and E501 twice in ipapython/ipautil.py because IMO breaking >>>>>>>> string >>>>>>>> constant into multiple lines does not help readability. >>>>>>>> >>>>>>>> Updated patches also attached. >>>>>>> >>>>>>> We are almost there, but NACK for now. >>>>>>> >>>>>>> 1) Following attempt to install IPA explodes. Please note that >>>>>>> dom-245.idm.lab.eng.brq.redhat.com DNS domain is delegated to the >>>>>>> IPA server >>>>>>> before installation is started, so it gives 'timeout' or possibly >>>>>>> SERVFAIL. >>>>>>> >>>>>>> # ipa-server-install --ds-password=root4lab >>>>>>> --admin-password=root4lab >>>>>>> --setup-dns --forwarder=10.38.5.26 --no-reverse --allow-zone-overlap >>>>>>> --domain=dom-245.idm.lab.eng.brq.redhat.com >>>>>>> --realm=DOM-245.IDM.LAB.ENG.BRQ.REDHAT.COM >>>>>>> [...] >>>>>>> >>>>>>> Configuring DNS (named) >>>>>>> [1/11]: generating rndc key file >>>>>>> [2/11]: adding DNS container >>>>>>> [3/11]: setting up our zone >>>>>>> [error] InvocationError: DNS check for domain >>>>>>> dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation >>>>>>> timed out after >>>>>>> 30.000453949 seconds. Please make sure that the domain is properly >>>>>>> delegated >>>>>>> to this IPA server. >>>>>>> ipa.ipapython.install.cli.install_tool(Server): ERROR DNS check >>>>>>> for domain >>>>>>> dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation >>>>>>> timed out after >>>>>>> 30.000453949 seconds. Please make sure that the domain is properly >>>>>>> delegated >>>>>>> to this IPA server. >>>>>>> ipa.ipapython.install.cli.install_tool(Server): ERROR The >>>>>>> ipa-server-install command failed. See >>>>>>> /var/log/ipaserver-install.log for >>>>>>> more >>>>>>> information >>>>>>> >>>>>>> 2015-12-09T17:15:37Z DEBUG File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line >>>>>>> 171, in >>>>>>> execute >>>>>>> return_value = self.run() >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line >>>>>>> 318, >>>>>>> in run >>>>>>> cfgr.run() >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 310, >>>>>>> in run >>>>>>> self.execute() >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 332, >>>>>>> in execute >>>>>>> for nothing in self._executor(): >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 372, >>>>>>> in __runner >>>>>>> self._handle_exception(exc_info) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 394, >>>>>>> in _handle_exception >>>>>>> six.reraise(*exc_info) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 362, >>>>>>> in __runner >>>>>>> step() >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 359, >>>>>>> in >>>>>>> step = lambda: next(self.__gen) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>> line 81, >>>>>>> in run_generator_with_yield_from >>>>>>> six.reraise(*exc_info) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>> line 59, >>>>>>> in run_generator_with_yield_from >>>>>>> value = gen.send(prev_value) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 571, >>>>>>> in _configure >>>>>>> next(executor) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 372, >>>>>>> in __runner >>>>>>> self._handle_exception(exc_info) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 449, >>>>>>> in _handle_exception >>>>>>> self.__parent._handle_exception(exc_info) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 394, >>>>>>> in _handle_exception >>>>>>> six.reraise(*exc_info) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 446, >>>>>>> in _handle_exception >>>>>>> super(ComponentBase, self)._handle_exception(exc_info) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 394, >>>>>>> in _handle_exception >>>>>>> six.reraise(*exc_info) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 362, >>>>>>> in __runner >>>>>>> step() >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>> line 359, >>>>>>> in >>>>>>> step = lambda: next(self.__gen) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>> line 81, >>>>>>> in run_generator_with_yield_from >>>>>>> six.reraise(*exc_info) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>> line 59, >>>>>>> in run_generator_with_yield_from >>>>>>> value = gen.send(prev_value) >>>>>>> >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line >>>>>>> 63, in _install >>>>>>> for nothing in self._installer(self.parent): >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>> >>>>>>> >>>>>>> line 1471, in main >>>>>>> install(self) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>> >>>>>>> >>>>>>> line 265, in decorated >>>>>>> func(installer) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>> >>>>>>> >>>>>>> line 958, in install >>>>>>> dns.install(False, False, options) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/dns.py", line >>>>>>> 322, >>>>>>> in install >>>>>>> bind.create_instance() >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>>>>> >>>>>>> line 680, in create_instance >>>>>>> self.start_creation() >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", >>>>>>> line >>>>>>> 447, in start_creation >>>>>>> run_step(full_msg, method) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", >>>>>>> line >>>>>>> 437, in run_step >>>>>>> method() >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>>>>> >>>>>>> line 805, in __setup_zone >>>>>>> ns_hostname=self.api.env.host, force=True, api=self.api) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>>>>> >>>>>>> line 331, in add_zone >>>>>>> force=force) >>>>>>> File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", >>>>>>> line 447, in >>>>>>> __call__ >>>>>>> ret = self.run(*args, **options) >>>>>>> File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", >>>>>>> line 764, >>>>>>> in run >>>>>>> return self.execute(*args, **options) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>>>>> 2781, in >>>>>>> execute >>>>>>> result = super(dnszone_add, self).execute(*keys, **options) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/baseldap.py", line >>>>>>> 1233, in execute >>>>>>> *keys, **options) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>>>>> 2747, in >>>>>>> pre_callback >>>>>>> ldap, dn, entry_attrs, attrs_list, *keys, **options) >>>>>>> File >>>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>>>>> 2153, in >>>>>>> pre_callback >>>>>>> raise errors.InvocationError(e.message) >>>>>>> >>>>>>> >>>>>>> 2) Please print 'Checking DNS domain , please wait ...' when >>>>>>> validating >>>>>>> domain parameter in installer. The timeout is 30 seconds and users >>>>>>> may be >>>>>>> nervous when the installed blocks for 30 seconds without a visible >>>>>>> reason. >>>>>>> >>>>>>> Precedent for this is in check_forwarders() in >>>>>>> ipaserver/install/bindinstance.py . Encapsulating >>>>>>> check_zone_overlap() in >>>>>>> auxiliary method may be an option. >>>>>>> >>>>>>> >>>>>>> 3) Timeout is a fatal error instead of warning. We have been >>>>>>> discussing this >>>>>>> and it should really be just a warning. >>>>>>> >>>>>>> >>>>>>> 4) Nitpicks are attached in .patch file. >>>>>>> >>>>>>> >>>>>>> 5) ipa dnszone-add checks work as expected, good job! >>>>>>> >>>>>>> >>>>>>> Thank you for patience! >>>>>>> >>>>>> Updated patches attached. Added patch introducing --auto-reverse >>>>>> option that >>>>>> should generate needed reverse zones automatically. >>>>>> >>>>> NACK: >>>>> >>>>> $ ipa-server-install --setup-dns >>>>> >>>>> Do you want to configure the reverse zone? [yes]: >>>>> ipa.ipapython.install.cli.install_tool(Server): ERROR >>>>> 'CheckedIPAddress' >>>>> object has no attribute 'split' >>>>> >>>> Updated patches attached. >>>> >>> NACK >>> >>> 1) It breaks replica installation if replica is second+ DNS server. >>> >>> # ipa-replica-install --setup-dns >>> Password for admin at ABC.IDM.LAB.ENG.BRQ.REDHAT.COM: >>> Checking DNS domain abc.idm.lab.eng.brq.redhat.com., please wait ... >>> Your system may be partly configured. >>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>> >>> ipa.ipapython.install.cli.install_tool(Replica): ERROR DNS zone >>> abc.idm.lab.eng.brq.redhat.com. already exists in DNS and is handled by >>> server(s): vm-058-155.abc.idm.lab.eng.brq.redhat.com. >>> >>> Maybe the validator should have condition like >>> "if not replica or not dns_is_configured()" or so ... >>> >>> >>> 2) Check for automatic empty zones does not work: >>> # ipa dnsforwardzone-add 10.in-addr.arpa. --forwarder=10.34.78.1 >>> Server will check DNS forwarder(s). >>> This may take some time, please wait ... >>> ipa: ERROR: DNS zone 10.in-addr.arpa. already exists in DNS and is >>> handled by >>> server(s): 10.IN-ADDR.ARPA. >>> >>> Here you have to compare names as actual DNS names and not as strings. >>> >>> >>> Have a nice weekend! >>> >> Updated patches attached. >> > As always, FreeIPA's code is really live before release. Rebased patches > attached. > Updated patches attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0072.4-dns-Add-auto-reverse-option.patch Type: text/x-patch Size: 6702 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0058.15-dns-do-not-add-forward-zone-if-it-is-already-resolva.patch Type: text/x-patch Size: 12390 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0064.12-dns-Check-if-domain-already-exists.patch Type: text/x-patch Size: 19528 bytes Desc: not available URL: From pspacek at redhat.com Mon Dec 14 15:54:35 2015 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 14 Dec 2015 16:54:35 +0100 Subject: [Freeipa-devel] [PATCH 0058, 0064] dns: do not add (forward)zone if it is already resolvable. In-Reply-To: <566EE0E7.6060206@redhat.com> References: <55B8DF32.70208@redhat.com> <55BA1ABC.8020108@redhat.com> <55BB5CD3.5060603@redhat.com> <55D58FBA.6070105@redhat.com> <55DB2F73.6050608@redhat.com> <55DC2951.60508@redhat.com> <55DC61EB.4050205@redhat.com> <55DF0117.3010800@redhat.com> <55E015DA.902@redhat.com> <55E047BE.5000507@redhat.com> <55EEF0EE.2020208@redhat.com> <55F01A60.9010506@redhat.com> <5665843C.2090507@redhat.com> <56658C89.8050103@redhat.com> <56668D30.8060308@redhat.com> <5666BCBC.9040005@redhat.com> <5666E822.90103@redhat.com> <56682093.3010505@redhat.com> <56686AFE.4000401@redhat.com> <5669A8D8.5030405@redhat.com> <5669B20E.8000803@redhat.com> <566AB505.3060508@redhat.com> <566AD6FF.7020409@redhat.com> <566EC9B3.6030008@redhat.com> <566ED167.9060208@redhat.com> <566EE0E7.6060206@redhat.com> Message-ID: <566EE63B.3020403@redhat.com> On 14.12.2015 16:31, David Kupka wrote: > On 14/12/15 15:25, David Kupka wrote: >> On 14/12/15 14:52, David Kupka wrote: >>> On 11/12/15 15:00, Petr Spacek wrote: >>>> On 11.12.2015 12:35, David Kupka wrote: >>>>> On 10/12/15 18:10, Petr Spacek wrote: >>>>>> On 10.12.2015 17:31, David Kupka wrote: >>>>>>> On 09/12/15 18:55, Petr Spacek wrote: >>>>>>>> On 9.12.2015 13:37, David Kupka wrote: >>>>>>>>> On 08/12/15 15:24, Petr Spacek wrote: >>>>>>>>>> On 8.12.2015 12:19, David Kupka wrote: >>>>>>>>>>> On 08/12/15 08:56, Petr Spacek wrote: >>>>>>>>>>>> On 7.12.2015 14:41, David Kupka wrote: >>>>>>>>>>>>> +def is_host_resolvable(fqdn): >>>>>>>>>>>>> + if not isinstance(fqdn, DNSName): >>>>>>>>>>>>> + fqdn = DNSName(fqdn) >>>>>>>>>>>>> + for rdtype in (rdatatype.A, rdatatype.AAAA): >>>>>>>>>>>>> + try: >>>>>>>>>>>>> + resolver.query(fqdn.make_absolute(), rdtype) >>>>>>>>>>>>> + except DNSException: >>>>>>>>>>>>> + continue >>>>>>>>>>>>> + else: >>>>>>>>>>>>> + return True >>>>>>>>>>>>> + >>>>>>>>>>>>> + return False >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> NACK, you are re-introducing duplicate function which was >>>>>>>>>>>> removed in >>>>>>>>>>>> 498471e4aed1367b72cd74d15811d0584a6ee268. >>>>>>>>>>>> >>>>>>>>>>>> Please amend the patch ASAP to use new >>>>>>>>>>>> verify_host_resolvable() function >>>>>>>>>>>> so I >>>>>>>>>>>> can test it and get it into 4.3. >>>>>>>>>>>> >>>>>>>>>>> Updated patches attached. >>>>>>>>>> >>>>>>>>>> Hmm, my review script screams: >>>>>>>>>> >>>>>>>>>> Detected base branch: origin/master >>>>>>>>>> Checks will be made against following base commit: 848912a add >>>>>>>>>> missing >>>>>>>>>> /ipaplatform/constants.py to .gitignore >>>>>>>>>> Writing API to API.txt >>>>>>>>>> ./ipalib/plugins/dns.py:2127:9: E124 closing bracket does not >>>>>>>>>> match visual >>>>>>>>>> indentation >>>>>>>>>> ./ipalib/plugins/dns.py:2711:13: E128 continuation line >>>>>>>>>> under-indented for >>>>>>>>>> visual indent >>>>>>>>>> ./ipalib/plugins/dns.py:2713:9: E124 closing bracket does not >>>>>>>>>> match visual >>>>>>>>>> indentation >>>>>>>>>> ./ipalib/plugins/dns.py:2716:13: E128 continuation line >>>>>>>>>> under-indented for >>>>>>>>>> visual indent >>>>>>>>>> ./ipapython/ipautil.py:928:1: E302 expected 2 blank lines, found 1 >>>>>>>>>> ./ipapython/ipautil.py:948:17: E128 continuation line >>>>>>>>>> under-indented for >>>>>>>>>> visual indent >>>>>>>>>> ./ipapython/ipautil.py:956:1: E302 expected 2 blank lines, found 1 >>>>>>>>>> ./ipapython/ipautil.py:997:80: E501 line too long (83 > 79 >>>>>>>>>> characters) >>>>>>>>>> ./ipapython/ipautil.py:998:80: E501 line too long (83 > 79 >>>>>>>>>> characters) >>>>>>>>>> ./ipaserver/install/bindinstance.py:49:9: E128 continuation line >>>>>>>>>> under-indented for visual indent >>>>>>>>>> ./ipaserver/install/bindinstance.py:292:80: E501 line too long >>>>>>>>>> (80 > 79 >>>>>>>>>> characters) >>>>>>>>>> ./ipaserver/install/bindinstance.py:438:19: E128 continuation line >>>>>>>>>> under-indented for visual indent >>>>>>>>>> ./ipaserver/install/server/common.py:271:63: E261 at least two >>>>>>>>>> spaces >>>>>>>>>> before >>>>>>>>>> inline comment >>>>>>>>>> ./ipaserver/install/server/common.py:271:80: E501 line too long >>>>>>>>>> (90 > 79 >>>>>>>>>> characters) >>>>>>>>>> ERROR: PEP8 --diff failed, continuing ... >>>>>>>>>> ERROR: API.txt was changed without a change in VERSION, >>>>>>>>>> continuing ... >>>>>>>>>> Summary of detected errors: >>>>>>>>>> ERROR: PEP8 --diff failed >>>>>>>>>> ERROR: API.txt was changed without a change in VERSION >>>>>>>>>> >>>>>>>>>> Please fix it :-) >>>>>>>>>> >>>>>>>>>> Make make this more pleasant for you, you can use (and review) >>>>>>>>>> my attached >>>>>>>>>> patch. It adds 'review' target to Makefile, it will do the same >>>>>>>>>> checks as >>>>>>>>>> I do. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Unfortunately your tool does not work for me (output w/ -o xtrace >>>>>>>>> attached). >>>>>>>>> Anyway I have incremented API version and fixed most of the pep8 >>>>>>>>> errors >>>>>>>>> except >>>>>>>>> for E124 twice in ipalib/plugins/dns.py as it seems to be >>>>>>>>> convention in the >>>>>>>>> file and E501 twice in ipapython/ipautil.py because IMO breaking >>>>>>>>> string >>>>>>>>> constant into multiple lines does not help readability. >>>>>>>>> >>>>>>>>> Updated patches also attached. >>>>>>>> >>>>>>>> We are almost there, but NACK for now. >>>>>>>> >>>>>>>> 1) Following attempt to install IPA explodes. Please note that >>>>>>>> dom-245.idm.lab.eng.brq.redhat.com DNS domain is delegated to the >>>>>>>> IPA server >>>>>>>> before installation is started, so it gives 'timeout' or possibly >>>>>>>> SERVFAIL. >>>>>>>> >>>>>>>> # ipa-server-install --ds-password=root4lab >>>>>>>> --admin-password=root4lab >>>>>>>> --setup-dns --forwarder=10.38.5.26 --no-reverse --allow-zone-overlap >>>>>>>> --domain=dom-245.idm.lab.eng.brq.redhat.com >>>>>>>> --realm=DOM-245.IDM.LAB.ENG.BRQ.REDHAT.COM >>>>>>>> [...] >>>>>>>> >>>>>>>> Configuring DNS (named) >>>>>>>> [1/11]: generating rndc key file >>>>>>>> [2/11]: adding DNS container >>>>>>>> [3/11]: setting up our zone >>>>>>>> [error] InvocationError: DNS check for domain >>>>>>>> dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation >>>>>>>> timed out after >>>>>>>> 30.000453949 seconds. Please make sure that the domain is properly >>>>>>>> delegated >>>>>>>> to this IPA server. >>>>>>>> ipa.ipapython.install.cli.install_tool(Server): ERROR DNS check >>>>>>>> for domain >>>>>>>> dom-245.idm.lab.eng.brq.redhat.com. failed: The DNS operation >>>>>>>> timed out after >>>>>>>> 30.000453949 seconds. Please make sure that the domain is properly >>>>>>>> delegated >>>>>>>> to this IPA server. >>>>>>>> ipa.ipapython.install.cli.install_tool(Server): ERROR The >>>>>>>> ipa-server-install command failed. See >>>>>>>> /var/log/ipaserver-install.log for >>>>>>>> more >>>>>>>> information >>>>>>>> >>>>>>>> 2015-12-09T17:15:37Z DEBUG File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line >>>>>>>> 171, in >>>>>>>> execute >>>>>>>> return_value = self.run() >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line >>>>>>>> 318, >>>>>>>> in run >>>>>>>> cfgr.run() >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 310, >>>>>>>> in run >>>>>>>> self.execute() >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 332, >>>>>>>> in execute >>>>>>>> for nothing in self._executor(): >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 372, >>>>>>>> in __runner >>>>>>>> self._handle_exception(exc_info) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 394, >>>>>>>> in _handle_exception >>>>>>>> six.reraise(*exc_info) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 362, >>>>>>>> in __runner >>>>>>>> step() >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 359, >>>>>>>> in >>>>>>>> step = lambda: next(self.__gen) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>>> line 81, >>>>>>>> in run_generator_with_yield_from >>>>>>>> six.reraise(*exc_info) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>>> line 59, >>>>>>>> in run_generator_with_yield_from >>>>>>>> value = gen.send(prev_value) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 571, >>>>>>>> in _configure >>>>>>>> next(executor) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 372, >>>>>>>> in __runner >>>>>>>> self._handle_exception(exc_info) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 449, >>>>>>>> in _handle_exception >>>>>>>> self.__parent._handle_exception(exc_info) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 394, >>>>>>>> in _handle_exception >>>>>>>> six.reraise(*exc_info) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 446, >>>>>>>> in _handle_exception >>>>>>>> super(ComponentBase, self)._handle_exception(exc_info) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 394, >>>>>>>> in _handle_exception >>>>>>>> six.reraise(*exc_info) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 362, >>>>>>>> in __runner >>>>>>>> step() >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/core.py", >>>>>>>> line 359, >>>>>>>> in >>>>>>>> step = lambda: next(self.__gen) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>>> line 81, >>>>>>>> in run_generator_with_yield_from >>>>>>>> six.reraise(*exc_info) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/util.py", >>>>>>>> line 59, >>>>>>>> in run_generator_with_yield_from >>>>>>>> value = gen.send(prev_value) >>>>>>>> >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line >>>>>>>> 63, in _install >>>>>>>> for nothing in self._installer(self.parent): >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>>> >>>>>>>> >>>>>>>> line 1471, in main >>>>>>>> install(self) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>>> >>>>>>>> >>>>>>>> line 265, in decorated >>>>>>>> func(installer) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", >>>>>>>> >>>>>>>> >>>>>>>> line 958, in install >>>>>>>> dns.install(False, False, options) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/dns.py", line >>>>>>>> 322, >>>>>>>> in install >>>>>>>> bind.create_instance() >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>>>>>> >>>>>>>> line 680, in create_instance >>>>>>>> self.start_creation() >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", >>>>>>>> line >>>>>>>> 447, in start_creation >>>>>>>> run_step(full_msg, method) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", >>>>>>>> line >>>>>>>> 437, in run_step >>>>>>>> method() >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>>>>>> >>>>>>>> line 805, in __setup_zone >>>>>>>> ns_hostname=self.api.env.host, force=True, api=self.api) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", >>>>>>>> >>>>>>>> line 331, in add_zone >>>>>>>> force=force) >>>>>>>> File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", >>>>>>>> line 447, in >>>>>>>> __call__ >>>>>>>> ret = self.run(*args, **options) >>>>>>>> File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", >>>>>>>> line 764, >>>>>>>> in run >>>>>>>> return self.execute(*args, **options) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>>>>>> 2781, in >>>>>>>> execute >>>>>>>> result = super(dnszone_add, self).execute(*keys, **options) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/baseldap.py", line >>>>>>>> 1233, in execute >>>>>>>> *keys, **options) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>>>>>> 2747, in >>>>>>>> pre_callback >>>>>>>> ldap, dn, entry_attrs, attrs_list, *keys, **options) >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py", line >>>>>>>> 2153, in >>>>>>>> pre_callback >>>>>>>> raise errors.InvocationError(e.message) >>>>>>>> >>>>>>>> >>>>>>>> 2) Please print 'Checking DNS domain , please wait ...' when >>>>>>>> validating >>>>>>>> domain parameter in installer. The timeout is 30 seconds and users >>>>>>>> may be >>>>>>>> nervous when the installed blocks for 30 seconds without a visible >>>>>>>> reason. >>>>>>>> >>>>>>>> Precedent for this is in check_forwarders() in >>>>>>>> ipaserver/install/bindinstance.py . Encapsulating >>>>>>>> check_zone_overlap() in >>>>>>>> auxiliary method may be an option. >>>>>>>> >>>>>>>> >>>>>>>> 3) Timeout is a fatal error instead of warning. We have been >>>>>>>> discussing this >>>>>>>> and it should really be just a warning. >>>>>>>> >>>>>>>> >>>>>>>> 4) Nitpicks are attached in .patch file. >>>>>>>> >>>>>>>> >>>>>>>> 5) ipa dnszone-add checks work as expected, good job! >>>>>>>> >>>>>>>> >>>>>>>> Thank you for patience! >>>>>>>> >>>>>>> Updated patches attached. Added patch introducing --auto-reverse >>>>>>> option that >>>>>>> should generate needed reverse zones automatically. >>>>>>> >>>>>> NACK: >>>>>> >>>>>> $ ipa-server-install --setup-dns >>>>>> >>>>>> Do you want to configure the reverse zone? [yes]: >>>>>> ipa.ipapython.install.cli.install_tool(Server): ERROR >>>>>> 'CheckedIPAddress' >>>>>> object has no attribute 'split' >>>>>> >>>>> Updated patches attached. >>>>> >>>> NACK >>>> >>>> 1) It breaks replica installation if replica is second+ DNS server. >>>> >>>> # ipa-replica-install --setup-dns >>>> Password for admin at ABC.IDM.LAB.ENG.BRQ.REDHAT.COM: >>>> Checking DNS domain abc.idm.lab.eng.brq.redhat.com., please wait ... >>>> Your system may be partly configured. >>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>> >>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR DNS zone >>>> abc.idm.lab.eng.brq.redhat.com. already exists in DNS and is handled by >>>> server(s): vm-058-155.abc.idm.lab.eng.brq.redhat.com. >>>> >>>> Maybe the validator should have condition like >>>> "if not replica or not dns_is_configured()" or so ... >>>> >>>> >>>> 2) Check for automatic empty zones does not work: >>>> # ipa dnsforwardzone-add 10.in-addr.arpa. --forwarder=10.34.78.1 >>>> Server will check DNS forwarder(s). >>>> This may take some time, please wait ... >>>> ipa: ERROR: DNS zone 10.in-addr.arpa. already exists in DNS and is >>>> handled by >>>> server(s): 10.IN-ADDR.ARPA. >>>> >>>> Here you have to compare names as actual DNS names and not as strings. >>>> >>>> >>>> Have a nice weekend! >>>> >>> Updated patches attached. >>> >> As always, FreeIPA's code is really live before release. Rebased patches >> attached. >> > Updated patches attached. ACK! Push it NOW! :-D -- Petr^2 Spacek From abokovoy at redhat.com Mon Dec 14 15:54:42 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 14 Dec 2015 17:54:42 +0200 Subject: [Freeipa-devel] [PATCH 0365] Remove unused KRA code from ipa-server-install In-Reply-To: <566ED2A6.8000202@redhat.com> References: <565C6BBC.4050902@redhat.com> <566EC95C.2010208@redhat.com> <20151214140540.GT4620@redhat.com> <566ED2A6.8000202@redhat.com> Message-ID: <20151214155442.GV4620@redhat.com> On Mon, 14 Dec 2015, David Kupka wrote: >On 14/12/15 15:05, Alexander Bokovoy wrote: >>On Mon, 14 Dec 2015, David Kupka wrote: >>>On 30/11/15 16:31, Martin Basti wrote: >>>>First instance of KRA should be installed only by ipa-kra-install >>>> >>>>Patch attached. >>>> >>>> >>>> >>>> >>>Hi, >>>patch works, but I don't like the approach. >>> >>>Do we really want to remove '--setup-kra' option from ipa-server-install? >>>Why do we remove '--setup-kra' while keeping '--setup-dns'? Why do we >>>keep installing PKI CA when it can be also installed later? >>Yes, we do want. Adding the option was an error in the first place. This >>patch fixes the error. >> >>>I would love if 'ipa-server-install' installed only the bare minimum >>>needed and then other features was added using ipa-{feature}-install >>>but also I don't mind one almighty installer that can install all >>>possible combinations of features. >>>But this is neither of it. This just brings another inconsistency into >>>FreeIPA behavior. >>We don't have --with-adtrust either. And we don't want it. >> >Ok, then are we aiming for 'ipa-server-install' that only installs the >bare minimum and everything else should be installed later? > >Or, do we decide per feature if it's appropriate to include it in >'ipa-server-install'? What are the criteria in this case? Per feature decision is a bit better description of an ad-hoc process we have so far. As we had this discussion with Simo today, let me quickly capture arguments for both. We typically allow options in ipa-server-install for features that can be present and used very early. Certificates are issued early in the process, DNS records are critical to exist before hosts can be added and can start using Kerberos and so on. However, trust to AD cannot be established to 'just deployed' FreeIPA realm, you need to pre-configure your and that remote realm. Certificates were in particular important part of the story before we added support for GSSAPI authentication between replicas (4.3 release is going to have it). This meant we were constrained by the fact that some entity had to issue certificates before we even create a replica. I was arguing that having KRA would require us to have full fledged CA as we depend on ability to issue certificates for KRA feature to be useful as well. While standard CA is somewhat optional in the case only end-point service certs are needed (Let's Encrypt is a solution there), having KRA means use of some CA. So to me having KRA means we have CA already, why not to simply merge the two setting into --setup-ca at all? Separate installers also were used in past as a gap-bridging tool to allow people to upgrade their old deployments and gain new functionality. So separate installer has another function than just deploying a service. Having --with-kra option thus makes less sense. We can consider KRA functionality to be in a default feature set at some point, that would still require us to have a separate installer, ipa-kra-install, to allow extending old deployments to provide new feature. -- / Alexander Bokovoy From mbabinsk at redhat.com Mon Dec 14 16:31:44 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Mon, 14 Dec 2015 17:31:44 +0100 Subject: [Freeipa-devel] [PATCH 0117] ipa-client-install: create a temporary directory for ccache files Message-ID: <566EEEF0.6040800@redhat.com> fixes https://fedorahosted.org/freeipa/ticket/5528 -- Martin^3 Babinsky -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbabinsk-0117-ipa-client-install-create-a-temporary-directory-for-.patch Type: text/x-patch Size: 1895 bytes Desc: not available URL: From tbabej at redhat.com Mon Dec 14 17:51:40 2015 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 14 Dec 2015 18:51:40 +0100 Subject: [Freeipa-devel] [PATCH 0117] ipa-client-install: create a temporary directory for ccache files In-Reply-To: <566EEEF0.6040800@redhat.com> References: <566EEEF0.6040800@redhat.com> Message-ID: <566F01AC.1010307@redhat.com> On 12/14/2015 05:31 PM, Martin Babinsky wrote: > fixes https://fedorahosted.org/freeipa/ticket/5528 Works as expected, code-wise looks good. Thanks for looking into this, ACK! Pushed to master: 5886f87f974fa508047a21350c2e6e75a3001da6 From pvoborni at redhat.com Tue Dec 15 00:05:50 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 15 Dec 2015 01:05:50 +0100 Subject: [Freeipa-devel] Testing FreeIPA 4.3 for GA Message-ID: <566F595E.9070807@redhat.com> Blocking patches for FreeIPA 4.3 were pushed, ipa-4-3 branch was created. Master branch is ready for 4.4 development. A build is available for testing in my pvoborni/freeipa-4-3 COPR repo [1] until the official 4.3 repo is created. The repo contains only this build. The build is not pure upstream ipa-4-3 branch but rather a build which will go to Fedora rawhide and official 4.3 copr repo - Fedora downstream spec with the SELinux workaround applied [2][3]. Known issues: * https://fedorahosted.org/freeipa/ticket/5469 - requires fix in PKI * https://bugzilla.redhat.com/show_bug.cgi?id=1289930 - SELinux Policy update needed for connection check I have started drafting release page [4]. [1] https://copr.fedoraproject.org/coprs/pvoborni/freeipa-4-3/ [2] https://fedorahosted.org/freeipa/ticket/5533 [3] http://www.redhat.com/archives/freeipa-devel/2015-December/msg00234.html [4] http://www.freeipa.org/page/Releases/4.3.0 -- Petr Vobornik From jcholast at redhat.com Tue Dec 15 06:19:01 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 15 Dec 2015 07:19:01 +0100 Subject: [Freeipa-devel] [PATCH 0117] ipa-client-install: create a temporary directory for ccache files In-Reply-To: <566F01AC.1010307@redhat.com> References: <566EEEF0.6040800@redhat.com> <566F01AC.1010307@redhat.com> Message-ID: <566FB0D5.8070306@redhat.com> On 14.12.2015 18:51, Tomas Babej wrote: > > > On 12/14/2015 05:31 PM, Martin Babinsky wrote: >> fixes https://fedorahosted.org/freeipa/ticket/5528 > > Works as expected, code-wise looks good. > > Thanks for looking into this, ACK! > > Pushed to master: 5886f87f974fa508047a21350c2e6e75a3001da6 It would probably be better if ipa-client-install used ipautil.private_ccache(), but I guess we can fix that later, once we start digging into ipa-client-install componentization / rectofaring. -- Jan Cholasta From lslebodn at redhat.com Tue Dec 15 06:24:57 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Tue, 15 Dec 2015 07:24:57 +0100 Subject: [Freeipa-devel] Testing FreeIPA 4.3 for GA In-Reply-To: <566F595E.9070807@redhat.com> References: <566F595E.9070807@redhat.com> Message-ID: <20151215062456.GA14717@mail.corp.redhat.com> On (15/12/15 01:05), Petr Vobornik wrote: >Blocking patches for FreeIPA 4.3 were pushed, ipa-4-3 branch was created. >Master branch is ready for 4.4 development. > >A build is available for testing in my pvoborni/freeipa-4-3 COPR repo [1] >until the official 4.3 repo is created. The repo contains only this build. >The build is not pure upstream ipa-4-3 branch but rather a build which will >go to Fedora rawhide and official 4.3 copr repo - Fedora downstream spec with >the SELinux workaround applied [2][3]. > >Known issues: >* https://fedorahosted.org/freeipa/ticket/5469 - requires fix in PKI >* https://bugzilla.redhat.com/show_bug.cgi?id=1289930 - SELinux Policy update >needed for connection check > I would suggest to do small change in packaging. sh$ rpm -qp --requires ./freeipa-client-4.3.0-1.fc23.x86_64.rpm | grep pam pam_krb5 pam_krb5 is not used by default. I will suggest to use weak dependencies here If you do not want to remove it completely. http://rpm.org/wiki/PackagerDocs/Dependencies#Weakdependencies LS From jcholast at redhat.com Tue Dec 15 06:42:06 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 15 Dec 2015 07:42:06 +0100 Subject: [Freeipa-devel] [PATCH 0365] Remove unused KRA code from ipa-server-install In-Reply-To: <20151214155442.GV4620@redhat.com> References: <565C6BBC.4050902@redhat.com> <566EC95C.2010208@redhat.com> <20151214140540.GT4620@redhat.com> <566ED2A6.8000202@redhat.com> <20151214155442.GV4620@redhat.com> Message-ID: <566FB63E.8080205@redhat.com> On 14.12.2015 16:54, Alexander Bokovoy wrote: > On Mon, 14 Dec 2015, David Kupka wrote: >> On 14/12/15 15:05, Alexander Bokovoy wrote: >>> On Mon, 14 Dec 2015, David Kupka wrote: >>>> On 30/11/15 16:31, Martin Basti wrote: >>>>> First instance of KRA should be installed only by ipa-kra-install >>>>> >>>>> Patch attached. >>>>> >>>>> >>>>> >>>>> >>>> Hi, >>>> patch works, but I don't like the approach. >>>> >>>> Do we really want to remove '--setup-kra' option from >>>> ipa-server-install? >>>> Why do we remove '--setup-kra' while keeping '--setup-dns'? Why do we >>>> keep installing PKI CA when it can be also installed later? >>> Yes, we do want. Adding the option was an error in the first place. This >>> patch fixes the error. >>> >>>> I would love if 'ipa-server-install' installed only the bare minimum >>>> needed and then other features was added using ipa-{feature}-install >>>> but also I don't mind one almighty installer that can install all >>>> possible combinations of features. >>>> But this is neither of it. This just brings another inconsistency into >>>> FreeIPA behavior. >>> We don't have --with-adtrust either. And we don't want it. >>> >> Ok, then are we aiming for 'ipa-server-install' that only installs the >> bare minimum and everything else should be installed later? >> >> Or, do we decide per feature if it's appropriate to include it in >> 'ipa-server-install'? What are the criteria in this case? > Per feature decision is a bit better description of an ad-hoc process we > have so far. > > As we had this discussion with Simo today, let me quickly capture > arguments for both. We typically allow options in ipa-server-install for > features that can be present and used very early. Certificates are > issued early in the process, DNS records are critical to exist before > hosts can be added and can start using Kerberos and so on. However, > trust to AD cannot be established to 'just deployed' FreeIPA realm, you > need to pre-configure your and that remote realm. > > Certificates were in particular important part of the story before we > added support for GSSAPI authentication between replicas (4.3 release is > going to have it). This meant we were constrained by the fact that some > entity had to issue certificates before we even create a replica. > > I was arguing that having KRA would require us to have full fledged CA > as we depend on ability to issue certificates for KRA feature to be > useful as well. While standard CA is somewhat optional in the case only > end-point service certs are needed (Let's Encrypt is a solution there), > having KRA means use of some CA. So to me having KRA means we have CA > already, why not to simply merge the two setting into --setup-ca at all? Because KRA is a separate component that does something completely different than a CA and is configured separately even in Dogtag. The logic here is exactly the reverse - if you want to install KRA, you have to install (Dogtag) CA, not the other way around, so merging both into --setup-ca does not make sense. > > Separate installers also were used in past as a gap-bridging tool to > allow people to upgrade their old deployments and gain new > functionality. So separate installer has another function than just > deploying a service. > > Having --with-kra option thus makes less sense. We can consider KRA > functionality to be in a default feature set at some point, that would > still require us to have a separate installer, ipa-kra-install, to allow > extending old deployments to provide new feature. We can always have both. With the new installer framework it is trivial to fold installers like this without code duplication. It is still work in progress though, so I would prefer not to add --setup-kra to ipa-server-install now (if ever). -- Jan Cholasta From jcholast at redhat.com Tue Dec 15 06:46:34 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 15 Dec 2015 07:46:34 +0100 Subject: [Freeipa-devel] Testing FreeIPA 4.3 for GA In-Reply-To: <20151215062456.GA14717@mail.corp.redhat.com> References: <566F595E.9070807@redhat.com> <20151215062456.GA14717@mail.corp.redhat.com> Message-ID: <566FB74A.4080506@redhat.com> On 15.12.2015 07:24, Lukas Slebodnik wrote: > On (15/12/15 01:05), Petr Vobornik wrote: >> Blocking patches for FreeIPA 4.3 were pushed, ipa-4-3 branch was created. >> Master branch is ready for 4.4 development. >> >> A build is available for testing in my pvoborni/freeipa-4-3 COPR repo [1] >> until the official 4.3 repo is created. The repo contains only this build. >> The build is not pure upstream ipa-4-3 branch but rather a build which will >> go to Fedora rawhide and official 4.3 copr repo - Fedora downstream spec with >> the SELinux workaround applied [2][3]. >> >> Known issues: >> * https://fedorahosted.org/freeipa/ticket/5469 - requires fix in PKI >> * https://bugzilla.redhat.com/show_bug.cgi?id=1289930 - SELinux Policy update >> needed for connection check >> > I would suggest to do small change in packaging. > > sh$ rpm -qp --requires ./freeipa-client-4.3.0-1.fc23.x86_64.rpm | grep pam > pam_krb5 > > pam_krb5 is not used by default. I will suggest to use weak dependencies here > If you do not want to remove it completely. > http://rpm.org/wiki/PackagerDocs/Dependencies#Weakdependencies I would rather remove it completely, along with nss_ldap and pam-nss-ldapd support. -- Jan Cholasta From mbabinsk at redhat.com Tue Dec 15 07:03:25 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Tue, 15 Dec 2015 08:03:25 +0100 Subject: [Freeipa-devel] [PATCH 0117] ipa-client-install: create a temporary directory for ccache files In-Reply-To: <566FB0D5.8070306@redhat.com> References: <566EEEF0.6040800@redhat.com> <566F01AC.1010307@redhat.com> <566FB0D5.8070306@redhat.com> Message-ID: <566FBB3D.1060708@redhat.com> On 12/15/2015 07:19 AM, Jan Cholasta wrote: > On 14.12.2015 18:51, Tomas Babej wrote: >> >> >> On 12/14/2015 05:31 PM, Martin Babinsky wrote: >>> fixes https://fedorahosted.org/freeipa/ticket/5528 >> >> Works as expected, code-wise looks good. >> >> Thanks for looking into this, ACK! >> >> Pushed to master: 5886f87f974fa508047a21350c2e6e75a3001da6 > > It would probably be better if ipa-client-install used > ipautil.private_ccache(), but I guess we can fix that later, once we > start digging into ipa-client-install componentization / rectofaring. > Yes since that would probably require heroic amounts of spaghetti untangling to get it right. Anyway we have any milestone for ipa-client refactoring? Is it a part of 4.4 installer redesign effort? -- Martin^3 Babinsky From lslebodn at redhat.com Tue Dec 15 07:32:56 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Tue, 15 Dec 2015 08:32:56 +0100 Subject: [Freeipa-devel] Testing FreeIPA 4.3 for GA In-Reply-To: <566F595E.9070807@redhat.com> References: <566F595E.9070807@redhat.com> Message-ID: <20151215073255.GA17683@mail.corp.redhat.com> On (15/12/15 01:05), Petr Vobornik wrote: >Blocking patches for FreeIPA 4.3 were pushed, ipa-4-3 branch was created. >Master branch is ready for 4.4 development. > >A build is available for testing in my pvoborni/freeipa-4-3 COPR repo [1] >until the official 4.3 repo is created. The repo contains only this build. >The build is not pure upstream ipa-4-3 branch but rather a build which will >go to Fedora rawhide and official 4.3 copr repo - Fedora downstream spec with >the SELinux workaround applied [2][3]. > >Known issues: >* https://fedorahosted.org/freeipa/ticket/5469 - requires fix in PKI >* https://bugzilla.redhat.com/show_bug.cgi?id=1289930 - SELinux Policy update >needed for connection check > >I have started drafting release page [4]. > It looks like there is a bug in dnf, because it cannot install freeipa-tests package; which was renamed to python2-ipatests. [root at 3a8359f9e1cd ~]# rpm -q freeipa-server freeipa-server-4.3.0-1.fc23.x86_64 [root at 3a8359f9e1cd ~]# dnf install -y --best freeipa-tests Last metadata expiration check performed 0:09:23 ago on Tue Dec 15 07:16:26 2015. Error: package freeipa-tests-4.2.3-1.1.fc23.x86_64 requires freeipa-client = 4.2.3-1.1.fc23, but none of the providers can be installed. package freeipa-tests-4.2.2-1.fc23.x86_64 requires freeipa-python = 4.2.2-1.fc23, but none of the providers can be installed (try to add '--allowerasing' to command line to replace conflicting packages) [root at 3a8359f9e1cd ~]# rpm -qp --provides ./python2-ipatests-4.3.0-1.fc23.noarch.rpm freeipa-tests(x86-64) = 4.3.0-1.fc23 ipa-tests(x86-64) = 4.3.0 python-ipatests = 4.3.0-1.fc23 python2-ipatests = 4.3.0-1.fc23 FYI: It works with yum-deprecated. LS From jcholast at redhat.com Tue Dec 15 07:54:19 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 15 Dec 2015 08:54:19 +0100 Subject: [Freeipa-devel] certmonger everywhere Message-ID: <566FC72B.3050406@redhat.com> Hi, recently I and David discussed the direction of installers with regard to requesting certificates. Currently there are four (!) different ways of requesting certificates in the installer [1][2][3][4]. We would like to reduce it to one. Since all the certificates are tracked by certmonger and certmonger already knows how to request certificates from Dogtag (and other CAs), we believe that all certificates should be requested using certmonger. Taking our meditation further, we thought "Why not use certmonger for the cert-request command as well?" What is the benefit, do you ask? a) single code path for requesting certificates (seriously, the current state is ridiculous) b) use any CA supported by certmonger as the IPA CA (i.e. Let's Encrypt [5], once certmonger gains support for it) c) automate external CA install, using any CA supported by certmonger [6] d) support multiple different CAs at once (generalization of the Sub-CA feature) e) uniform configuration on clients (configure once, use forever, even for CA-less) The idea is to store configuration for the different CAs in LDAP and have cert-request redirect requests to a proper CA helper according to that configuration. This would require a new certmonger D-Bus method to call a CA helper without associated certificate storage, but that should be rather easy to add. In return, it would be possible to do all of the above. Note that this should not conflict with tighter integration with Dogtag (profiles, ACLs, etc.). Comments are welcome. Honza [1] [2] [3] [4] [5] [6] -- Jan Cholasta From dkupka at redhat.com Tue Dec 15 07:57:40 2015 From: dkupka at redhat.com (David Kupka) Date: Tue, 15 Dec 2015 08:57:40 +0100 Subject: [Freeipa-devel] [PATCH 0365] Remove unused KRA code from ipa-server-install In-Reply-To: <20151214155442.GV4620@redhat.com> References: <565C6BBC.4050902@redhat.com> <566EC95C.2010208@redhat.com> <20151214140540.GT4620@redhat.com> <566ED2A6.8000202@redhat.com> <20151214155442.GV4620@redhat.com> Message-ID: <566FC7F4.9060305@redhat.com> On 14/12/15 16:54, Alexander Bokovoy wrote: > On Mon, 14 Dec 2015, David Kupka wrote: >> On 14/12/15 15:05, Alexander Bokovoy wrote: >>> On Mon, 14 Dec 2015, David Kupka wrote: >>>> On 30/11/15 16:31, Martin Basti wrote: >>>>> First instance of KRA should be installed only by ipa-kra-install >>>>> >>>>> Patch attached. >>>>> >>>>> >>>>> >>>>> >>>> Hi, >>>> patch works, but I don't like the approach. >>>> >>>> Do we really want to remove '--setup-kra' option from >>>> ipa-server-install? >>>> Why do we remove '--setup-kra' while keeping '--setup-dns'? Why do we >>>> keep installing PKI CA when it can be also installed later? >>> Yes, we do want. Adding the option was an error in the first place. This >>> patch fixes the error. >>> >>>> I would love if 'ipa-server-install' installed only the bare minimum >>>> needed and then other features was added using ipa-{feature}-install >>>> but also I don't mind one almighty installer that can install all >>>> possible combinations of features. >>>> But this is neither of it. This just brings another inconsistency into >>>> FreeIPA behavior. >>> We don't have --with-adtrust either. And we don't want it. >>> >> Ok, then are we aiming for 'ipa-server-install' that only installs the >> bare minimum and everything else should be installed later? >> >> Or, do we decide per feature if it's appropriate to include it in >> 'ipa-server-install'? What are the criteria in this case? > Per feature decision is a bit better description of an ad-hoc process we > have so far. > > As we had this discussion with Simo today, let me quickly capture > arguments for both. We typically allow options in ipa-server-install for > features that can be present and used very early. Certificates are > issued early in the process, DNS records are critical to exist before > hosts can be added and can start using Kerberos and so on. However, > trust to AD cannot be established to 'just deployed' FreeIPA realm, you > need to pre-configure your and that remote realm. > > Certificates were in particular important part of the story before we > added support for GSSAPI authentication between replicas (4.3 release is > going to have it). This meant we were constrained by the fact that some > entity had to issue certificates before we even create a replica. > > I was arguing that having KRA would require us to have full fledged CA > as we depend on ability to issue certificates for KRA feature to be > useful as well. While standard CA is somewhat optional in the case only > end-point service certs are needed (Let's Encrypt is a solution there), > having KRA means use of some CA. So to me having KRA means we have CA > already, why not to simply merge the two setting into --setup-ca at all? > > Separate installers also were used in past as a gap-bridging tool to > allow people to upgrade their old deployments and gain new > functionality. So separate installer has another function than just > deploying a service. > > Having --with-kra option thus makes less sense. We can consider KRA > functionality to be in a default feature set at some point, that would > still require us to have a separate installer, ipa-kra-install, to allow > extending old deployments to provide new feature. > The fact that the DNS records need to exist for some (core) IPA functionality does not necessary mean that we need DNS server installed as a part of FreeIPA. We even support DNS-less install and user is then responsible for maintaining DNS records in server of his choice. IOW 'ipa-server-install --setup-dns' results in the same as 'ipa-server-install && ipa-dns-install'. In case of trust, would it be really impossible to preconfigure the remote realm and then provide all necessary information to 'ipa-server-install --setup-trust'? Or is it just the way we're doing it right now for some maybe really good reason? So the only reason to add the '--setup-' or '--with-' to ipa-server-install I can see is user comfort and ease of installation. That's good reason for me but in that case I still do not understand why do not include options that can be easily included. -- David Kupka From abokovoy at redhat.com Tue Dec 15 08:18:21 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 15 Dec 2015 10:18:21 +0200 Subject: [Freeipa-devel] [PATCH 0365] Remove unused KRA code from ipa-server-install In-Reply-To: <566FB63E.8080205@redhat.com> References: <565C6BBC.4050902@redhat.com> <566EC95C.2010208@redhat.com> <20151214140540.GT4620@redhat.com> <566ED2A6.8000202@redhat.com> <20151214155442.GV4620@redhat.com> <566FB63E.8080205@redhat.com> Message-ID: <20151215081820.GZ4620@redhat.com> On Tue, 15 Dec 2015, Jan Cholasta wrote: >On 14.12.2015 16:54, Alexander Bokovoy wrote: >>On Mon, 14 Dec 2015, David Kupka wrote: >>>On 14/12/15 15:05, Alexander Bokovoy wrote: >>>>On Mon, 14 Dec 2015, David Kupka wrote: >>>>>On 30/11/15 16:31, Martin Basti wrote: >>>>>>First instance of KRA should be installed only by ipa-kra-install >>>>>> >>>>>>Patch attached. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>Hi, >>>>>patch works, but I don't like the approach. >>>>> >>>>>Do we really want to remove '--setup-kra' option from >>>>>ipa-server-install? >>>>>Why do we remove '--setup-kra' while keeping '--setup-dns'? Why do we >>>>>keep installing PKI CA when it can be also installed later? >>>>Yes, we do want. Adding the option was an error in the first place. This >>>>patch fixes the error. >>>> >>>>>I would love if 'ipa-server-install' installed only the bare minimum >>>>>needed and then other features was added using ipa-{feature}-install >>>>>but also I don't mind one almighty installer that can install all >>>>>possible combinations of features. >>>>>But this is neither of it. This just brings another inconsistency into >>>>>FreeIPA behavior. >>>>We don't have --with-adtrust either. And we don't want it. >>>> >>>Ok, then are we aiming for 'ipa-server-install' that only installs the >>>bare minimum and everything else should be installed later? >>> >>>Or, do we decide per feature if it's appropriate to include it in >>>'ipa-server-install'? What are the criteria in this case? >>Per feature decision is a bit better description of an ad-hoc process we >>have so far. >> >>As we had this discussion with Simo today, let me quickly capture >>arguments for both. We typically allow options in ipa-server-install for >>features that can be present and used very early. Certificates are >>issued early in the process, DNS records are critical to exist before >>hosts can be added and can start using Kerberos and so on. However, >>trust to AD cannot be established to 'just deployed' FreeIPA realm, you >>need to pre-configure your and that remote realm. >> >>Certificates were in particular important part of the story before we >>added support for GSSAPI authentication between replicas (4.3 release is >>going to have it). This meant we were constrained by the fact that some >>entity had to issue certificates before we even create a replica. >> >>I was arguing that having KRA would require us to have full fledged CA >>as we depend on ability to issue certificates for KRA feature to be >>useful as well. While standard CA is somewhat optional in the case only >>end-point service certs are needed (Let's Encrypt is a solution there), >>having KRA means use of some CA. So to me having KRA means we have CA >>already, why not to simply merge the two setting into --setup-ca at all? > >Because KRA is a separate component that does something completely >different than a CA and is configured separately even in Dogtag. The >logic here is exactly the reverse - if you want to install KRA, you >have to install (Dogtag) CA, not the other way around, so merging both >into --setup-ca does not make sense. That's exactly what I said, isn't it: CA is required to be installed before KRA. However, if we would like to have KRA functionality by default every time we have CA (we are not going to have a separate replication topology for KRA, aren't we?), then --setup-ca would cover installing both and --setup-kra would not be needed. >> >>Separate installers also were used in past as a gap-bridging tool to >>allow people to upgrade their old deployments and gain new >>functionality. So separate installer has another function than just >>deploying a service. >> >>Having --with-kra option thus makes less sense. We can consider KRA >>functionality to be in a default feature set at some point, that would >>still require us to have a separate installer, ipa-kra-install, to allow >>extending old deployments to provide new feature. > >We can always have both. With the new installer framework it is >trivial to fold installers like this without code duplication. It is >still work in progress though, so I would prefer not to add >--setup-kra to ipa-server-install now (if ever). Yep. -- / Alexander Bokovoy From abokovoy at redhat.com Tue Dec 15 08:28:04 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 15 Dec 2015 10:28:04 +0200 Subject: [Freeipa-devel] [PATCH 0365] Remove unused KRA code from ipa-server-install In-Reply-To: <566FC7F4.9060305@redhat.com> References: <565C6BBC.4050902@redhat.com> <566EC95C.2010208@redhat.com> <20151214140540.GT4620@redhat.com> <566ED2A6.8000202@redhat.com> <20151214155442.GV4620@redhat.com> <566FC7F4.9060305@redhat.com> Message-ID: <20151215082804.GA4620@redhat.com> On Tue, 15 Dec 2015, David Kupka wrote: >On 14/12/15 16:54, Alexander Bokovoy wrote: >>On Mon, 14 Dec 2015, David Kupka wrote: >>>On 14/12/15 15:05, Alexander Bokovoy wrote: >>>>On Mon, 14 Dec 2015, David Kupka wrote: >>>>>On 30/11/15 16:31, Martin Basti wrote: >>>>>>First instance of KRA should be installed only by ipa-kra-install >>>>>> >>>>>>Patch attached. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>Hi, >>>>>patch works, but I don't like the approach. >>>>> >>>>>Do we really want to remove '--setup-kra' option from >>>>>ipa-server-install? >>>>>Why do we remove '--setup-kra' while keeping '--setup-dns'? Why do we >>>>>keep installing PKI CA when it can be also installed later? >>>>Yes, we do want. Adding the option was an error in the first place. This >>>>patch fixes the error. >>>> >>>>>I would love if 'ipa-server-install' installed only the bare minimum >>>>>needed and then other features was added using ipa-{feature}-install >>>>>but also I don't mind one almighty installer that can install all >>>>>possible combinations of features. >>>>>But this is neither of it. This just brings another inconsistency into >>>>>FreeIPA behavior. >>>>We don't have --with-adtrust either. And we don't want it. >>>> >>>Ok, then are we aiming for 'ipa-server-install' that only installs the >>>bare minimum and everything else should be installed later? >>> >>>Or, do we decide per feature if it's appropriate to include it in >>>'ipa-server-install'? What are the criteria in this case? >>Per feature decision is a bit better description of an ad-hoc process we >>have so far. >> >>As we had this discussion with Simo today, let me quickly capture >>arguments for both. We typically allow options in ipa-server-install for >>features that can be present and used very early. Certificates are >>issued early in the process, DNS records are critical to exist before >>hosts can be added and can start using Kerberos and so on. However, >>trust to AD cannot be established to 'just deployed' FreeIPA realm, you >>need to pre-configure your and that remote realm. >> >>Certificates were in particular important part of the story before we >>added support for GSSAPI authentication between replicas (4.3 release is >>going to have it). This meant we were constrained by the fact that some >>entity had to issue certificates before we even create a replica. >> >>I was arguing that having KRA would require us to have full fledged CA >>as we depend on ability to issue certificates for KRA feature to be >>useful as well. While standard CA is somewhat optional in the case only >>end-point service certs are needed (Let's Encrypt is a solution there), >>having KRA means use of some CA. So to me having KRA means we have CA >>already, why not to simply merge the two setting into --setup-ca at all? >> >>Separate installers also were used in past as a gap-bridging tool to >>allow people to upgrade their old deployments and gain new >>functionality. So separate installer has another function than just >>deploying a service. >> >>Having --with-kra option thus makes less sense. We can consider KRA >>functionality to be in a default feature set at some point, that would >>still require us to have a separate installer, ipa-kra-install, to allow >>extending old deployments to provide new feature. >> > >The fact that the DNS records need to exist for some (core) IPA >functionality does not necessary mean that we need DNS server >installed as a part of FreeIPA. We even support DNS-less install and >user is then responsible for maintaining DNS records in server of his >choice. >IOW 'ipa-server-install --setup-dns' results in the same as >'ipa-server-install && ipa-dns-install'. It means we require DNS is there -- be it internal or as external service provided by something else. There is no DNS-less install in reality, something has to provide SRV records, you cannot have FreeIPA deployment without DNS at all for some use cases -- for example, trust to AD requires DNS infrastructure to exist. The difference we have for DNS and CA/KRA is that we cannot work with any other CA for all certificate types yet and KRA is simply not supporting anything but Dogtag CA on the same IPA master, so --setup-kra is implying --setup-ca, and thus could be simply merged into --setup-ca once we decide KRA functionality is stable enough to be part of default install. >In case of trust, would it be really impossible to preconfigure the >remote realm and then provide all necessary information to >'ipa-server-install --setup-trust'? Or is it just the way we're doing >it right now for some maybe really good reason? It is always possible to merge functionality into a single installer but I'm not sure we'll gain anything reasonable from it in this particular case. Configuring trust is about two things: (a) setup software on IPA master, and (b) create a number of special objects in both IPA and Active Directory. To do (a) you can merge things into the installer but to do (b) you need to perform operations on AD side and which operations you would perform depends on what other choices you did when installed IPA -- if you did not install internal DNS, you would need to create a number of SRV records for IPA DNS zone first somewhere, then gain AD admin credentials and perform trust-add somehow, whether directly or by splitting the procedure into two different steps (shared secret case). It means at some point such an installer would need to stop and wait until someone else would complete required actions outside the scope of IPA realm. As I said, per-feature decisions are what drives our ad-hoc process. :) -- / Alexander Bokovoy From pspacek at redhat.com Tue Dec 15 09:07:44 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 15 Dec 2015 10:07:44 +0100 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <566FC72B.3050406@redhat.com> References: <566FC72B.3050406@redhat.com> Message-ID: <566FD860.2080602@redhat.com> On 15.12.2015 08:54, Jan Cholasta wrote: > Hi, > > recently I and David discussed the direction of installers with regard to > requesting certificates. Currently there are four (!) different ways of > requesting certificates in the installer [1][2][3][4]. We would like to reduce > it to one. > > Since all the certificates are tracked by certmonger and certmonger already > knows how to request certificates from Dogtag (and other CAs), we believe that > all certificates should be requested using certmonger. > > Taking our meditation further, we thought "Why not use certmonger for the > cert-request command as well?" What is the benefit, do you ask? > > a) single code path for requesting certificates (seriously, the current state > is ridiculous) > > b) use any CA supported by certmonger as the IPA CA (i.e. Let's Encrypt [5], > once certmonger gains support for it) > > c) automate external CA install, using any CA supported by certmonger [6] > > d) support multiple different CAs at once (generalization of the Sub-CA feature) > > e) uniform configuration on clients (configure once, use forever, even for > CA-less) > > The idea is to store configuration for the different CAs in LDAP and have > cert-request redirect requests to a proper CA helper according to that > configuration. This would require a new certmonger D-Bus method to call a CA > helper without associated certificate storage, but that should be rather easy > to add. In return, it would be possible to do all of the above. > > Note that this should not conflict with tighter integration with Dogtag > (profiles, ACLs, etc.). > > Comments are welcome. It makes a lot of sense to me! Petr^2 Spacek > Honza > > [1] > > [2] > > > [3] > > > [4] > > > [5] > [6] From ofayans at redhat.com Tue Dec 15 09:29:36 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Tue, 15 Dec 2015 10:29:36 +0100 Subject: [Freeipa-devel] [PATCH 016 - 017] First part of the replica promotion tests + testplan In-Reply-To: <56680DDD.2000106@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> <565DB7D3.5040500@redhat.com> <565F0BDD.8080206@redhat.com> <565F17EE.3020701@redhat.com> <5666DC18.2040906@redhat.com> <56675E31.3050203@redhat.com> <5667F49B.5020709@redhat.com> <5667FEFB.6090904@redhat.com> <56680C2E.7060805@redhat.com> <56680DDD.2000106@redhat.com> Message-ID: <566FDD80.7030805@redhat.com> Hi Martin, The updated patches are attached. Patch 0017 includes all changes from patch 0018, so, if you approve this one, there would be no need to continue with the review of 0018. This one contains all changes related to you remarks from 0018 review. Please see my explanation on the stdout+stderr part in the thread from patch 0018. With these two patches applied one of the tests fails due this bug: https://fedorahosted.org/freeipa/ticket/5550 On 12/09/2015 12:17 PM, Martin Basti wrote: > > > On 09.12.2015 12:10, Martin Basti wrote: >> >> >> On 09.12.2015 11:14, Oleg Fayans wrote: >>> Hi Martin >>> >>> On 12/09/2015 10:30 AM, Martin Basti wrote: >>>> >>>> On 08.12.2015 23:48, Oleg Fayans wrote: >>>>> Substituted a hardcoded suffix name with a constant DOMAIN_SUFFIX_NAME >>>>> >>>>> On 12/08/2015 02:33 PM, Oleg Fayans wrote: >>>>>> Hi all, >>>>>> >>>>>> >>>>>> The patches are rebased against the current master. >>>>>> >>>>>> On 12/02/2015 05:10 PM, Martin Basti wrote: >>>>>>> On 02.12.2015 16:18, Oleg Fayans wrote: >>>>>>>> Hi Martin, >>>>>>>> >>>>>>>> On 12/01/2015 04:08 PM, Martin Basti wrote: >>>>>>>>> On 27.11.2015 16:26, Oleg Fayans wrote: >>>>>>>>>> And patch N 16 passes lint too: >>>>>>>>>> >>>>>>>>>> On 11/27/2015 04:03 PM, Oleg Fayans wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> On 11/27/2015 03:26 PM, Martin Basti wrote: >>>>>>>>>>>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>> >>>>>>>>>>>>> All your suggestions were taken into account. Both patches are >>>>>>>>>>>>> updated. Thank you for your help! >>>>>>>>>>>>> >>>>>>>>>>>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>>>>>>>>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I agree to all your points but one. please, see my comment >>>>>>>>>>>>>>> below >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 0) Note >>>>>>>>>>>>>>>> Please be aware of >>>>>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5469 >>>>>>>>>>>>>>>> during >>>>>>>>>>>>>>>> KRA testing >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, >>>>>>>>>>>>>>>> this may >>>>>>>>>>>>>>>> change >>>>>>>>>>>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for domain >>>>>>>>>>>>>>>> level 0 >>>>>>>>>>>>>>>> and 1 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>>> Why uninstall KRA then server, is not enough just uninstall >>>>>>>>>>>>>>>> server >>>>>>>>>>>>>>>> which >>>>>>>>>>>>>>>> covers KRA uninstall? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> + def teardown_method(self, method): >>>>>>>>>>>>>>>> + for host in self.replicas: >>>>>>>>>>>>>>>> + host.run_command(self.kra_uninstall, >>>>>>>>>>>>>>>> raiseonerr=False) >>>>>>>>>>>>>>>> + tasks.uninstall_master(host) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>>> Can be this function more generic? It should allow specify >>>>>>>>>>>>>>>> host >>>>>>>>>>>>>>>> where >>>>>>>>>>>>>>>> KRA should be installed not just master >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> + def test_kra_install_master(self): >>>>>>>>>>>>>>>> + self.master.run_command(self.kra_install) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> TestLevel0(Dummy): >>>>>>>>>>>>>>>> Can be the test name more specific, something like >>>>>>>>>>>>>>>> TestReplicaPromotionLevel0 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 5) >>>>>>>>>>>>>>>> please remove this, the patch is on review and it will be >>>>>>>>>>>>>>>> pushed >>>>>>>>>>>>>>>> sooner >>>>>>>>>>>>>>>> than tests >>>>>>>>>>>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> and as I mentioned in ticket #5455, I cannot reproduce >>>>>>>>>>>>>>>> it with >>>>>>>>>>>>>>>> ipa-kra-install, so please provide steps to reproduce if >>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>> insist >>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>> this still does not work as expected with KRA. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 6) This is completely wrong, it removes everything that we >>>>>>>>>>>>>>>> tried to >>>>>>>>>>>>>>>> achieve with previous patches with domain level in CI >>>>>>>>>>>>>>> Actually, being able to configure domain level per class >>>>>>>>>>>>>>> is WAY >>>>>>>>>>>>>>> more >>>>>>>>>>>>>>> convenient, than to always have to think which domain >>>>>>>>>>>>>>> level is >>>>>>>>>>>>>>> appropriate for which particular test during jenkins job >>>>>>>>>>>>>>> configuration. In fact, I should have thought about it >>>>>>>>>>>>>>> from the >>>>>>>>>>>>>>> very >>>>>>>>>>>>>>> beginning. For example, in test_replica_promotion.py we >>>>>>>>>>>>>>> have on >>>>>>>>>>>>>>> class, >>>>>>>>>>>>>>> which intiates with domain level = 1, while others - with >>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>> level >>>>>>>>>>>>>>> 0. With config-based approach, we would have to implement a >>>>>>>>>>>>>>> separate >>>>>>>>>>>>>>> step that raises domain level. Overall, I am against the >>>>>>>>>>>>>>> approach, >>>>>>>>>>>>>>> when you have to remember to set certain domain level in >>>>>>>>>>>>>>> config >>>>>>>>>>>>>>> for >>>>>>>>>>>>>>> any particular test. The tests themselves should be aware of >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> domain level they need. >>>>>>>>>>>>>> I do not say that we should not have something that overrides >>>>>>>>>>>>>> settings >>>>>>>>>>>>>> in from config in a particular test case, I say your patch is >>>>>>>>>>>>>> doing it >>>>>>>>>>>>>> wrong. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I agree it is useful to have param domain_level in >>>>>>>>>>>>>> install_master, >>>>>>>>>>>>>> and >>>>>>>>>>>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by >>>>>>>>>>>>>> default, >>>>>>>>>>>>>> because with your current patch the domain_level in config is >>>>>>>>>>>>>> not >>>>>>>>>>>>>> used >>>>>>>>>>>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>>>>>>>>>>> >>>>>>>>>>>>>> For example I want to achieve this goal: >>>>>>>>>>>>>> test_vault.py, this test suite can run on domain level1 >>>>>>>>>>>>>> and on >>>>>>>>>>>>>> domain >>>>>>>>>>>>>> level0, so with one test we can test 2 domain levels just >>>>>>>>>>>>>> with >>>>>>>>>>>>>> putting >>>>>>>>>>>>>> domain level into config file. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I agree that with extraordinary test like replica >>>>>>>>>>>>>> promotion test >>>>>>>>>>>>>> is, we >>>>>>>>>>>>>> need something that allows override the config file. >>>>>>>>>>>>>> >>>>>>>>>>>>>> As I said bellow, domain_level default value should be >>>>>>>>>>>>>> None in >>>>>>>>>>>>>> install_master and install_topo plugin. If domain level was >>>>>>>>>>>>>> specified >>>>>>>>>>>>>> use the specified one, if not (value is None) use the domain >>>>>>>>>>>>>> level >>>>>>>>>>>>>> from >>>>>>>>>>>>>> config file. >>>>>>>>>>>>> Agreed :) >>>>>>>>>>>>> >>>>>>>>>>>>>> Martin >>>>>>>>>>>>>>>> [PATCH] Enabled setting domain_level per class derived from >>>>>>>>>>>>>>>> TestIntegration >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> When I configure domain level 0 in yaml config, how is this >>>>>>>>>>>>>>>> supposed to >>>>>>>>>>>>>>>> get into install methods when you removed that code? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>>>>>>>>>>> + "--domain-level=%i" % domain_level >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> You always use MAX_DOMAIN_LEVEL in this case or whatever is >>>>>>>>>>>>>>>> specified in >>>>>>>>>>>>>>>> domain_level option. >>>>>>>>>>>>>>>> I suggest to use domain_level=None, and when it is None use >>>>>>>>>>>>>>>> 'host.config.domain_level', if it is not None, use >>>>>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> With this we can specify domain level in config file for >>>>>>>>>>>>>>>> test >>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>> be used for both domain levels and you can manually specify >>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>> for test that requires specific domain level. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Also this should go away >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> @classmethod >>>>>>>>>>>>>>>> def install(cls, mh): >>>>>>>>>>>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>>>>>>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>>>>>>>>>>> if cls.topology is None: >>>>>>>>>>>>>>>> return >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I do not see reason why test should override >>>>>>>>>>>>>>>> configuration in >>>>>>>>>>>>>>>> config in >>>>>>>>>>>>>>>> this case. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Martin >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Here is the updated version of the patch (more tests + >>>>>>>>>>>>>>>>> fixed the >>>>>>>>>>>>>>>>> issues of the first one) + patch 0017, that implements the >>>>>>>>>>>>>>>>> necessary >>>>>>>>>>>>>>>>> changes in the background code, i. e. patch 16 does not >>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>> without >>>>>>>>>>>>>>>>> patch 17 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>>>>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>>>>>>>>>>> Hi guys, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Here are first two automated testcases from this (so far >>>>>>>>>>>>>>>>>>> incomplete) >>>>>>>>>>>>>>>>>>> testplan: >>>>>>>>>>>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> PATCH 16: NACK >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>>>>>>>>>>> 'domain_level' to >>>>>>>>>>>>>>>>>> install_topo()? >>>>>>>>>>>>>>>>>> Also it is good practise to add new option as the last >>>>>>>>>>>>>>>>>> parameter. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>>>>> cab you in both tests specify a domain level with >>>>>>>>>>>>>>>>>> constant >>>>>>>>>>>>>>>>>> instead of >>>>>>>>>>>>>>>>>> number literal? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>>>>> both test call install_topo with custom domain level, >>>>>>>>>>>>>>>>>> but it >>>>>>>>>>>>>>>>>> cannot >>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>> because 1) (did you run the test?) >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>>>>>>>>>>> Respectively why there is call of install_topo() that >>>>>>>>>>>>>>>>>> installs >>>>>>>>>>>>>>>>>> replica. >>>>>>>>>>>>>>>>>> As this test just tests that ipa-replica-prepare is not >>>>>>>>>>>>>>>>>> working >>>>>>>>>>>>>>>>>> anymore, >>>>>>>>>>>>>>>>>> is it worth to spend 20 minutes with installing >>>>>>>>>>>>>>>>>> replica and >>>>>>>>>>>>>>>>>> then >>>>>>>>>>>>>>>>>> just no >>>>>>>>>>>>>>>>>> tot use it? IMO to install master in install step is >>>>>>>>>>>>>>>>>> enough. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Martin^2 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>> ./make-lint >>>>>>>>>>>> ************* Module ipatests.test_integration.base >>>>>>>>>>>> ipatests/test_integration/base.py:66: [E1101(no-member), >>>>>>>>>>>> IntegrationTest.install] Class 'IntegrationTest' has no >>>>>>>>>>>> 'domain_level' >>>>>>>>>>>> member) >>>>>>>>>>>> ************* Module >>>>>>>>>>>> ipatests.test_integration.test_replica_promotion >>>>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:16: >>>>>>>>>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no >>>>>>>>>>>> 'domain_level' >>>>>>>>>>>> member) >>>>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:117: >>>>>>>>>>>> [E1101(no-member), >>>>>>>>>>>> TestCAInstall.test_ca_install_without_replica_file] >>>>>>>>>>>> Module 'ipatests.test_integration.tasks' has no 'setup_replica' >>>>>>>>>>>> member) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Is it so hard to run pylint before patch is posted for review? >>>>>>>>>>> Sorry, my bad. Fixed. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> 1) >>>>>>>>> Why is this change in the patch? >>>>>>>>> - # Clean up the test directory >>>>>>>>> - host.run_command(['rm', '-rvf', host.config.test_dir]) >>>>>>>> Otherwise 2 out of 8 tests fail with IOError at line 78 of >>>>>>>> ipatests/test_integration/tasks.py >>>>>>>> >>>>>>>> I do not understand yet how does this happen, but if you remove >>>>>>>> ipatests folder once, it then fails to be created again. >>>>>>>> >>>>>>> So this should be in separated patch and investigated properly. >>>>>> Agree. Removed >>>>>>>>> - >>>>>>>>> >>>>>>>>> >>>>>>>>> 2) >>>>>>>>> is enough to have this check only in install_master, >>>>>>>>> install_topo can >>>>>>>>> pass None to install_master >>>>>>>>> + if domain_level is None: >>>>>>>>> + domain_level = master.config.domain_level >>>>>>>> Done >>>>>>>> >>>>>>>>> 3) >>>>>>>>> IMO replica-manage del should cleanup hosts entry, so following >>>>>>>>> code >>>>>>>>> should not be needed. >>>>>>>>> + if cleanhost: >>>>>>>>> + kinit_admin(master) >>>>>>>>> + master.run_command(["ipa", "host-del", "--updatedns", >>>>>>>>> replica.hostname], >>>>>>>>> + raiseonerr=False) >>>>>>>> Well, in fact it does not. At least the corresponding dns record >>>>>>>> stays >>>>>>>> and causes the subsequent ipa-client-install to fail. Probably >>>>>>>> it's a >>>>>>>> bug. On the other hand, if I promote an existing client to >>>>>>>> replica and >>>>>>>> then delete this replica, then, I probably want the host record >>>>>>>> (that >>>>>>>> was created during client-install) to stay in the system. So, >>>>>>>> does not >>>>>>>> look like a bug to me. >>>>>>> No you don't, because replica uninstallation also removes the >>>>>>> client. >>>>>>> >>>>>>> I tried it with ipa42, ipa-replica-manage del removes host entry, >>>>>>> and >>>>>>> DNS A records, only SSHFP records stay there (I'm not sure if it >>>>>>> is bug >>>>>>> or feature) >>>>>>> >>>>>>> Also I received this message >>>>>>> """ >>>>>>> Failed to cleanup replica1.ipa.test DNS entries: no matching entry >>>>>>> found >>>>>>> You may need to manually remove them from the tree >>>>>>> """ >>>>>>> But, A record has been removed, so this is probably false >>>>>>> positive and >>>>>>> it needs to have a ticket. >>>>>> Agree, that was an issue with my setup. >>>>>> Removed >>>>>> >>>>>>>>> 4) >>>>>>>>> This variable is not used >>>>>>>>> + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] >>>>>>>> Removed >>>>>>>> >>>>>>>>> 5) >>>>>>>>> Why do you need this >>>>>>>>> + kra_install = ["ipa-kra-install", "-U", "-p", >>>>>>>>> config.dirman_password] >>>>>>>>> when you implemented tasks.install_kra that returns the exactly >>>>>>>>> the same >>>>>>>>> result? >>>>>>>> Right. Removed >>>>>>>> >>>>>>>>> 6) >>>>>>>>> PEP8 >>>>>>>>> ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 blank >>>>>>>>> lines, >>>>>>>>> found 1 >>>>>>>>> ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 blank >>>>>>>>> lines, >>>>>>>>> found 1 >>>>>>>>> ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 blank >>>>>>>>> lines, >>>>>>>>> found 1 >>>>>>>>> ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 blank >>>>>>>>> lines, >>>>>>>>> found 1 >>>>>>>>> ./ipatests/test_integration/tasks.py:950:80: E501 line too long >>>>>>>>> (80 > 79 >>>>>>>>> characters) >>>>>>>>> >>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:29:80: E501 >>>>>>>>> line >>>>>>>>> too long (85 > 79 characters) >>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:64:80: E501 >>>>>>>>> line >>>>>>>>> too long (85 > 79 characters) >>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:67:80: E501 >>>>>>>>> line >>>>>>>>> too long (88 > 79 characters) >>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:93:80: E501 >>>>>>>>> line >>>>>>>>> too long (80 > 79 characters) >>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:94:80: E501 >>>>>>>>> line >>>>>>>>> too long (83 > 79 characters) >>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 >>>>>>>>> line >>>>>>>>> too long (81 > 79 characters) >>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 >>>>>>>>> line >>>>>>>>> too long (80 > 79 characters) >>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 >>>>>>>>> line >>>>>>>>> too long (82 > 79 characters) >>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 >>>>>>>>> line >>>>>>>>> too long (80 > 79 characters) >>>>>>>> Most of these complaints are unrelated to the current patches. >>>>>>>> It's better to create a separate patch addressing PEP8 errors. >>>>>>>> >>>>>>> I beg for your pardon, those PEP8 errors have been introduced by >>>>>>> your >>>>>>> patches. >>>>>> Fixed >>>>>> >>>>>>>>> 7) >>>>>>>>> Why this must be stored in instance? IMO to have it stored as >>>>>>>>> local >>>>>>>>> variable is perfect >>>>>>>>> TestKRAInstall, TestCAInstall >>>>>>>>> self.replica1_filename = >>>>>>>>> tasks.get_replica_filename(replica1) >>>>>>>>> self.replica2_filename = >>>>>>>>> tasks.get_replica_filename(replica2) >>>>>>>> Agree. Fixed >>>>>>>> >>>>>> >>>> This patch is missing something. >>> I am sorry, I forgot to revert my previous change. The correct patch is >>> attached >>> >> ************* Module ipatests.test_integration.test_replica_promotion >> ipatests/test_integration/test_replica_promotion.py:15: >> [E1123(unexpected-keyword-arg), Dummy.install] Unexpected keyword >> argument 'domain_level' in function call) >> ipatests/test_integration/test_replica_promotion.py:15: >> [E1101(no-member), Dummy.install] Class 'Dummy' has no 'domain_level' >> member) >> ipatests/test_integration/test_replica_promotion.py:19: >> [E1101(no-member), Dummy.teardown_method] Module >> 'ipatests.test_integration.tasks' has no 'uninstall_replica' member) >> ipatests/test_integration/test_replica_promotion.py:67: >> [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] >> Module 'ipatests.test_integration.tasks' has no 'ipa_backup' member) >> ipatests/test_integration/test_replica_promotion.py:72: >> [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] >> Module 'ipatests.test_integration.tasks' has no 'ipa_restore' member) >> ipatests/test_integration/test_replica_promotion.py:120: >> [E1123(unexpected-keyword-arg), TestCAInstall.install] Unexpected >> keyword argument 'domain_level' in function call) >> > Sorry I forgot to apply patch 17, my bad, I'm continuing with review -- Oleg Fayans Quality Engineer FreeIPA team RedHat. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0016.7-First-part-of-replica-promotion-tests.patch Type: text/x-patch Size: 10166 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0017.4-Enabled-setting-domain_level-per-class.patch Type: text/x-patch Size: 5802 bytes Desc: not available URL: From mbabinsk at redhat.com Tue Dec 15 10:21:53 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Tue, 15 Dec 2015 11:21:53 +0100 Subject: [Freeipa-devel] [PATCH 016 - 017] First part of the replica promotion tests + testplan In-Reply-To: <566FDD80.7030805@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> <565DB7D3.5040500@redhat.com> <565F0BDD.8080206@redhat.com> <565F17EE.3020701@redhat.com> <5666DC18.2040906@redhat.com> <56675E31.3050203@redhat.com> <5667F49B.5020709@redhat.com> <5667FEFB.6090904@redhat.com> <56680C2E.7060805@redhat.com> <56680DDD.2000106@redhat.com> <566FDD80.7030805@redhat.com> Message-ID: <566FE9C1.1060405@redhat.com> On 12/15/2015 10:29 AM, Oleg Fayans wrote: > Hi Martin, > > The updated patches are attached. Patch 0017 includes all changes from > patch 0018, so, if you approve this one, there would be no need to > continue with the review of 0018. This one contains all changes related > to you remarks from 0018 review. Please see my explanation on the > stdout+stderr part in the thread from patch 0018. > With these two patches applied one of the tests fails due this bug: > https://fedorahosted.org/freeipa/ticket/5550 > That may be due to SELinux preventing oddjob to run 'org.freeipa.server.conn_check'. Try running the tests with permissive SELinux on all VMs. > On 12/09/2015 12:17 PM, Martin Basti wrote: >> >> >> On 09.12.2015 12:10, Martin Basti wrote: >>> >>> >>> On 09.12.2015 11:14, Oleg Fayans wrote: >>>> Hi Martin >>>> >>>> On 12/09/2015 10:30 AM, Martin Basti wrote: >>>>> >>>>> On 08.12.2015 23:48, Oleg Fayans wrote: >>>>>> Substituted a hardcoded suffix name with a constant DOMAIN_SUFFIX_NAME >>>>>> >>>>>> On 12/08/2015 02:33 PM, Oleg Fayans wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> >>>>>>> The patches are rebased against the current master. >>>>>>> >>>>>>> On 12/02/2015 05:10 PM, Martin Basti wrote: >>>>>>>> On 02.12.2015 16:18, Oleg Fayans wrote: >>>>>>>>> Hi Martin, >>>>>>>>> >>>>>>>>> On 12/01/2015 04:08 PM, Martin Basti wrote: >>>>>>>>>> On 27.11.2015 16:26, Oleg Fayans wrote: >>>>>>>>>>> And patch N 16 passes lint too: >>>>>>>>>>> >>>>>>>>>>> On 11/27/2015 04:03 PM, Oleg Fayans wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> On 11/27/2015 03:26 PM, Martin Basti wrote: >>>>>>>>>>>>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>>> >>>>>>>>>>>>>> All your suggestions were taken into account. Both patches are >>>>>>>>>>>>>> updated. Thank you for your help! >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>>>>>>>>>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I agree to all your points but one. please, see my comment >>>>>>>>>>>>>>>> below >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 0) Note >>>>>>>>>>>>>>>>> Please be aware of >>>>>>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5469 >>>>>>>>>>>>>>>>> during >>>>>>>>>>>>>>>>> KRA testing >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, >>>>>>>>>>>>>>>>> this may >>>>>>>>>>>>>>>>> change >>>>>>>>>>>>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for domain >>>>>>>>>>>>>>>>> level 0 >>>>>>>>>>>>>>>>> and 1 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>>>> Why uninstall KRA then server, is not enough just uninstall >>>>>>>>>>>>>>>>> server >>>>>>>>>>>>>>>>> which >>>>>>>>>>>>>>>>> covers KRA uninstall? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> + def teardown_method(self, method): >>>>>>>>>>>>>>>>> + for host in self.replicas: >>>>>>>>>>>>>>>>> + host.run_command(self.kra_uninstall, >>>>>>>>>>>>>>>>> raiseonerr=False) >>>>>>>>>>>>>>>>> + tasks.uninstall_master(host) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>>>> Can be this function more generic? It should allow specify >>>>>>>>>>>>>>>>> host >>>>>>>>>>>>>>>>> where >>>>>>>>>>>>>>>>> KRA should be installed not just master >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> + def test_kra_install_master(self): >>>>>>>>>>>>>>>>> + self.master.run_command(self.kra_install) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> TestLevel0(Dummy): >>>>>>>>>>>>>>>>> Can be the test name more specific, something like >>>>>>>>>>>>>>>>> TestReplicaPromotionLevel0 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 5) >>>>>>>>>>>>>>>>> please remove this, the patch is on review and it will be >>>>>>>>>>>>>>>>> pushed >>>>>>>>>>>>>>>>> sooner >>>>>>>>>>>>>>>>> than tests >>>>>>>>>>>>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> and as I mentioned in ticket #5455, I cannot reproduce >>>>>>>>>>>>>>>>> it with >>>>>>>>>>>>>>>>> ipa-kra-install, so please provide steps to reproduce if >>>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>> insist >>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>> this still does not work as expected with KRA. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 6) This is completely wrong, it removes everything that we >>>>>>>>>>>>>>>>> tried to >>>>>>>>>>>>>>>>> achieve with previous patches with domain level in CI >>>>>>>>>>>>>>>> Actually, being able to configure domain level per class >>>>>>>>>>>>>>>> is WAY >>>>>>>>>>>>>>>> more >>>>>>>>>>>>>>>> convenient, than to always have to think which domain >>>>>>>>>>>>>>>> level is >>>>>>>>>>>>>>>> appropriate for which particular test during jenkins job >>>>>>>>>>>>>>>> configuration. In fact, I should have thought about it >>>>>>>>>>>>>>>> from the >>>>>>>>>>>>>>>> very >>>>>>>>>>>>>>>> beginning. For example, in test_replica_promotion.py we >>>>>>>>>>>>>>>> have on >>>>>>>>>>>>>>>> class, >>>>>>>>>>>>>>>> which intiates with domain level = 1, while others - with >>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>> 0. With config-based approach, we would have to implement a >>>>>>>>>>>>>>>> separate >>>>>>>>>>>>>>>> step that raises domain level. Overall, I am against the >>>>>>>>>>>>>>>> approach, >>>>>>>>>>>>>>>> when you have to remember to set certain domain level in >>>>>>>>>>>>>>>> config >>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>> any particular test. The tests themselves should be aware of >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>> domain level they need. >>>>>>>>>>>>>>> I do not say that we should not have something that overrides >>>>>>>>>>>>>>> settings >>>>>>>>>>>>>>> in from config in a particular test case, I say your patch is >>>>>>>>>>>>>>> doing it >>>>>>>>>>>>>>> wrong. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I agree it is useful to have param domain_level in >>>>>>>>>>>>>>> install_master, >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by >>>>>>>>>>>>>>> default, >>>>>>>>>>>>>>> because with your current patch the domain_level in config is >>>>>>>>>>>>>>> not >>>>>>>>>>>>>>> used >>>>>>>>>>>>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> For example I want to achieve this goal: >>>>>>>>>>>>>>> test_vault.py, this test suite can run on domain level1 >>>>>>>>>>>>>>> and on >>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>> level0, so with one test we can test 2 domain levels just >>>>>>>>>>>>>>> with >>>>>>>>>>>>>>> putting >>>>>>>>>>>>>>> domain level into config file. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I agree that with extraordinary test like replica >>>>>>>>>>>>>>> promotion test >>>>>>>>>>>>>>> is, we >>>>>>>>>>>>>>> need something that allows override the config file. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> As I said bellow, domain_level default value should be >>>>>>>>>>>>>>> None in >>>>>>>>>>>>>>> install_master and install_topo plugin. If domain level was >>>>>>>>>>>>>>> specified >>>>>>>>>>>>>>> use the specified one, if not (value is None) use the domain >>>>>>>>>>>>>>> level >>>>>>>>>>>>>>> from >>>>>>>>>>>>>>> config file. >>>>>>>>>>>>>> Agreed :) >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Martin >>>>>>>>>>>>>>>>> [PATCH] Enabled setting domain_level per class derived from >>>>>>>>>>>>>>>>> TestIntegration >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> When I configure domain level 0 in yaml config, how is this >>>>>>>>>>>>>>>>> supposed to >>>>>>>>>>>>>>>>> get into install methods when you removed that code? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>>>>>>>>>>>> + "--domain-level=%i" % domain_level >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> You always use MAX_DOMAIN_LEVEL in this case or whatever is >>>>>>>>>>>>>>>>> specified in >>>>>>>>>>>>>>>>> domain_level option. >>>>>>>>>>>>>>>>> I suggest to use domain_level=None, and when it is None use >>>>>>>>>>>>>>>>> 'host.config.domain_level', if it is not None, use >>>>>>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> With this we can specify domain level in config file for >>>>>>>>>>>>>>>>> test >>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>> be used for both domain levels and you can manually specify >>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>>> for test that requires specific domain level. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Also this should go away >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> @classmethod >>>>>>>>>>>>>>>>> def install(cls, mh): >>>>>>>>>>>>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>>>>>>>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>>>>>>>>>>>> if cls.topology is None: >>>>>>>>>>>>>>>>> return >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I do not see reason why test should override >>>>>>>>>>>>>>>>> configuration in >>>>>>>>>>>>>>>>> config in >>>>>>>>>>>>>>>>> this case. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Martin >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Here is the updated version of the patch (more tests + >>>>>>>>>>>>>>>>>> fixed the >>>>>>>>>>>>>>>>>> issues of the first one) + patch 0017, that implements the >>>>>>>>>>>>>>>>>> necessary >>>>>>>>>>>>>>>>>> changes in the background code, i. e. patch 16 does not >>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>> without >>>>>>>>>>>>>>>>>> patch 17 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>>>>>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>>>>>>>>>>>> Hi guys, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Here are first two automated testcases from this (so far >>>>>>>>>>>>>>>>>>>> incomplete) >>>>>>>>>>>>>>>>>>>> testplan: >>>>>>>>>>>>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> PATCH 16: NACK >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>>>>>>>>>>>> 'domain_level' to >>>>>>>>>>>>>>>>>>> install_topo()? >>>>>>>>>>>>>>>>>>> Also it is good practise to add new option as the last >>>>>>>>>>>>>>>>>>> parameter. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>>>>>> cab you in both tests specify a domain level with >>>>>>>>>>>>>>>>>>> constant >>>>>>>>>>>>>>>>>>> instead of >>>>>>>>>>>>>>>>>>> number literal? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>>>>>> both test call install_topo with custom domain level, >>>>>>>>>>>>>>>>>>> but it >>>>>>>>>>>>>>>>>>> cannot >>>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>>> because 1) (did you run the test?) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>>>>>>>>>>>> Respectively why there is call of install_topo() that >>>>>>>>>>>>>>>>>>> installs >>>>>>>>>>>>>>>>>>> replica. >>>>>>>>>>>>>>>>>>> As this test just tests that ipa-replica-prepare is not >>>>>>>>>>>>>>>>>>> working >>>>>>>>>>>>>>>>>>> anymore, >>>>>>>>>>>>>>>>>>> is it worth to spend 20 minutes with installing >>>>>>>>>>>>>>>>>>> replica and >>>>>>>>>>>>>>>>>>> then >>>>>>>>>>>>>>>>>>> just no >>>>>>>>>>>>>>>>>>> tot use it? IMO to install master in install step is >>>>>>>>>>>>>>>>>>> enough. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Martin^2 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>> ./make-lint >>>>>>>>>>>>> ************* Module ipatests.test_integration.base >>>>>>>>>>>>> ipatests/test_integration/base.py:66: [E1101(no-member), >>>>>>>>>>>>> IntegrationTest.install] Class 'IntegrationTest' has no >>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>> member) >>>>>>>>>>>>> ************* Module >>>>>>>>>>>>> ipatests.test_integration.test_replica_promotion >>>>>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:16: >>>>>>>>>>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no >>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>> member) >>>>>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:117: >>>>>>>>>>>>> [E1101(no-member), >>>>>>>>>>>>> TestCAInstall.test_ca_install_without_replica_file] >>>>>>>>>>>>> Module 'ipatests.test_integration.tasks' has no 'setup_replica' >>>>>>>>>>>>> member) >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Is it so hard to run pylint before patch is posted for review? >>>>>>>>>>>> Sorry, my bad. Fixed. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> 1) >>>>>>>>>> Why is this change in the patch? >>>>>>>>>> - # Clean up the test directory >>>>>>>>>> - host.run_command(['rm', '-rvf', host.config.test_dir]) >>>>>>>>> Otherwise 2 out of 8 tests fail with IOError at line 78 of >>>>>>>>> ipatests/test_integration/tasks.py >>>>>>>>> >>>>>>>>> I do not understand yet how does this happen, but if you remove >>>>>>>>> ipatests folder once, it then fails to be created again. >>>>>>>>> >>>>>>>> So this should be in separated patch and investigated properly. >>>>>>> Agree. Removed >>>>>>>>>> - >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2) >>>>>>>>>> is enough to have this check only in install_master, >>>>>>>>>> install_topo can >>>>>>>>>> pass None to install_master >>>>>>>>>> + if domain_level is None: >>>>>>>>>> + domain_level = master.config.domain_level >>>>>>>>> Done >>>>>>>>> >>>>>>>>>> 3) >>>>>>>>>> IMO replica-manage del should cleanup hosts entry, so following >>>>>>>>>> code >>>>>>>>>> should not be needed. >>>>>>>>>> + if cleanhost: >>>>>>>>>> + kinit_admin(master) >>>>>>>>>> + master.run_command(["ipa", "host-del", "--updatedns", >>>>>>>>>> replica.hostname], >>>>>>>>>> + raiseonerr=False) >>>>>>>>> Well, in fact it does not. At least the corresponding dns record >>>>>>>>> stays >>>>>>>>> and causes the subsequent ipa-client-install to fail. Probably >>>>>>>>> it's a >>>>>>>>> bug. On the other hand, if I promote an existing client to >>>>>>>>> replica and >>>>>>>>> then delete this replica, then, I probably want the host record >>>>>>>>> (that >>>>>>>>> was created during client-install) to stay in the system. So, >>>>>>>>> does not >>>>>>>>> look like a bug to me. >>>>>>>> No you don't, because replica uninstallation also removes the >>>>>>>> client. >>>>>>>> >>>>>>>> I tried it with ipa42, ipa-replica-manage del removes host entry, >>>>>>>> and >>>>>>>> DNS A records, only SSHFP records stay there (I'm not sure if it >>>>>>>> is bug >>>>>>>> or feature) >>>>>>>> >>>>>>>> Also I received this message >>>>>>>> """ >>>>>>>> Failed to cleanup replica1.ipa.test DNS entries: no matching entry >>>>>>>> found >>>>>>>> You may need to manually remove them from the tree >>>>>>>> """ >>>>>>>> But, A record has been removed, so this is probably false >>>>>>>> positive and >>>>>>>> it needs to have a ticket. >>>>>>> Agree, that was an issue with my setup. >>>>>>> Removed >>>>>>> >>>>>>>>>> 4) >>>>>>>>>> This variable is not used >>>>>>>>>> + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] >>>>>>>>> Removed >>>>>>>>> >>>>>>>>>> 5) >>>>>>>>>> Why do you need this >>>>>>>>>> + kra_install = ["ipa-kra-install", "-U", "-p", >>>>>>>>>> config.dirman_password] >>>>>>>>>> when you implemented tasks.install_kra that returns the exactly >>>>>>>>>> the same >>>>>>>>>> result? >>>>>>>>> Right. Removed >>>>>>>>> >>>>>>>>>> 6) >>>>>>>>>> PEP8 >>>>>>>>>> ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 blank >>>>>>>>>> lines, >>>>>>>>>> found 1 >>>>>>>>>> ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 blank >>>>>>>>>> lines, >>>>>>>>>> found 1 >>>>>>>>>> ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 blank >>>>>>>>>> lines, >>>>>>>>>> found 1 >>>>>>>>>> ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 blank >>>>>>>>>> lines, >>>>>>>>>> found 1 >>>>>>>>>> ./ipatests/test_integration/tasks.py:950:80: E501 line too long >>>>>>>>>> (80 > 79 >>>>>>>>>> characters) >>>>>>>>>> >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:29:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (85 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:64:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (85 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:67:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (88 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:93:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (80 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:94:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (83 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (81 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (80 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (82 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (80 > 79 characters) >>>>>>>>> Most of these complaints are unrelated to the current patches. >>>>>>>>> It's better to create a separate patch addressing PEP8 errors. >>>>>>>>> >>>>>>>> I beg for your pardon, those PEP8 errors have been introduced by >>>>>>>> your >>>>>>>> patches. >>>>>>> Fixed >>>>>>> >>>>>>>>>> 7) >>>>>>>>>> Why this must be stored in instance? IMO to have it stored as >>>>>>>>>> local >>>>>>>>>> variable is perfect >>>>>>>>>> TestKRAInstall, TestCAInstall >>>>>>>>>> self.replica1_filename = >>>>>>>>>> tasks.get_replica_filename(replica1) >>>>>>>>>> self.replica2_filename = >>>>>>>>>> tasks.get_replica_filename(replica2) >>>>>>>>> Agree. Fixed >>>>>>>>> >>>>>>> >>>>> This patch is missing something. >>>> I am sorry, I forgot to revert my previous change. The correct patch is >>>> attached >>>> >>> ************* Module ipatests.test_integration.test_replica_promotion >>> ipatests/test_integration/test_replica_promotion.py:15: >>> [E1123(unexpected-keyword-arg), Dummy.install] Unexpected keyword >>> argument 'domain_level' in function call) >>> ipatests/test_integration/test_replica_promotion.py:15: >>> [E1101(no-member), Dummy.install] Class 'Dummy' has no 'domain_level' >>> member) >>> ipatests/test_integration/test_replica_promotion.py:19: >>> [E1101(no-member), Dummy.teardown_method] Module >>> 'ipatests.test_integration.tasks' has no 'uninstall_replica' member) >>> ipatests/test_integration/test_replica_promotion.py:67: >>> [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] >>> Module 'ipatests.test_integration.tasks' has no 'ipa_backup' member) >>> ipatests/test_integration/test_replica_promotion.py:72: >>> [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] >>> Module 'ipatests.test_integration.tasks' has no 'ipa_restore' member) >>> ipatests/test_integration/test_replica_promotion.py:120: >>> [E1123(unexpected-keyword-arg), TestCAInstall.install] Unexpected >>> keyword argument 'domain_level' in function call) >>> >> Sorry I forgot to apply patch 17, my bad, I'm continuing with review > > > -- Martin^3 Babinsky From mbasti at redhat.com Tue Dec 15 11:01:51 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 15 Dec 2015 12:01:51 +0100 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <566FC72B.3050406@redhat.com> References: <566FC72B.3050406@redhat.com> Message-ID: <566FF31F.7030503@redhat.com> On 15.12.2015 08:54, Jan Cholasta wrote: > Hi, > > recently I and David discussed the direction of installers with regard > to requesting certificates. Currently there are four (!) different > ways of requesting certificates in the installer [1][2][3][4]. We > would like to reduce it to one. > > Since all the certificates are tracked by certmonger and certmonger > already knows how to request certificates from Dogtag (and other CAs), > we believe that all certificates should be requested using certmonger. > > Taking our meditation further, we thought "Why not use certmonger for > the cert-request command as well?" What is the benefit, do you ask? > > a) single code path for requesting certificates (seriously, the > current state is ridiculous) > > b) use any CA supported by certmonger as the IPA CA (i.e. Let's > Encrypt [5], once certmonger gains support for it) > > c) automate external CA install, using any CA supported by certmonger > [6] > > d) support multiple different CAs at once (generalization of the > Sub-CA feature) > > e) uniform configuration on clients (configure once, use forever, > even for CA-less) > > The idea is to store configuration for the different CAs in LDAP and > have cert-request redirect requests to a proper CA helper according to > that configuration. This would require a new certmonger D-Bus method > to call a CA helper without associated certificate storage, but that > should be rather easy to add. In return, it would be possible to do > all of the above. > > Note that this should not conflict with tighter integration with > Dogtag (profiles, ACLs, etc.). > > Comments are welcome. > > Honza > > [1] > > [2] > > [3] > > [4] > > [5] > [6] > LGTM From mkosek at redhat.com Tue Dec 15 11:39:29 2015 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 Dec 2015 12:39:29 +0100 Subject: [Freeipa-devel] [PATCH 0117] ipa-client-install: create a temporary directory for ccache files In-Reply-To: <566F01AC.1010307@redhat.com> References: <566EEEF0.6040800@redhat.com> <566F01AC.1010307@redhat.com> Message-ID: <566FFBF1.2070803@redhat.com> On 12/14/2015 06:51 PM, Tomas Babej wrote: > > > On 12/14/2015 05:31 PM, Martin Babinsky wrote: >> fixes https://fedorahosted.org/freeipa/ticket/5528 > > Works as expected, code-wise looks good. > > Thanks for looking into this, ACK! > > Pushed to master: 5886f87f974fa508047a21350c2e6e75a3001da6 > Looks like something that should also land in ipa-4-3, no? From mkosek at redhat.com Tue Dec 15 11:40:25 2015 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 Dec 2015 12:40:25 +0100 Subject: [Freeipa-devel] [PATCH 0117] ipa-client-install: create a temporary directory for ccache files In-Reply-To: <566FFBF1.2070803@redhat.com> References: <566EEEF0.6040800@redhat.com> <566F01AC.1010307@redhat.com> <566FFBF1.2070803@redhat.com> Message-ID: <566FFC29.8070807@redhat.com> On 12/15/2015 12:39 PM, Martin Kosek wrote: > On 12/14/2015 06:51 PM, Tomas Babej wrote: >> >> >> On 12/14/2015 05:31 PM, Martin Babinsky wrote: >>> fixes https://fedorahosted.org/freeipa/ticket/5528 >> >> Works as expected, code-wise looks good. >> >> Thanks for looking into this, ACK! >> >> Pushed to master: 5886f87f974fa508047a21350c2e6e75a3001da6 >> > > Looks like something that should also land in ipa-4-3, no? > Ah, the actual push was yesterday before the branch - please disregard my message :-) From mkosek at redhat.com Tue Dec 15 11:41:56 2015 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 Dec 2015 12:41:56 +0100 Subject: [Freeipa-devel] [PATCH 0117] ipa-client-install: create a temporary directory for ccache files In-Reply-To: <566FBB3D.1060708@redhat.com> References: <566EEEF0.6040800@redhat.com> <566F01AC.1010307@redhat.com> <566FB0D5.8070306@redhat.com> <566FBB3D.1060708@redhat.com> Message-ID: <566FFC84.4060304@redhat.com> On 12/15/2015 08:03 AM, Martin Babinsky wrote: > On 12/15/2015 07:19 AM, Jan Cholasta wrote: >> On 14.12.2015 18:51, Tomas Babej wrote: >>> >>> >>> On 12/14/2015 05:31 PM, Martin Babinsky wrote: >>>> fixes https://fedorahosted.org/freeipa/ticket/5528 >>> >>> Works as expected, code-wise looks good. >>> >>> Thanks for looking into this, ACK! >>> >>> Pushed to master: 5886f87f974fa508047a21350c2e6e75a3001da6 >> >> It would probably be better if ipa-client-install used >> ipautil.private_ccache(), but I guess we can fix that later, once we >> start digging into ipa-client-install componentization / rectofaring. >> > Yes since that would probably require heroic amounts of spaghetti untangling to > get it right. Anyway we have any milestone for ipa-client refactoring? Is it a > part of 4.4 installer redesign effort? Client refactoring is not currently planned for 4.4. There are plans for "Thin Client", but not ipa-client-install refactoring. This would fall in 4.5 or even later, I think. From ldoudova at redhat.com Tue Dec 15 11:42:26 2015 From: ldoudova at redhat.com (Lenka Doudova) Date: Tue, 15 Dec 2015 12:42:26 +0100 Subject: [Freeipa-devel] [TESTS][PATCH 0007] Multiple managers per user Message-ID: <566FFCA2.2060801@redhat.com> Hi, I updated the (stage)user tests to reflect the multiple managers per user feature. Corresponding ticket: https://fedorahosted.org/freeipa/ticket/5344 Lenka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ldoudova-0007-Tests-Multiple-managers-per-user.patch Type: text/x-patch Size: 22208 bytes Desc: not available URL: From jcholast at redhat.com Tue Dec 15 11:51:15 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 15 Dec 2015 12:51:15 +0100 Subject: [Freeipa-devel] [PATCH 0117] ipa-client-install: create a temporary directory for ccache files In-Reply-To: <566FFC84.4060304@redhat.com> References: <566EEEF0.6040800@redhat.com> <566F01AC.1010307@redhat.com> <566FB0D5.8070306@redhat.com> <566FBB3D.1060708@redhat.com> <566FFC84.4060304@redhat.com> Message-ID: <566FFEB3.1000409@redhat.com> On 15.12.2015 12:41, Martin Kosek wrote: > On 12/15/2015 08:03 AM, Martin Babinsky wrote: >> On 12/15/2015 07:19 AM, Jan Cholasta wrote: >>> On 14.12.2015 18:51, Tomas Babej wrote: >>>> >>>> >>>> On 12/14/2015 05:31 PM, Martin Babinsky wrote: >>>>> fixes https://fedorahosted.org/freeipa/ticket/5528 >>>> >>>> Works as expected, code-wise looks good. >>>> >>>> Thanks for looking into this, ACK! >>>> >>>> Pushed to master: 5886f87f974fa508047a21350c2e6e75a3001da6 >>> >>> It would probably be better if ipa-client-install used >>> ipautil.private_ccache(), but I guess we can fix that later, once we >>> start digging into ipa-client-install componentization / rectofaring. >>> >> Yes since that would probably require heroic amounts of spaghetti untangling to >> get it right. Anyway we have any milestone for ipa-client refactoring? Is it a >> part of 4.4 installer redesign effort? > > Client refactoring is not currently planned for 4.4. There are plans for "Thin > Client", but not ipa-client-install refactoring. This would fall in 4.5 or even > later, I think. For further server/replica installer improvements, it's necessary to at least create a module from ipa-client-install. So, unless you want to freeze server installers in their current (WIP) state, we have to touch ipa-client-install as well. -- Jan Cholasta From tscherf at redhat.com Tue Dec 15 13:21:44 2015 From: tscherf at redhat.com (Thorsten Scherf) Date: Tue, 15 Dec 2015 14:21:44 +0100 Subject: [Freeipa-devel] [PATCH 0001] ipa-server-certinstall should not tell certmonger to track 3rd party certificates Message-ID: <20151215132144.GI25566@tscherf.redhat.com> -------------- next part -------------- >From 13c564e106300f19ba59531ebdf3ba90b7b8ef35 Mon Sep 17 00:00:00 2001 From: Thorsten Scherf Date: Tue, 15 Dec 2015 14:17:26 +0100 Subject: [PATCH] ipa-server-certinstall should not tell certmonger to track 3rd party certs There is no point in tracking a 3rd party cert, as it cannot be renewed automatically. https://fedorahosted.org/freeipa/ticket/4785 --- ipaserver/install/ipa_server_certinstall.py | 3 --- 1 file changed, 3 deletions(-) diff --git a/ipaserver/install/ipa_server_certinstall.py b/ipaserver/install/ipa_server_certinstall.py index ac0b0274e4e36db4ea6fb695afb527e2b83a8c77..b5f317cd6328dbaec694b5eff6230f8a9fe302a7 100644 --- a/ipaserver/install/ipa_server_certinstall.py +++ b/ipaserver/install/ipa_server_certinstall.py @@ -175,9 +175,6 @@ class ServerCertInstall(admintool.AdminTool): cdb.import_pkcs12(pkcs12_file.name, pin) server_cert = cdb.find_server_certs()[0][0] - if ca_enabled: - cdb.track_server_cert(server_cert, principal, cdb.passwd_fname, - command) except RuntimeError as e: raise admintool.ScriptError(str(e)) -- 1.9.3 From jcholast at redhat.com Tue Dec 15 14:39:38 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 15 Dec 2015 15:39:38 +0100 Subject: [Freeipa-devel] [PATCH 535] ipautil: remove unused import causing cyclic import in tests Message-ID: <5670262A.9060304@redhat.com> Hi, the attached patch fixes . Pushed to under the one-liner rule: master: c265e8736e51d5b4fede94a414d83b3e0ada2853 ipa-4-3: 2b28704f926d24e1562b481880b111d12d2020f2 Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-535-ipautil-remove-unused-import-causing-cyclic-import-i.patch Type: text/x-patch Size: 760 bytes Desc: not available URL: From pviktori at redhat.com Tue Dec 15 14:53:37 2015 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 15 Dec 2015 15:53:37 +0100 Subject: [Freeipa-devel] [PATCH] 0751 spec: Split out python-ipap11helper and, python-default_encoding_utf8 In-Reply-To: <566E6D61.80002@redhat.com> References: <56606E90.3000909@redhat.com> <56619535.1010700@redhat.com> <566E6D61.80002@redhat.com> Message-ID: <56702971.1060709@redhat.com> On 12/14/2015 08:18 AM, Jan Cholasta wrote: > On 4.12.2015 14:29, Jan Cholasta wrote: >> Hi, >> >> On 3.12.2015 17:32, Petr Viktorin wrote: >>> Hello, >>> This specfile patch makes python-ipalib noarch, by splitting out the >>> arch-dependent stuff. >> >> I'm not sure if this should be done at all. Both py_default_encoding and >> (especially) _ipap11helper are internal submodules of ipapython and I >> don't think they should be packaged separately. Technically they aren't ? it's "import _ipap11helper", not "import ipapython._ipap11helper". Their source is in the ipapython directory, but they're built as independent modules. >> If this is just about packaging arch-specific code separately from >> noarch code, I'm not sure either - all other packages with both Python >> and C modules I have seen (e.g. python-ldap, PyYAML) simply do not care >> and put everything in a single arch-specific package. IMHO we should >> follow. > > This would mean putting everything into the architecture-specific > prefix. Currently the python files in python2-ipalib are installed into > the architecture-agnostic prefix, which is wrong, because you can't > install both python2-ipalib.i686 and python2-ipalib.x86_64 at the same > time. Putting everything into arch-specific prefix would mean that if you do install both arches simultaneously, you'd get two identical copies of ipalib, ipapython, and ipaplatform. So if having both installed is a concern, I think the noarch bits should be split out. -- Petr Viktorin From jcholast at redhat.com Tue Dec 15 15:01:51 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 15 Dec 2015 16:01:51 +0100 Subject: [Freeipa-devel] [PATCH] 0751 spec: Split out python-ipap11helper and, python-default_encoding_utf8 In-Reply-To: <56702971.1060709@redhat.com> References: <56606E90.3000909@redhat.com> <56619535.1010700@redhat.com> <566E6D61.80002@redhat.com> <56702971.1060709@redhat.com> Message-ID: <56702B5F.1090403@redhat.com> On 15.12.2015 15:53, Petr Viktorin wrote: > On 12/14/2015 08:18 AM, Jan Cholasta wrote: >> On 4.12.2015 14:29, Jan Cholasta wrote: >>> Hi, >>> >>> On 3.12.2015 17:32, Petr Viktorin wrote: >>>> Hello, >>>> This specfile patch makes python-ipalib noarch, by splitting out the >>>> arch-dependent stuff. >>> >>> I'm not sure if this should be done at all. Both py_default_encoding and >>> (especially) _ipap11helper are internal submodules of ipapython and I >>> don't think they should be packaged separately. > > Technically they aren't ? it's "import _ipap11helper", not "import > ipapython._ipap11helper". Their source is in the ipapython directory, > but they're built as independent modules. This is an implementation detail rather than an architectural decision. For _ipap11helper, we haven't figured out how to properly put it inside ipapython, so it's build separately, but it actually is a part of ipapython. > >>> If this is just about packaging arch-specific code separately from >>> noarch code, I'm not sure either - all other packages with both Python >>> and C modules I have seen (e.g. python-ldap, PyYAML) simply do not care >>> and put everything in a single arch-specific package. IMHO we should >>> follow. >> >> This would mean putting everything into the architecture-specific >> prefix. Currently the python files in python2-ipalib are installed into >> the architecture-agnostic prefix, which is wrong, because you can't >> install both python2-ipalib.i686 and python2-ipalib.x86_64 at the same >> time. > > Putting everything into arch-specific prefix would mean that if you do > install both arches simultaneously, you'd get two identical copies of > ipalib, ipapython, and ipaplatform. So if having both installed is a > concern, I think the noarch bits should be split out. I'm aware this means two identical copies of the modules, but the same thing works for python-ldap and PyYAML (as I pointed out above), so why shouldn't it work for us? -- Jan Cholasta From mkosek at redhat.com Tue Dec 15 15:20:57 2015 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 Dec 2015 16:20:57 +0100 Subject: [Freeipa-devel] [PATCH 0365] Remove unused KRA code from ipa-server-install In-Reply-To: <566FB63E.8080205@redhat.com> References: <565C6BBC.4050902@redhat.com> <566EC95C.2010208@redhat.com> <20151214140540.GT4620@redhat.com> <566ED2A6.8000202@redhat.com> <20151214155442.GV4620@redhat.com> <566FB63E.8080205@redhat.com> Message-ID: <56702FD9.7050008@redhat.com> On 12/15/2015 07:42 AM, Jan Cholasta wrote: > On 14.12.2015 16:54, Alexander Bokovoy wrote: >> On Mon, 14 Dec 2015, David Kupka wrote: ... > We can always have both. With the new installer framework it is trivial to fold > installers like this without code duplication. It is still work in progress > though, so I would prefer not to add --setup-kra to ipa-server-install now (if > ever). +1, I would also rather not make ipa-server-install too monolithic. Even from testing POV, I think doing KRA fixes in one installer is much more easier than doing it also in replica or server installers (although Jan's patches help here, that's true). From mkosek at redhat.com Tue Dec 15 15:23:33 2015 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 Dec 2015 16:23:33 +0100 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <566FC72B.3050406@redhat.com> References: <566FC72B.3050406@redhat.com> Message-ID: <56703075.7040108@redhat.com> On 12/15/2015 08:54 AM, Jan Cholasta wrote: > Hi, > > recently I and David discussed the direction of installers with regard to > requesting certificates. Currently there are four (!) different ways of > requesting certificates in the installer [1][2][3][4]. We would like to reduce > it to one. > > Since all the certificates are tracked by certmonger and certmonger already > knows how to request certificates from Dogtag (and other CAs), we believe that > all certificates should be requested using certmonger. > > Taking our meditation further, we thought "Why not use certmonger for the > cert-request command as well?" What is the benefit, do you ask? > > a) single code path for requesting certificates (seriously, the current state > is ridiculous) > > b) use any CA supported by certmonger as the IPA CA (i.e. Let's Encrypt [5], > once certmonger gains support for it) > > c) automate external CA install, using any CA supported by certmonger [6] > > d) support multiple different CAs at once (generalization of the Sub-CA feature) > > e) uniform configuration on clients (configure once, use forever, even for > CA-less) > > The idea is to store configuration for the different CAs in LDAP and have > cert-request redirect requests to a proper CA helper according to that > configuration. This would require a new certmonger D-Bus method to call a CA > helper without associated certificate storage, but that should be rather easy > to add. In return, it would be possible to do all of the above. > > Note that this should not conflict with tighter integration with Dogtag > (profiles, ACLs, etc.). > > Comments are welcome. > > Honza > > [1] > > [2] > > > [3] > > > [4] > > > [5] > [6] Interesting idea! I would be definitely interested to hear what Fraser, Rob or Simo thinks here. From pvoborni at redhat.com Tue Dec 15 15:35:48 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 15 Dec 2015 16:35:48 +0100 Subject: [Freeipa-devel] Testing FreeIPA 4.3 for GA In-Reply-To: <566F595E.9070807@redhat.com> References: <566F595E.9070807@redhat.com> Message-ID: <56703354.5060602@redhat.com> On 12/15/2015 01:05 AM, Petr Vobornik wrote: > Blocking patches for FreeIPA 4.3 were pushed, ipa-4-3 branch was > created. Master branch is ready for 4.4 development. > > A build is available for testing in my pvoborni/freeipa-4-3 COPR repo > [1] until the official 4.3 repo is created. The repo contains only this > build. The build is not pure upstream ipa-4-3 branch but rather a build > which will go to Fedora rawhide and official 4.3 copr repo - Fedora > downstream spec with the SELinux workaround applied [2][3]. > > Known issues: > * https://fedorahosted.org/freeipa/ticket/5469 - requires fix in PKI > * https://bugzilla.redhat.com/show_bug.cgi?id=1289930 - SELinux Policy > update needed for connection check > > I have started drafting release page [4]. > > [1] https://copr.fedoraproject.org/coprs/pvoborni/freeipa-4-3/ > [2] https://fedorahosted.org/freeipa/ticket/5533 > [3] > http://www.redhat.com/archives/freeipa-devel/2015-December/msg00234.html > [4] http://www.freeipa.org/page/Releases/4.3.0 Copr contains a rebuild with a patch for ticket: https://fedorahosted.org/freeipa/ticket/5551 https://copr.fedoraproject.org/coprs/pvoborni/freeipa-4-3/build/147975/ The build has exactly the same version as the one before. Ales, Milan, do we want to differentiate that somehow? -- Petr Vobornik From dkupka at redhat.com Tue Dec 15 15:43:59 2015 From: dkupka at redhat.com (David Kupka) Date: Tue, 15 Dec 2015 16:43:59 +0100 Subject: [Freeipa-devel] [PATCH 0365] Remove unused KRA code from ipa-server-install In-Reply-To: <56702FD9.7050008@redhat.com> References: <565C6BBC.4050902@redhat.com> <566EC95C.2010208@redhat.com> <20151214140540.GT4620@redhat.com> <566ED2A6.8000202@redhat.com> <20151214155442.GV4620@redhat.com> <566FB63E.8080205@redhat.com> <56702FD9.7050008@redhat.com> Message-ID: <5670353F.6070801@redhat.com> On 15/12/15 16:20, Martin Kosek wrote: > On 12/15/2015 07:42 AM, Jan Cholasta wrote: >> On 14.12.2015 16:54, Alexander Bokovoy wrote: >>> On Mon, 14 Dec 2015, David Kupka wrote: > ... >> We can always have both. With the new installer framework it is trivial to fold >> installers like this without code duplication. It is still work in progress >> though, so I would prefer not to add --setup-kra to ipa-server-install now (if >> ever). > > +1, I would also rather not make ipa-server-install too monolithic. Even from > testing POV, I think doing KRA fixes in one installer is much more easier than > doing it also in replica or server installers (although Jan's patches help > here, that's true). > If you look on the patch Martin send, you will see that there is nothing monolithic in installing KRA in ipa-server-install. In fact, it calls kra.check_install() and kra.install(), the very same functions that are called from ipa-kra-install. But it seems that majority of us is decided to remove the ability to install KRA from ipa-server-install and since I've tested the patch and didn't find any issues we can push it, ACK. -- David Kupka From mkubik at redhat.com Tue Dec 15 15:53:18 2015 From: mkubik at redhat.com (=?UTF-8?Q?Milan_Kub=c3=adk?=) Date: Tue, 15 Dec 2015 16:53:18 +0100 Subject: [Freeipa-devel] Testing FreeIPA 4.3 for GA In-Reply-To: <56703354.5060602@redhat.com> References: <566F595E.9070807@redhat.com> <56703354.5060602@redhat.com> Message-ID: <5670376E.5080002@redhat.com> On 12/15/2015 04:35 PM, Petr Vobornik wrote: > On 12/15/2015 01:05 AM, Petr Vobornik wrote: >> Blocking patches for FreeIPA 4.3 were pushed, ipa-4-3 branch was >> created. Master branch is ready for 4.4 development. >> >> A build is available for testing in my pvoborni/freeipa-4-3 COPR repo >> [1] until the official 4.3 repo is created. The repo contains only this >> build. The build is not pure upstream ipa-4-3 branch but rather a build >> which will go to Fedora rawhide and official 4.3 copr repo - Fedora >> downstream spec with the SELinux workaround applied [2][3]. >> >> Known issues: >> * https://fedorahosted.org/freeipa/ticket/5469 - requires fix in PKI >> * https://bugzilla.redhat.com/show_bug.cgi?id=1289930 - SELinux Policy >> update needed for connection check >> >> I have started drafting release page [4]. >> >> [1] https://copr.fedoraproject.org/coprs/pvoborni/freeipa-4-3/ >> [2] https://fedorahosted.org/freeipa/ticket/5533 >> [3] >> http://www.redhat.com/archives/freeipa-devel/2015-December/msg00234.html >> [4] http://www.freeipa.org/page/Releases/4.3.0 > > Copr contains a rebuild with a patch for ticket: > https://fedorahosted.org/freeipa/ticket/5551 > > https://copr.fedoraproject.org/coprs/pvoborni/freeipa-4-3/build/147975/ > > The build has exactly the same version as the one before. Ales, Milan, > do we want to differentiate that somehow? > The job uses fresh install of the rpms. The same version and build shouldn't be a problem. -- Milan Kubik From amarecek at redhat.com Tue Dec 15 16:04:52 2015 From: amarecek at redhat.com (=?utf-8?Q?Ale=C5=A1_Mare=C4=8Dek?=) Date: Tue, 15 Dec 2015 11:04:52 -0500 (EST) Subject: [Freeipa-devel] Testing FreeIPA 4.3 for GA In-Reply-To: <5670376E.5080002@redhat.com> References: <566F595E.9070807@redhat.com> <56703354.5060602@redhat.com> <5670376E.5080002@redhat.com> Message-ID: <1128975006.28879270.1450195492837.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Milan Kub?k" > To: "Petr Vobornik" > Cc: "freeipa-devel" , "Ales Marecek" > Sent: Tuesday, December 15, 2015 4:53:18 PM > Subject: Re: [Freeipa-devel] Testing FreeIPA 4.3 for GA > > On 12/15/2015 04:35 PM, Petr Vobornik wrote: > > On 12/15/2015 01:05 AM, Petr Vobornik wrote: > >> Blocking patches for FreeIPA 4.3 were pushed, ipa-4-3 branch was > >> created. Master branch is ready for 4.4 development. > >> > >> A build is available for testing in my pvoborni/freeipa-4-3 COPR repo > >> [1] until the official 4.3 repo is created. The repo contains only this > >> build. The build is not pure upstream ipa-4-3 branch but rather a build > >> which will go to Fedora rawhide and official 4.3 copr repo - Fedora > >> downstream spec with the SELinux workaround applied [2][3]. > >> > >> Known issues: > >> * https://fedorahosted.org/freeipa/ticket/5469 - requires fix in PKI > >> * https://bugzilla.redhat.com/show_bug.cgi?id=1289930 - SELinux Policy > >> update needed for connection check > >> > >> I have started drafting release page [4]. > >> > >> [1] https://copr.fedoraproject.org/coprs/pvoborni/freeipa-4-3/ > >> [2] https://fedorahosted.org/freeipa/ticket/5533 > >> [3] > >> http://www.redhat.com/archives/freeipa-devel/2015-December/msg00234.html > >> [4] http://www.freeipa.org/page/Releases/4.3.0 > > > > Copr contains a rebuild with a patch for ticket: > > https://fedorahosted.org/freeipa/ticket/5551 > > > > https://copr.fedoraproject.org/coprs/pvoborni/freeipa-4-3/build/147975/ > > > > The build has exactly the same version as the one before. Ales, Milan, > > do we want to differentiate that somehow? > > > The job uses fresh install of the rpms. The same version and build > shouldn't be a problem. +1, I don't see any issue to use the "same" (not-yet-released) version. > > -- > Milan Kubik > > From mkubik at redhat.com Tue Dec 15 16:48:15 2015 From: mkubik at redhat.com (=?UTF-8?Q?Milan_Kub=c3=adk?=) Date: Tue, 15 Dec 2015 17:48:15 +0100 Subject: [Freeipa-devel] Testing FreeIPA 4.3 for GA In-Reply-To: <1128975006.28879270.1450195492837.JavaMail.zimbra@redhat.com> References: <566F595E.9070807@redhat.com> <56703354.5060602@redhat.com> <5670376E.5080002@redhat.com> <1128975006.28879270.1450195492837.JavaMail.zimbra@redhat.com> Message-ID: <5670444F.7020906@redhat.com> On 12/15/2015 05:04 PM, Ale? Mare?ek wrote: > > ----- Original Message ----- >> From: "Milan Kub?k" >> To: "Petr Vobornik" >> Cc: "freeipa-devel" , "Ales Marecek" >> Sent: Tuesday, December 15, 2015 4:53:18 PM >> Subject: Re: [Freeipa-devel] Testing FreeIPA 4.3 for GA >> >> On 12/15/2015 04:35 PM, Petr Vobornik wrote: >>> On 12/15/2015 01:05 AM, Petr Vobornik wrote: >>>> Blocking patches for FreeIPA 4.3 were pushed, ipa-4-3 branch was >>>> created. Master branch is ready for 4.4 development. >>>> >>>> A build is available for testing in my pvoborni/freeipa-4-3 COPR repo >>>> [1] until the official 4.3 repo is created. The repo contains only this >>>> build. The build is not pure upstream ipa-4-3 branch but rather a build >>>> which will go to Fedora rawhide and official 4.3 copr repo - Fedora >>>> downstream spec with the SELinux workaround applied [2][3]. >>>> >>>> Known issues: >>>> * https://fedorahosted.org/freeipa/ticket/5469 - requires fix in PKI >>>> * https://bugzilla.redhat.com/show_bug.cgi?id=1289930 - SELinux Policy >>>> update needed for connection check >>>> >>>> I have started drafting release page [4]. >>>> >>>> [1] https://copr.fedoraproject.org/coprs/pvoborni/freeipa-4-3/ >>>> [2] https://fedorahosted.org/freeipa/ticket/5533 >>>> [3] >>>> http://www.redhat.com/archives/freeipa-devel/2015-December/msg00234.html >>>> [4] http://www.freeipa.org/page/Releases/4.3.0 >>> Copr contains a rebuild with a patch for ticket: >>> https://fedorahosted.org/freeipa/ticket/5551 >>> >>> https://copr.fedoraproject.org/coprs/pvoborni/freeipa-4-3/build/147975/ >>> >>> The build has exactly the same version as the one before. Ales, Milan, >>> do we want to differentiate that somehow? >>> >> The job uses fresh install of the rpms. The same version and build >> shouldn't be a problem. > +1, I don't see any issue to use the "same" (not-yet-released) version. > >> -- >> Milan Kubik >> >> On a second thought, increasing the build number may be useful. I just ran into the cyclic import issue again. Until I looked into the rpm I wasn't sure if I got the newer packages. -- Milan Kubik From ssorce at redhat.com Tue Dec 15 17:22:56 2015 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 15 Dec 2015 12:22:56 -0500 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <56703075.7040108@redhat.com> References: <566FC72B.3050406@redhat.com> <56703075.7040108@redhat.com> Message-ID: <1450200176.17418.173.camel@redhat.com> On Tue, 2015-12-15 at 16:23 +0100, Martin Kosek wrote: > On 12/15/2015 08:54 AM, Jan Cholasta wrote: > > Hi, > > > > recently I and David discussed the direction of installers with regard to > > requesting certificates. Currently there are four (!) different ways of > > requesting certificates in the installer [1][2][3][4]. We would like to reduce > > it to one. > > > > Since all the certificates are tracked by certmonger and certmonger already > > knows how to request certificates from Dogtag (and other CAs), we believe that > > all certificates should be requested using certmonger. > > > > Taking our meditation further, we thought "Why not use certmonger for the > > cert-request command as well?" What is the benefit, do you ask? > > > > a) single code path for requesting certificates (seriously, the current state > > is ridiculous) > > > > b) use any CA supported by certmonger as the IPA CA (i.e. Let's Encrypt [5], > > once certmonger gains support for it) > > > > c) automate external CA install, using any CA supported by certmonger [6] > > > > d) support multiple different CAs at once (generalization of the Sub-CA feature) > > > > e) uniform configuration on clients (configure once, use forever, even for > > CA-less) > > > > The idea is to store configuration for the different CAs in LDAP and have > > cert-request redirect requests to a proper CA helper according to that > > configuration. This would require a new certmonger D-Bus method to call a CA > > helper without associated certificate storage, but that should be rather easy > > to add. In return, it would be possible to do all of the above. > > > > Note that this should not conflict with tighter integration with Dogtag > > (profiles, ACLs, etc.). > > > > Comments are welcome. > > > > Honza > > > > [1] > > > > [2] > > > > > > [3] > > > > > > [4] > > > > > > [5] > > [6] > > Interesting idea! I would be definitely interested to hear what Fraser, Rob or > Simo thinks here. Sounds great to me in principle. How do you handle CAs that do not have automatic workflows for csr handling ? That's the reason we did the 2 step process (in reference to [6]) Simo. From cheimes at redhat.com Tue Dec 15 18:10:57 2015 From: cheimes at redhat.com (Christian Heimes) Date: Tue, 15 Dec 2015 19:10:57 +0100 Subject: [Freeipa-devel] Retro Changelog for bind-dyndb-ldap Message-ID: <567057B1.4020909@redhat.com> Hi, in ticket https://fedorahosted.org/freeipa/ticket/5538 Ludwig has suggested to exclude Dogtag's o=ipaca tree from the changelog. Sometimes vault-archive fails because of a failed write to the Retro Changelog. The RetroCL was enabled in https://fedorahosted.org/freeipa/ticket/3967 for the bind-dyndb-ldap plugin. Otherwise it is not needed under normal circumstances because 389 doesn't use SyncRepl for replication. In #3967 Nathan has expressed his concerns for possible performance issues, too. Petr, Ludwig, would it makes sense to restrict RetroCL to cn=dns,$SUFFIX rather than excluding o=ipaca? The plugin supports both includes and exclude, http://directory.fedoraproject.org/docs/389ds/design/retrocl-scoping.html. Christian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From ftweedal at redhat.com Wed Dec 16 00:40:06 2015 From: ftweedal at redhat.com (Fraser Tweedale) Date: Wed, 16 Dec 2015 10:40:06 +1000 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <56703075.7040108@redhat.com> References: <566FC72B.3050406@redhat.com> <56703075.7040108@redhat.com> Message-ID: <20151216004006.GK23644@dhcp-40-8.bne.redhat.com> On Tue, Dec 15, 2015 at 04:23:33PM +0100, Martin Kosek wrote: > On 12/15/2015 08:54 AM, Jan Cholasta wrote: > > Hi, > > > > recently I and David discussed the direction of installers with regard to > > requesting certificates. Currently there are four (!) different ways of > > requesting certificates in the installer [1][2][3][4]. We would like to reduce > > it to one. > > > > Since all the certificates are tracked by certmonger and certmonger already > > knows how to request certificates from Dogtag (and other CAs), we believe that > > all certificates should be requested using certmonger. > > > > Taking our meditation further, we thought "Why not use certmonger for the > > cert-request command as well?" What is the benefit, do you ask? > > > > a) single code path for requesting certificates (seriously, the current state > > is ridiculous) > > > > b) use any CA supported by certmonger as the IPA CA (i.e. Let's Encrypt [5], > > once certmonger gains support for it) > > > > c) automate external CA install, using any CA supported by certmonger [6] > > > > d) support multiple different CAs at once (generalization of the Sub-CA feature) > > > > e) uniform configuration on clients (configure once, use forever, even for > > CA-less) > > > > The idea is to store configuration for the different CAs in LDAP and have > > cert-request redirect requests to a proper CA helper according to that > > configuration. This would require a new certmonger D-Bus method to call a CA > > helper without associated certificate storage, but that should be rather easy > > to add. In return, it would be possible to do all of the above. > > > > Note that this should not conflict with tighter integration with Dogtag > > (profiles, ACLs, etc.). > > > > Comments are welcome. > > > > Honza > > > > [1] > > > > [2] > > > > > > [3] > > > > > > [4] > > > > > > [5] > > [6] > > Interesting idea! I would be definitely interested to hear what Fraser, Rob or > Simo thinks here. > For the installer cases, +1 from me. For the general cert-request case, currently Certmonger calls `ipa cert-request' using host credentials (Kerberos), and IPA framework does some validation (e.g. ensure CN and altNames correspond to subject principal), before requesting cert from Dogtag using RA Agent credentials (TLS client cert). There are a few things that have to happen before Certmonger can request certs directly from Dogtag, in IPA scenario: 1. support SNPEGO authentication in Dogtag (work has started) 2. perform all validation that IPA framework currently performs in Dogtag, or make the validation no longer required by pulling cert data straight from LDAP according to the profile. 3. Requestor credentials must be delegated to Certmonger so that the correct principal is used to talk to Dogtag. Presumably this would be S4U2Proxy constrained delegation with host/ as impersonator and Dogtag service principal as target. At that point `ipa cert-request' can become a client-side command that merely configures Certmonger to request the cert. Cheers, Fraser From jcholast at redhat.com Wed Dec 16 06:17:22 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 16 Dec 2015 07:17:22 +0100 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <1450200176.17418.173.camel@redhat.com> References: <566FC72B.3050406@redhat.com> <56703075.7040108@redhat.com> <1450200176.17418.173.camel@redhat.com> Message-ID: <567101F2.30508@redhat.com> On 15.12.2015 18:22, Simo Sorce wrote: > On Tue, 2015-12-15 at 16:23 +0100, Martin Kosek wrote: >> On 12/15/2015 08:54 AM, Jan Cholasta wrote: >>> Hi, >>> >>> recently I and David discussed the direction of installers with regard to >>> requesting certificates. Currently there are four (!) different ways of >>> requesting certificates in the installer [1][2][3][4]. We would like to reduce >>> it to one. >>> >>> Since all the certificates are tracked by certmonger and certmonger already >>> knows how to request certificates from Dogtag (and other CAs), we believe that >>> all certificates should be requested using certmonger. >>> >>> Taking our meditation further, we thought "Why not use certmonger for the >>> cert-request command as well?" What is the benefit, do you ask? >>> >>> a) single code path for requesting certificates (seriously, the current state >>> is ridiculous) >>> >>> b) use any CA supported by certmonger as the IPA CA (i.e. Let's Encrypt [5], >>> once certmonger gains support for it) >>> >>> c) automate external CA install, using any CA supported by certmonger [6] >>> >>> d) support multiple different CAs at once (generalization of the Sub-CA feature) >>> >>> e) uniform configuration on clients (configure once, use forever, even for >>> CA-less) >>> >>> The idea is to store configuration for the different CAs in LDAP and have >>> cert-request redirect requests to a proper CA helper according to that >>> configuration. This would require a new certmonger D-Bus method to call a CA >>> helper without associated certificate storage, but that should be rather easy >>> to add. In return, it would be possible to do all of the above. >>> >>> Note that this should not conflict with tighter integration with Dogtag >>> (profiles, ACLs, etc.). >>> >>> Comments are welcome. >>> >>> Honza >>> >>> [1] >>> >>> [2] >>> >>> >>> [3] >>> >>> >>> [4] >>> >>> >>> [5] >>> [6] >> >> Interesting idea! I would be definitely interested to hear what Fraser, Rob or >> Simo thinks here. > > Sounds great to me in principle. > > How do you handle CAs that do not have automatic workflows for csr > handling ? > > That's the reason we did the 2 step process (in reference to [6]) This could be handled by a dummy "manual" CA helper in certmonger, which would dump the CSR into the filesystem and wait for the user to provide the signed certificate in the filesystem and resume the certmonger request. As you pointed out, we currently do the same, although manually, in external CA install. It is also done when renewing the externally signed CA certificate - this time it is done using certmonger, but it is handled by the dogtag-ipa-ca-renewal-agent helper rather than a separate helper. -- Jan Cholasta From jcholast at redhat.com Wed Dec 16 07:09:11 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 16 Dec 2015 08:09:11 +0100 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <20151216004006.GK23644@dhcp-40-8.bne.redhat.com> References: <566FC72B.3050406@redhat.com> <56703075.7040108@redhat.com> <20151216004006.GK23644@dhcp-40-8.bne.redhat.com> Message-ID: <56710E17.9060706@redhat.com> On 16.12.2015 01:40, Fraser Tweedale wrote: > On Tue, Dec 15, 2015 at 04:23:33PM +0100, Martin Kosek wrote: >> On 12/15/2015 08:54 AM, Jan Cholasta wrote: >>> Hi, >>> >>> recently I and David discussed the direction of installers with regard to >>> requesting certificates. Currently there are four (!) different ways of >>> requesting certificates in the installer [1][2][3][4]. We would like to reduce >>> it to one. >>> >>> Since all the certificates are tracked by certmonger and certmonger already >>> knows how to request certificates from Dogtag (and other CAs), we believe that >>> all certificates should be requested using certmonger. >>> >>> Taking our meditation further, we thought "Why not use certmonger for the >>> cert-request command as well?" What is the benefit, do you ask? >>> >>> a) single code path for requesting certificates (seriously, the current state >>> is ridiculous) >>> >>> b) use any CA supported by certmonger as the IPA CA (i.e. Let's Encrypt [5], >>> once certmonger gains support for it) >>> >>> c) automate external CA install, using any CA supported by certmonger [6] >>> >>> d) support multiple different CAs at once (generalization of the Sub-CA feature) >>> >>> e) uniform configuration on clients (configure once, use forever, even for >>> CA-less) >>> >>> The idea is to store configuration for the different CAs in LDAP and have >>> cert-request redirect requests to a proper CA helper according to that >>> configuration. This would require a new certmonger D-Bus method to call a CA >>> helper without associated certificate storage, but that should be rather easy >>> to add. In return, it would be possible to do all of the above. >>> >>> Note that this should not conflict with tighter integration with Dogtag >>> (profiles, ACLs, etc.). >>> >>> Comments are welcome. >>> >>> Honza >>> >>> [1] >>> >>> [2] >>> >>> >>> [3] >>> >>> >>> [4] >>> >>> >>> [5] >>> [6] >> >> Interesting idea! I would be definitely interested to hear what Fraser, Rob or >> Simo thinks here. >> > For the installer cases, +1 from me. > > For the general cert-request case, currently Certmonger calls `ipa > cert-request' using host credentials (Kerberos), and IPA framework > does some validation (e.g. ensure CN and altNames correspond to > subject principal), before requesting cert from Dogtag using RA > Agent credentials (TLS client cert). > > There are a few things that have to happen before Certmonger can > request certs directly from Dogtag, in IPA scenario: > > 1. support SNPEGO authentication in Dogtag (work has started) > > 2. perform all validation that IPA framework currently performs in > Dogtag, or make the validation no longer required by pulling cert > data straight from LDAP according to the profile. > > 3. Requestor credentials must be delegated to Certmonger so that the > correct principal is used to talk to Dogtag. Presumably this would > be S4U2Proxy constrained delegation with host/ as > impersonator and Dogtag service principal as target. > > At that point `ipa cert-request' can become a client-side command > that merely configures Certmonger to request the cert. I'm not proposing to change cert-request to a client side command - I'm proposing to change the way cert-request is handled *on the server*. This way we can keep all the configuration on the server and make changes to it without having to reconfigure all clients. This is how I envision the workflow: 1. client requests a certificate with "getcert request", using "IPA" as the CA and, optionally, a string identifying the sub-CA (for the lack of better term) 2. "getcert request" forwards the request to certmonger over D-Bus and exits 3. certmonger creates CSR for the request 4. certmonger executes the IPA CA helper to handle the request 5. the IPA CA helper calls the cert-request command on the server over RPC, using local host credentials for authentication 6. cert-request on the server validates the request 7. cert-request fetches the configuration for the specified sub-CA, or the default sub-CA if none was specified, from LDAP 8. cert-request forwards the request to the certmonger CA helper specified in the LDAP configuration over D-Bus (this is the D-Bus method that currently does not exist and needs to be implemented) 9. certmonger executes the specified CA helper to handle the request 10. the CA helper requests the certificate from the CA and returns either the certificate, wait delay or error 11. certmonger returns the result back to cert-request 12. cert-request returns the result back to IPA CA helper on the client 13. the IPA CA helper on the client returns the result back to certmonger 14. if the result was wait delay, certmonger waits and then retries the request from step 4, otherwise it stores the certificate or sets error status -- Jan Cholasta From pspacek at redhat.com Wed Dec 16 07:49:04 2015 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 16 Dec 2015 08:49:04 +0100 Subject: [Freeipa-devel] limiting SyncRepl's scope In-Reply-To: <567057B1.4020909@redhat.com> References: <567057B1.4020909@redhat.com> Message-ID: <56711770.60308@redhat.com> On 15.12.2015 19:10, Christian Heimes wrote: > Hi, > > in ticket https://fedorahosted.org/freeipa/ticket/5538 Ludwig has > suggested to exclude Dogtag's o=ipaca tree from the changelog. Sometimes > vault-archive fails because of a failed write to the Retro Changelog. > The RetroCL was enabled in https://fedorahosted.org/freeipa/ticket/3967 > for the bind-dyndb-ldap plugin. Otherwise it is not needed under normal > circumstances because 389 doesn't use SyncRepl for replication. In #3967 > Nathan has expressed his concerns for possible performance issues, too. > > Petr, Ludwig, > would it makes sense to restrict RetroCL to cn=dns,$SUFFIX rather than > excluding o=ipaca? The plugin supports both includes and exclude, > http://directory.fedoraproject.org/docs/389ds/design/retrocl-scoping.html. >From IPA DNS perspective it is okay to limit SyncRepl to cn=dns,$SUFFIX. One other thing to consider is theoretical use of SyncRepl for future versions of slapi-nis, Alexander can tell you more about it. In any case, if we decide to limit scope where SyncRepl is applicable, I would like to see checks in SyncRepl plugin which will ensure that error UNWILLING_TO_PERFORM is returned when somebody attempts to use SyncRepl in a 'wrong' scope. -- Petr^2 Spacek From mkosek at redhat.com Wed Dec 16 07:54:26 2015 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 16 Dec 2015 08:54:26 +0100 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <56710E17.9060706@redhat.com> References: <566FC72B.3050406@redhat.com> <56703075.7040108@redhat.com> <20151216004006.GK23644@dhcp-40-8.bne.redhat.com> <56710E17.9060706@redhat.com> Message-ID: <567118B2.9000601@redhat.com> On 12/16/2015 08:09 AM, Jan Cholasta wrote: > On 16.12.2015 01:40, Fraser Tweedale wrote: >> On Tue, Dec 15, 2015 at 04:23:33PM +0100, Martin Kosek wrote: >>> On 12/15/2015 08:54 AM, Jan Cholasta wrote: >>>> Hi, >>>> >>>> recently I and David discussed the direction of installers with regard to >>>> requesting certificates. Currently there are four (!) different ways of >>>> requesting certificates in the installer [1][2][3][4]. We would like to reduce >>>> it to one. >>>> >>>> Since all the certificates are tracked by certmonger and certmonger already >>>> knows how to request certificates from Dogtag (and other CAs), we believe that >>>> all certificates should be requested using certmonger. >>>> >>>> Taking our meditation further, we thought "Why not use certmonger for the >>>> cert-request command as well?" What is the benefit, do you ask? >>>> >>>> a) single code path for requesting certificates (seriously, the current >>>> state >>>> is ridiculous) >>>> >>>> b) use any CA supported by certmonger as the IPA CA (i.e. Let's Encrypt [5], >>>> once certmonger gains support for it) >>>> >>>> c) automate external CA install, using any CA supported by certmonger [6] >>>> >>>> d) support multiple different CAs at once (generalization of the Sub-CA >>>> feature) >>>> >>>> e) uniform configuration on clients (configure once, use forever, even for >>>> CA-less) >>>> >>>> The idea is to store configuration for the different CAs in LDAP and have >>>> cert-request redirect requests to a proper CA helper according to that >>>> configuration. This would require a new certmonger D-Bus method to call a CA >>>> helper without associated certificate storage, but that should be rather easy >>>> to add. In return, it would be possible to do all of the above. >>>> >>>> Note that this should not conflict with tighter integration with Dogtag >>>> (profiles, ACLs, etc.). >>>> >>>> Comments are welcome. >>>> >>>> Honza >>>> >>>> [1] >>>> >>>> >>>> [2] >>>> >>>> >>>> >>>> [3] >>>> >>>> >>>> >>>> [4] >>>> >>>> >>>> >>>> [5] >>>> [6] >>> >>> Interesting idea! I would be definitely interested to hear what Fraser, Rob or >>> Simo thinks here. >>> >> For the installer cases, +1 from me. >> >> For the general cert-request case, currently Certmonger calls `ipa >> cert-request' using host credentials (Kerberos), and IPA framework >> does some validation (e.g. ensure CN and altNames correspond to >> subject principal), before requesting cert from Dogtag using RA >> Agent credentials (TLS client cert). >> >> There are a few things that have to happen before Certmonger can >> request certs directly from Dogtag, in IPA scenario: >> >> 1. support SNPEGO authentication in Dogtag (work has started) >> >> 2. perform all validation that IPA framework currently performs in >> Dogtag, or make the validation no longer required by pulling cert >> data straight from LDAP according to the profile. >> >> 3. Requestor credentials must be delegated to Certmonger so that the >> correct principal is used to talk to Dogtag. Presumably this would >> be S4U2Proxy constrained delegation with host/ as >> impersonator and Dogtag service principal as target. >> >> At that point `ipa cert-request' can become a client-side command >> that merely configures Certmonger to request the cert. > > I'm not proposing to change cert-request to a client side command - I'm > proposing to change the way cert-request is handled *on the server*. I do not think Fraser was suggesting to change cert-request to a client side command. More below. > This way > we can keep all the configuration on the server and make changes to it without > having to reconfigure all clients. > > This is how I envision the workflow: > > 1. client requests a certificate with "getcert request", using "IPA" as the > CA and, optionally, a string identifying the sub-CA (for the lack of better term) > > 2. "getcert request" forwards the request to certmonger over D-Bus and exits > > 3. certmonger creates CSR for the request > > 4. certmonger executes the IPA CA helper to handle the request > > 5. the IPA CA helper calls the cert-request command on the server over RPC, > using local host credentials for authentication > > 6. cert-request on the server validates the request Right. These are the steps that happen already, AFAIK. > 7. cert-request fetches the configuration for the specified sub-CA, or the > default sub-CA if none was specified, from LDAP > > 8. cert-request forwards the request to the certmonger CA helper specified in > the LDAP configuration over D-Bus (this is the D-Bus method that currently does > not exist and needs to be implemented) > > 9. certmonger executes the specified CA helper to handle the request > > 10. the CA helper requests the certificate from the CA and returns either the > certificate, wait delay or error > > 11. certmonger returns the result back to cert-request These steps are subject to Fraser's question (and I am curious too), i.e.: - how is authentication done? certmonger runs with FreeIPA server host principal. - how will we handle 3-step certificate request, i.e.: - certificate is requested and in moderation/wait queue - request have to be acked by Dogtag administrator (we do not have API yet) - client should be able to ask for generated certificate > 12. cert-request returns the result back to IPA CA helper on the client > > 13. the IPA CA helper on the client returns the result back to certmonger > > 14. if the result was wait delay, certmonger waits and then retries the > request from step 4, otherwise it stores the certificate or sets error status > Right, 12-14 is again the standard flow. Good summary of the steps! From jcholast at redhat.com Wed Dec 16 08:17:09 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 16 Dec 2015 09:17:09 +0100 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <567118B2.9000601@redhat.com> References: <566FC72B.3050406@redhat.com> <56703075.7040108@redhat.com> <20151216004006.GK23644@dhcp-40-8.bne.redhat.com> <56710E17.9060706@redhat.com> <567118B2.9000601@redhat.com> Message-ID: <56711E05.5010102@redhat.com> On 16.12.2015 08:54, Martin Kosek wrote: > On 12/16/2015 08:09 AM, Jan Cholasta wrote: >> On 16.12.2015 01:40, Fraser Tweedale wrote: >>> On Tue, Dec 15, 2015 at 04:23:33PM +0100, Martin Kosek wrote: >>>> On 12/15/2015 08:54 AM, Jan Cholasta wrote: >>>>> Hi, >>>>> >>>>> recently I and David discussed the direction of installers with >>>>> regard to >>>>> requesting certificates. Currently there are four (!) different >>>>> ways of >>>>> requesting certificates in the installer [1][2][3][4]. We would >>>>> like to reduce >>>>> it to one. >>>>> >>>>> Since all the certificates are tracked by certmonger and certmonger >>>>> already >>>>> knows how to request certificates from Dogtag (and other CAs), we >>>>> believe that >>>>> all certificates should be requested using certmonger. >>>>> >>>>> Taking our meditation further, we thought "Why not use certmonger >>>>> for the >>>>> cert-request command as well?" What is the benefit, do you ask? >>>>> >>>>> a) single code path for requesting certificates (seriously, the >>>>> current >>>>> state >>>>> is ridiculous) >>>>> >>>>> b) use any CA supported by certmonger as the IPA CA (i.e. Let's >>>>> Encrypt [5], >>>>> once certmonger gains support for it) >>>>> >>>>> c) automate external CA install, using any CA supported by >>>>> certmonger [6] >>>>> >>>>> d) support multiple different CAs at once (generalization of the >>>>> Sub-CA >>>>> feature) >>>>> >>>>> e) uniform configuration on clients (configure once, use forever, >>>>> even for >>>>> CA-less) >>>>> >>>>> The idea is to store configuration for the different CAs in LDAP >>>>> and have >>>>> cert-request redirect requests to a proper CA helper according to that >>>>> configuration. This would require a new certmonger D-Bus method to >>>>> call a CA >>>>> helper without associated certificate storage, but that should be >>>>> rather easy >>>>> to add. In return, it would be possible to do all of the above. >>>>> >>>>> Note that this should not conflict with tighter integration with >>>>> Dogtag >>>>> (profiles, ACLs, etc.). >>>>> >>>>> Comments are welcome. >>>>> >>>>> Honza >>>>> >>>>> [1] >>>>> >>>>> >>>>> >>>>> [2] >>>>> >>>>> >>>>> >>>>> >>>>> [3] >>>>> >>>>> >>>>> >>>>> >>>>> [4] >>>>> >>>>> >>>>> >>>>> >>>>> [5] >>>>> [6] >>>> >>>> Interesting idea! I would be definitely interested to hear what >>>> Fraser, Rob or >>>> Simo thinks here. >>>> >>> For the installer cases, +1 from me. >>> >>> For the general cert-request case, currently Certmonger calls `ipa >>> cert-request' using host credentials (Kerberos), and IPA framework >>> does some validation (e.g. ensure CN and altNames correspond to >>> subject principal), before requesting cert from Dogtag using RA >>> Agent credentials (TLS client cert). >>> >>> There are a few things that have to happen before Certmonger can >>> request certs directly from Dogtag, in IPA scenario: >>> >>> 1. support SNPEGO authentication in Dogtag (work has started) >>> >>> 2. perform all validation that IPA framework currently performs in >>> Dogtag, or make the validation no longer required by pulling cert >>> data straight from LDAP according to the profile. >>> >>> 3. Requestor credentials must be delegated to Certmonger so that the >>> correct principal is used to talk to Dogtag. Presumably this would >>> be S4U2Proxy constrained delegation with host/ as >>> impersonator and Dogtag service principal as target. >>> >>> At that point `ipa cert-request' can become a client-side command >>> that merely configures Certmonger to request the cert. >> >> I'm not proposing to change cert-request to a client side command - I'm >> proposing to change the way cert-request is handled *on the server*. > > I do not think Fraser was suggesting to change cert-request to a client > side command. More below. See 3 paragraphs above: "... `ipa cert-request' can become a client-side command...". > >> This way >> we can keep all the configuration on the server and make changes to it >> without >> having to reconfigure all clients. >> >> This is how I envision the workflow: >> >> 1. client requests a certificate with "getcert request", using "IPA" >> as the >> CA and, optionally, a string identifying the sub-CA (for the lack of >> better term) >> >> 2. "getcert request" forwards the request to certmonger over D-Bus >> and exits >> >> 3. certmonger creates CSR for the request >> >> 4. certmonger executes the IPA CA helper to handle the request >> >> 5. the IPA CA helper calls the cert-request command on the server >> over RPC, >> using local host credentials for authentication >> >> 6. cert-request on the server validates the request > > Right. These are the steps that happen already, AFAIK. Correct. > >> 7. cert-request fetches the configuration for the specified sub-CA, >> or the >> default sub-CA if none was specified, from LDAP >> >> 8. cert-request forwards the request to the certmonger CA helper >> specified in >> the LDAP configuration over D-Bus (this is the D-Bus method that >> currently does >> not exist and needs to be implemented) >> >> 9. certmonger executes the specified CA helper to handle the request >> >> 10. the CA helper requests the certificate from the CA and returns >> either the >> certificate, wait delay or error >> >> 11. certmonger returns the result back to cert-request > > These steps are subject to Fraser's question (and I am curious too), i.e.: > > - how is authentication done? certmonger runs with FreeIPA server host > principal. We are on the server, so the RA agent cert is used to authenticate to Dogtag as usual, and whatever authentication is configured for other CAs is used for other CAs. > - how will we handle 3-step certificate request, i.e.: > - certificate is requested and in moderation/wait queue > - request have to be acked by Dogtag administrator (we do not have > API yet) > - client should be able to ask for generated certificate This is not really related to my proposal, since we have to figure this out for our Dogtag IPA CA anyway, but the CA helper can return a wait delay in this case, so certmonger can poll the request until it is approved. > >> 12. cert-request returns the result back to IPA CA helper on the client >> >> 13. the IPA CA helper on the client returns the result back to >> certmonger >> >> 14. if the result was wait delay, certmonger waits and then retries the >> request from step 4, otherwise it stores the certificate or sets error >> status >> > > Right, 12-14 is again the standard flow. Good summary of the steps! -- Jan Cholasta From sbose at redhat.com Wed Dec 16 08:26:11 2015 From: sbose at redhat.com (Sumit Bose) Date: Wed, 16 Dec 2015 09:26:11 +0100 Subject: [Freeipa-devel] limiting SyncRepl's scope In-Reply-To: <56711770.60308@redhat.com> References: <567057B1.4020909@redhat.com> <56711770.60308@redhat.com> Message-ID: <20151216082611.GF24928@p.redhat.com> On Wed, Dec 16, 2015 at 08:49:04AM +0100, Petr Spacek wrote: > On 15.12.2015 19:10, Christian Heimes wrote: > > Hi, > > > > in ticket https://fedorahosted.org/freeipa/ticket/5538 Ludwig has > > suggested to exclude Dogtag's o=ipaca tree from the changelog. Sometimes > > vault-archive fails because of a failed write to the Retro Changelog. > > The RetroCL was enabled in https://fedorahosted.org/freeipa/ticket/3967 > > for the bind-dyndb-ldap plugin. Otherwise it is not needed under normal > > circumstances because 389 doesn't use SyncRepl for replication. In #3967 > > Nathan has expressed his concerns for possible performance issues, too. > > > > Petr, Ludwig, > > would it makes sense to restrict RetroCL to cn=dns,$SUFFIX rather than > > excluding o=ipaca? The plugin supports both includes and exclude, > > http://directory.fedoraproject.org/docs/389ds/design/retrocl-scoping.html. > > >From IPA DNS perspective it is okay to limit SyncRepl to cn=dns,$SUFFIX. > > One other thing to consider is theoretical use of SyncRepl for future versions > of slapi-nis, Alexander can tell you more about it. > > In any case, if we decide to limit scope where SyncRepl is applicable, I would > like to see checks in SyncRepl plugin which will ensure that error > UNWILLING_TO_PERFORM is returned when somebody attempts to use SyncRepl in a > 'wrong' scope. > There are discussions about using SyncRepl in SSSD as well which would include users, groups, sudo and HBAC rules, trusted domains, ... But afaik no work in the direction has been started yet, so it might be ok to limit the scope for now and add it when there are patches for SSSD which really try to use it. bye, Sumit From lkrispen at redhat.com Wed Dec 16 08:35:08 2015 From: lkrispen at redhat.com (Ludwig Krispenz) Date: Wed, 16 Dec 2015 09:35:08 +0100 Subject: [Freeipa-devel] limiting SyncRepl's scope In-Reply-To: <56711770.60308@redhat.com> References: <567057B1.4020909@redhat.com> <56711770.60308@redhat.com> Message-ID: <5671223C.8080800@redhat.com> On 12/16/2015 08:49 AM, Petr Spacek wrote: > On 15.12.2015 19:10, Christian Heimes wrote: >> Hi, >> >> in ticket https://fedorahosted.org/freeipa/ticket/5538 Ludwig has >> suggested to exclude Dogtag's o=ipaca tree from the changelog. Sometimes >> vault-archive fails because of a failed write to the Retro Changelog. >> The RetroCL was enabled in https://fedorahosted.org/freeipa/ticket/3967 >> for the bind-dyndb-ldap plugin. Otherwise it is not needed under normal >> circumstances because 389 doesn't use SyncRepl for replication. In #3967 >> Nathan has expressed his concerns for possible performance issues, too. >> >> Petr, Ludwig, >> would it makes sense to restrict RetroCL to cn=dns,$SUFFIX rather than >> excluding o=ipaca? The plugin supports both includes and exclude, >> http://directory.fedoraproject.org/docs/389ds/design/retrocl-scoping.html. > From IPA DNS perspective it is okay to limit SyncRepl to cn=dns,$SUFFIX. > > One other thing to consider is theoretical use of SyncRepl for future versions > of slapi-nis, Alexander can tell you more about it. > > In any case, if we decide to limit scope where SyncRepl is applicable, I would > like to see checks in SyncRepl plugin which will ensure that error > UNWILLING_TO_PERFORM is returned when somebody attempts to use SyncRepl in a > 'wrong' scope. yes, that makes sense > From pspacek at redhat.com Wed Dec 16 08:45:10 2015 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 16 Dec 2015 09:45:10 +0100 Subject: [Freeipa-devel] [PATCH 0069] Add 'review' target for make In-Reply-To: <566AE2B3.3010201@redhat.com> References: <56683B03.40703@redhat.com> <5669B09B.9000001@redhat.com> <566AE2B3.3010201@redhat.com> Message-ID: <56712496.5060907@redhat.com> On 11.12.2015 15:50, Jan Cholasta wrote: > Hi, > > On 10.12.2015 18:04, Petr Spacek wrote: >> On 9.12.2015 15:30, Petr Spacek wrote: >>> Hello, >>> >>> this patch automates some of sanity checks proposed by Petr Vobornik. >>> >>> 'make review' should be used in root of clean Git tree which has patches under >>> review applied. >>> >>> Magic in review.sh attempts to detect nearest remote branch which can be used >>> as diff base for review. Please see review.sh for further details. >> >> And here is the patch! :-) > > Nice, but I would rather see this in ipatool > (). Or is there any obvious benefit > in having this in freeipa itself that I'm missing? For me the obvious benefit is: git clone git am make review Done. No need to find & learn other tool, no risk of using wrong version of the tool to wrong version of source tree etc. -- Petr^2 Spacek From jcholast at redhat.com Wed Dec 16 08:53:35 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 16 Dec 2015 09:53:35 +0100 Subject: [Freeipa-devel] [PATCH 0069] Add 'review' target for make In-Reply-To: <56712496.5060907@redhat.com> References: <56683B03.40703@redhat.com> <5669B09B.9000001@redhat.com> <566AE2B3.3010201@redhat.com> <56712496.5060907@redhat.com> Message-ID: <5671268F.8020302@redhat.com> On 16.12.2015 09:45, Petr Spacek wrote: > On 11.12.2015 15:50, Jan Cholasta wrote: >> Hi, >> >> On 10.12.2015 18:04, Petr Spacek wrote: >>> On 9.12.2015 15:30, Petr Spacek wrote: >>>> Hello, >>>> >>>> this patch automates some of sanity checks proposed by Petr Vobornik. >>>> >>>> 'make review' should be used in root of clean Git tree which has patches under >>>> review applied. >>>> >>>> Magic in review.sh attempts to detect nearest remote branch which can be used >>>> as diff base for review. Please see review.sh for further details. >>> >>> And here is the patch! :-) >> >> Nice, but I would rather see this in ipatool >> (). Or is there any obvious benefit >> in having this in freeipa itself that I'm missing? > > For me the obvious benefit is: > git clone > git am > make review > > Done. > > No need to find & learn other tool, no risk of using wrong version of the tool > to wrong version of source tree etc. AFAIK all IPA developers are supposed to use ipatool, and it already has a start-review command, so it would better fit in there. Or we could merge freeipa-tools into freeipa. My point is that I don't think having half of the stuff in ipatool and the other half in IPA itself is a good thing to do. -- Jan Cholasta From pspacek at redhat.com Wed Dec 16 09:02:42 2015 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 16 Dec 2015 10:02:42 +0100 Subject: [Freeipa-devel] [PATCH 0069] Add 'review' target for make In-Reply-To: <5671268F.8020302@redhat.com> References: <56683B03.40703@redhat.com> <5669B09B.9000001@redhat.com> <566AE2B3.3010201@redhat.com> <56712496.5060907@redhat.com> <5671268F.8020302@redhat.com> Message-ID: <567128B2.80009@redhat.com> On 16.12.2015 09:53, Jan Cholasta wrote: > On 16.12.2015 09:45, Petr Spacek wrote: >> On 11.12.2015 15:50, Jan Cholasta wrote: >>> Hi, >>> >>> On 10.12.2015 18:04, Petr Spacek wrote: >>>> On 9.12.2015 15:30, Petr Spacek wrote: >>>>> Hello, >>>>> >>>>> this patch automates some of sanity checks proposed by Petr Vobornik. >>>>> >>>>> 'make review' should be used in root of clean Git tree which has patches >>>>> under >>>>> review applied. >>>>> >>>>> Magic in review.sh attempts to detect nearest remote branch which can be >>>>> used >>>>> as diff base for review. Please see review.sh for further details. >>>> >>>> And here is the patch! :-) >>> >>> Nice, but I would rather see this in ipatool >>> (). Or is there any obvious benefit >>> in having this in freeipa itself that I'm missing? >> >> For me the obvious benefit is: >> git clone >> git am >> make review >> >> Done. >> >> No need to find & learn other tool, no risk of using wrong version of the tool >> to wrong version of source tree etc. > > AFAIK all IPA developers are supposed to use ipatool, and it already has a Good to know. How does a newcomer learn about it? Honestly I never used ipatool (or not even cloned it). > start-review command, so it would better fit in there. Or we could merge > freeipa-tools into freeipa. My point is that I don't think having half of the > stuff in ipatool and the other half in IPA itself is a good thing to do. I agree with this in general. Would it make sense to at least have review target for make which executes ipa-tool and if it is not installed it tells you where to grab it? Or possibly make ipatool submodule of ipa git tree, so there is no risk of using wrong review tool for particular checkout? -- Petr^2 Spacek From jhrozek at redhat.com Wed Dec 16 09:10:18 2015 From: jhrozek at redhat.com (Jakub Hrozek) Date: Wed, 16 Dec 2015 10:10:18 +0100 Subject: [Freeipa-devel] limiting SyncRepl's scope In-Reply-To: <20151216082611.GF24928@p.redhat.com> References: <567057B1.4020909@redhat.com> <56711770.60308@redhat.com> <20151216082611.GF24928@p.redhat.com> Message-ID: <20151216091018.GB4733@hendrix> On Wed, Dec 16, 2015 at 09:26:11AM +0100, Sumit Bose wrote: > On Wed, Dec 16, 2015 at 08:49:04AM +0100, Petr Spacek wrote: > > On 15.12.2015 19:10, Christian Heimes wrote: > > > Hi, > > > > > > in ticket https://fedorahosted.org/freeipa/ticket/5538 Ludwig has > > > suggested to exclude Dogtag's o=ipaca tree from the changelog. Sometimes > > > vault-archive fails because of a failed write to the Retro Changelog. > > > The RetroCL was enabled in https://fedorahosted.org/freeipa/ticket/3967 > > > for the bind-dyndb-ldap plugin. Otherwise it is not needed under normal > > > circumstances because 389 doesn't use SyncRepl for replication. In #3967 > > > Nathan has expressed his concerns for possible performance issues, too. > > > > > > Petr, Ludwig, > > > would it makes sense to restrict RetroCL to cn=dns,$SUFFIX rather than > > > excluding o=ipaca? The plugin supports both includes and exclude, > > > http://directory.fedoraproject.org/docs/389ds/design/retrocl-scoping.html. > > > > >From IPA DNS perspective it is okay to limit SyncRepl to cn=dns,$SUFFIX. > > > > One other thing to consider is theoretical use of SyncRepl for future versions > > of slapi-nis, Alexander can tell you more about it. > > > > In any case, if we decide to limit scope where SyncRepl is applicable, I would > > like to see checks in SyncRepl plugin which will ensure that error > > UNWILLING_TO_PERFORM is returned when somebody attempts to use SyncRepl in a > > 'wrong' scope. > > > > There are discussions about using SyncRepl in SSSD as well which would > include users, groups, sudo and HBAC rules, trusted domains, ... But > afaik no work in the direction has been started yet, so it might be ok > to limit the scope for now and add it when there are patches for SSSD > which really try to use it. The more I was looking into the sssd performance problems in the last couple of weeks, the more I think we don't actually need syncrepl on the clients, maybe only in server mode sssd.. Even the refreshOnly mode has cost associated (IIRC Ludwig told me the server has to check all changelog entries since the cookie) and I think on the clients we could improve performance with looking up entries as we do now, checking if the modifyTimestamp has changed and if not, avoid the cache write as we discussed over the phone the other day. For server mode sssd, syncrepl might be interesting, yes. But as you said, so far I only looked into issues that would also benefit the pure client case. From lslebodn at redhat.com Wed Dec 16 09:30:04 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Wed, 16 Dec 2015 10:30:04 +0100 Subject: [Freeipa-devel] Testing FreeIPA 4.3 for GA In-Reply-To: <20151215073255.GA17683@mail.corp.redhat.com> References: <566F595E.9070807@redhat.com> <20151215073255.GA17683@mail.corp.redhat.com> Message-ID: <20151216093003.GA5575@mail.corp.redhat.com> On (15/12/15 08:32), Lukas Slebodnik wrote: >On (15/12/15 01:05), Petr Vobornik wrote: >>Blocking patches for FreeIPA 4.3 were pushed, ipa-4-3 branch was created. >>Master branch is ready for 4.4 development. >> >>A build is available for testing in my pvoborni/freeipa-4-3 COPR repo [1] >>until the official 4.3 repo is created. The repo contains only this build. >>The build is not pure upstream ipa-4-3 branch but rather a build which will >>go to Fedora rawhide and official 4.3 copr repo - Fedora downstream spec with >>the SELinux workaround applied [2][3]. >> >>Known issues: >>* https://fedorahosted.org/freeipa/ticket/5469 - requires fix in PKI >>* https://bugzilla.redhat.com/show_bug.cgi?id=1289930 - SELinux Policy update >>needed for connection check >> >>I have started drafting release page [4]. >> >It looks like there is a bug in dnf, because it cannot install >freeipa-tests package; which was renamed to python2-ipatests. > >[root at 3a8359f9e1cd ~]# rpm -q freeipa-server >freeipa-server-4.3.0-1.fc23.x86_64 > > >[root at 3a8359f9e1cd ~]# dnf install -y --best freeipa-tests >Last metadata expiration check performed 0:09:23 ago on Tue Dec 15 07:16:26 >2015. >Error: package freeipa-tests-4.2.3-1.1.fc23.x86_64 requires freeipa-client = >4.2.3-1.1.fc23, but none of the providers can be installed. >package freeipa-tests-4.2.2-1.fc23.x86_64 requires freeipa-python = >4.2.2-1.fc23, but none of the providers can be installed >(try to add '--allowerasing' to command line to replace conflicting packages) > > >[root at 3a8359f9e1cd ~]# rpm -qp --provides >./python2-ipatests-4.3.0-1.fc23.noarch.rpm >freeipa-tests(x86-64) = 4.3.0-1.fc23 >ipa-tests(x86-64) = 4.3.0 >python-ipatests = 4.3.0-1.fc23 >python2-ipatests = 4.3.0-1.fc23 > > >FYI: It works with yum-deprecated. > dnf bug https://bugzilla.redhat.com/show_bug.cgi?id=1291850 LS From mkosek at redhat.com Wed Dec 16 10:11:42 2015 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 16 Dec 2015 11:11:42 +0100 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <56711E05.5010102@redhat.com> References: <566FC72B.3050406@redhat.com> <56703075.7040108@redhat.com> <20151216004006.GK23644@dhcp-40-8.bne.redhat.com> <56710E17.9060706@redhat.com> <567118B2.9000601@redhat.com> <56711E05.5010102@redhat.com> Message-ID: <567138DE.3000401@redhat.com> On 12/16/2015 09:17 AM, Jan Cholasta wrote: > On 16.12.2015 08:54, Martin Kosek wrote: ... >>> 7. cert-request fetches the configuration for the specified sub-CA, >>> or the >>> default sub-CA if none was specified, from LDAP >>> >>> 8. cert-request forwards the request to the certmonger CA helper >>> specified in >>> the LDAP configuration over D-Bus (this is the D-Bus method that >>> currently does >>> not exist and needs to be implemented) >>> >>> 9. certmonger executes the specified CA helper to handle the request >>> >>> 10. the CA helper requests the certificate from the CA and returns >>> either the >>> certificate, wait delay or error >>> >>> 11. certmonger returns the result back to cert-request >> >> These steps are subject to Fraser's question (and I am curious too), i.e.: >> >> - how is authentication done? certmonger runs with FreeIPA server host >> principal. > > We are on the server, so the RA agent cert is used to authenticate to Dogtag as > usual, and whatever authentication is configured for other CAs is used for > other CAs. Right, this is how it works now. However, in FreeIPA 4.4 or later, we plan to switch GSSAPI authentication with Dogtag to get better authorization capabilities: https://fedorahosted.org/freeipa/ticket/5011 But maybe this could be done via S4U2Proxy as Fraser suggested, although in this case it would be more complicated as certmonger itself does not have access to user HTTP/ipa.server ticket, like Apache does, given that Apache would contact certmonger via DBUS. > >> - how will we handle 3-step certificate request, i.e.: >> - certificate is requested and in moderation/wait queue >> - request have to be acked by Dogtag administrator (we do not have >> API yet) >> - client should be able to ask for generated certificate > > This is not really related to my proposal, since we have to figure this out for > our Dogtag IPA CA anyway, but the CA helper can return a wait delay in this > case, so certmonger can poll the request until it is approved. Ok. >>> 12. cert-request returns the result back to IPA CA helper on the client >>> >>> 13. the IPA CA helper on the client returns the result back to >>> certmonger >>> >>> 14. if the result was wait delay, certmonger waits and then retries the >>> request from step 4, otherwise it stores the certificate or sets error >>> status >>> >> >> Right, 12-14 is again the standard flow. Good summary of the steps! > > From mkosek at redhat.com Wed Dec 16 10:15:44 2015 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 16 Dec 2015 11:15:44 +0100 Subject: [Freeipa-devel] [PATCH 0069] Add 'review' target for make In-Reply-To: <567128B2.80009@redhat.com> References: <56683B03.40703@redhat.com> <5669B09B.9000001@redhat.com> <566AE2B3.3010201@redhat.com> <56712496.5060907@redhat.com> <5671268F.8020302@redhat.com> <567128B2.80009@redhat.com> Message-ID: <567139D0.7050807@redhat.com> On 12/16/2015 10:02 AM, Petr Spacek wrote: > On 16.12.2015 09:53, Jan Cholasta wrote: >> On 16.12.2015 09:45, Petr Spacek wrote: >>> On 11.12.2015 15:50, Jan Cholasta wrote: >>>> Hi, >>>> >>>> On 10.12.2015 18:04, Petr Spacek wrote: >>>>> On 9.12.2015 15:30, Petr Spacek wrote: >>>>>> Hello, >>>>>> >>>>>> this patch automates some of sanity checks proposed by Petr Vobornik. >>>>>> >>>>>> 'make review' should be used in root of clean Git tree which has patches >>>>>> under >>>>>> review applied. >>>>>> >>>>>> Magic in review.sh attempts to detect nearest remote branch which can be >>>>>> used >>>>>> as diff base for review. Please see review.sh for further details. >>>>> >>>>> And here is the patch! :-) >>>> >>>> Nice, but I would rather see this in ipatool >>>> (). Or is there any obvious benefit >>>> in having this in freeipa itself that I'm missing? >>> >>> For me the obvious benefit is: >>> git clone >>> git am >>> make review >>> >>> Done. >>> >>> No need to find & learn other tool, no risk of using wrong version of the tool >>> to wrong version of source tree etc. >> >> AFAIK all IPA developers are supposed to use ipatool, and it already has a > > Good to know. How does a newcomer learn about it? Honestly I never used > ipatool (or not even cloned it). ipatool targets rather upstream members with write access, so they are hardly newcomers. But still, here you go: https://www.freeipa.org/page/Contribute/Repository#Process_tools >> start-review command, so it would better fit in there. Or we could merge >> freeipa-tools into freeipa. My point is that I don't think having half of the >> stuff in ipatool and the other half in IPA itself is a good thing to do. > > I agree with this in general. > > Would it make sense to at least have review target for make which executes > ipa-tool and if it is not installed it tells you where to grab it? > > Or possibly make ipatool submodule of ipa git tree, so there is no risk of > using wrong review tool for particular checkout? Please do not overcomplicate it :-) ipatool works nicely at the moment, it is in a separate repo with other tools where every core developer can contribute and is easy to be update. From pspacek at redhat.com Wed Dec 16 11:01:33 2015 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 16 Dec 2015 12:01:33 +0100 Subject: [Freeipa-devel] [PATCH 0069] Add 'review' target for make In-Reply-To: <567139D0.7050807@redhat.com> References: <56683B03.40703@redhat.com> <5669B09B.9000001@redhat.com> <566AE2B3.3010201@redhat.com> <56712496.5060907@redhat.com> <5671268F.8020302@redhat.com> <567128B2.80009@redhat.com> <567139D0.7050807@redhat.com> Message-ID: <5671448D.1050102@redhat.com> On 16.12.2015 11:15, Martin Kosek wrote: > On 12/16/2015 10:02 AM, Petr Spacek wrote: >> On 16.12.2015 09:53, Jan Cholasta wrote: >>> On 16.12.2015 09:45, Petr Spacek wrote: >>>> On 11.12.2015 15:50, Jan Cholasta wrote: >>>>> Hi, >>>>> >>>>> On 10.12.2015 18:04, Petr Spacek wrote: >>>>>> On 9.12.2015 15:30, Petr Spacek wrote: >>>>>>> Hello, >>>>>>> >>>>>>> this patch automates some of sanity checks proposed by Petr Vobornik. >>>>>>> >>>>>>> 'make review' should be used in root of clean Git tree which has patches >>>>>>> under >>>>>>> review applied. >>>>>>> >>>>>>> Magic in review.sh attempts to detect nearest remote branch which can be >>>>>>> used >>>>>>> as diff base for review. Please see review.sh for further details. >>>>>> >>>>>> And here is the patch! :-) >>>>> >>>>> Nice, but I would rather see this in ipatool >>>>> (). Or is there any obvious benefit >>>>> in having this in freeipa itself that I'm missing? >>>> >>>> For me the obvious benefit is: >>>> git clone >>>> git am >>>> make review >>>> >>>> Done. >>>> >>>> No need to find & learn other tool, no risk of using wrong version of the tool >>>> to wrong version of source tree etc. >>> >>> AFAIK all IPA developers are supposed to use ipatool, and it already has a >> >> Good to know. How does a newcomer learn about it? Honestly I never used >> ipatool (or not even cloned it). > > ipatool targets rather upstream members with write access, so they are hardly > newcomers. I though that we want to make it easy to contribute, so why are you talking about core developers? Shouldn't we make it easy to self-review own patches for everyone? Including random people who want to submit few patches and go away? (Think how we can apply usability principles to development.) Petr^2 Spacek > But still, here you go: > https://www.freeipa.org/page/Contribute/Repository#Process_tools > >>> start-review command, so it would better fit in there. Or we could merge >>> freeipa-tools into freeipa. My point is that I don't think having half of the >>> stuff in ipatool and the other half in IPA itself is a good thing to do. >> >> I agree with this in general. >> >> Would it make sense to at least have review target for make which executes >> ipa-tool and if it is not installed it tells you where to grab it? >> >> Or possibly make ipatool submodule of ipa git tree, so there is no risk of >> using wrong review tool for particular checkout? > > Please do not overcomplicate it :-) ipatool works nicely at the moment, it is > in a separate repo with other tools where every core developer can contribute > and is easy to be update. From ftweedal at redhat.com Wed Dec 16 11:18:19 2015 From: ftweedal at redhat.com (Fraser Tweedale) Date: Wed, 16 Dec 2015 21:18:19 +1000 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <56711E05.5010102@redhat.com> References: <566FC72B.3050406@redhat.com> <56703075.7040108@redhat.com> <20151216004006.GK23644@dhcp-40-8.bne.redhat.com> <56710E17.9060706@redhat.com> <567118B2.9000601@redhat.com> <56711E05.5010102@redhat.com> Message-ID: <20151216111819.GM23644@dhcp-40-8.bne.redhat.com> On Wed, Dec 16, 2015 at 09:17:09AM +0100, Jan Cholasta wrote: > On 16.12.2015 08:54, Martin Kosek wrote: > >On 12/16/2015 08:09 AM, Jan Cholasta wrote: > >>On 16.12.2015 01:40, Fraser Tweedale wrote: > >>>On Tue, Dec 15, 2015 at 04:23:33PM +0100, Martin Kosek wrote: > >>>>On 12/15/2015 08:54 AM, Jan Cholasta wrote: > >>>>>Hi, > >>>>> > >>>>>recently I and David discussed the direction of installers with > >>>>>regard to > >>>>>requesting certificates. Currently there are four (!) different > >>>>>ways of > >>>>>requesting certificates in the installer [1][2][3][4]. We would > >>>>>like to reduce > >>>>>it to one. > >>>>> > >>>>>Since all the certificates are tracked by certmonger and certmonger > >>>>>already > >>>>>knows how to request certificates from Dogtag (and other CAs), we > >>>>>believe that > >>>>>all certificates should be requested using certmonger. > >>>>> > >>>>>Taking our meditation further, we thought "Why not use certmonger > >>>>>for the > >>>>>cert-request command as well?" What is the benefit, do you ask? > >>>>> > >>>>> a) single code path for requesting certificates (seriously, the > >>>>>current > >>>>>state > >>>>>is ridiculous) > >>>>> > >>>>> b) use any CA supported by certmonger as the IPA CA (i.e. Let's > >>>>>Encrypt [5], > >>>>>once certmonger gains support for it) > >>>>> > >>>>> c) automate external CA install, using any CA supported by > >>>>>certmonger [6] > >>>>> > >>>>> d) support multiple different CAs at once (generalization of the > >>>>>Sub-CA > >>>>>feature) > >>>>> > >>>>> e) uniform configuration on clients (configure once, use forever, > >>>>>even for > >>>>>CA-less) > >>>>> > >>>>>The idea is to store configuration for the different CAs in LDAP > >>>>>and have > >>>>>cert-request redirect requests to a proper CA helper according to that > >>>>>configuration. This would require a new certmonger D-Bus method to > >>>>>call a CA > >>>>>helper without associated certificate storage, but that should be > >>>>>rather easy > >>>>>to add. In return, it would be possible to do all of the above. > >>>>> > >>>>>Note that this should not conflict with tighter integration with > >>>>>Dogtag > >>>>>(profiles, ACLs, etc.). > >>>>> > >>>>>Comments are welcome. > >>>>> > >>>>>Honza > >>>>> > >>>>>[1] > >>>>> > >>>>> > >>>>> > >>>>>[2] > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>[3] > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>[4] > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>[5] > >>>>>[6] > >>>> > >>>>Interesting idea! I would be definitely interested to hear what > >>>>Fraser, Rob or > >>>>Simo thinks here. > >>>> > >>>For the installer cases, +1 from me. > >>> > >>>For the general cert-request case, currently Certmonger calls `ipa > >>>cert-request' using host credentials (Kerberos), and IPA framework > >>>does some validation (e.g. ensure CN and altNames correspond to > >>>subject principal), before requesting cert from Dogtag using RA > >>>Agent credentials (TLS client cert). > >>> > >>>There are a few things that have to happen before Certmonger can > >>>request certs directly from Dogtag, in IPA scenario: > >>> > >>>1. support SNPEGO authentication in Dogtag (work has started) > >>> > >>>2. perform all validation that IPA framework currently performs in > >>> Dogtag, or make the validation no longer required by pulling cert > >>> data straight from LDAP according to the profile. > >>> > >>>3. Requestor credentials must be delegated to Certmonger so that the > >>> correct principal is used to talk to Dogtag. Presumably this would > >>> be S4U2Proxy constrained delegation with host/ as > >>> impersonator and Dogtag service principal as target. > >>> > >>>At that point `ipa cert-request' can become a client-side command > >>>that merely configures Certmonger to request the cert. > >> > >>I'm not proposing to change cert-request to a client side command - I'm > >>proposing to change the way cert-request is handled *on the server*. > > > >I do not think Fraser was suggesting to change cert-request to a client > >side command. More below. > > See 3 paragraphs above: "... `ipa cert-request' can become a client-side > command...". > Indeed; my commentary was based on misunderstanding that Jan was proposing Certmonger on would talk to Dogtag. Thanks for clarifying, Jan. > > > >>This way > >>we can keep all the configuration on the server and make changes to it > >>without > >>having to reconfigure all clients. > >> > >>This is how I envision the workflow: > >> > >> 1. client requests a certificate with "getcert request", using "IPA" > >>as the > >>CA and, optionally, a string identifying the sub-CA (for the lack of > >>better term) > >> > >> 2. "getcert request" forwards the request to certmonger over D-Bus > >>and exits > >> > >> 3. certmonger creates CSR for the request > >> > >> 4. certmonger executes the IPA CA helper to handle the request > >> > >> 5. the IPA CA helper calls the cert-request command on the server > >>over RPC, > >>using local host credentials for authentication > >> > >> 6. cert-request on the server validates the request > > > >Right. These are the steps that happen already, AFAIK. > > Correct. > > > > >> 7. cert-request fetches the configuration for the specified sub-CA, > >>or the > >>default sub-CA if none was specified, from LDAP > >> > >> 8. cert-request forwards the request to the certmonger CA helper > >>specified in > >>the LDAP configuration over D-Bus (this is the D-Bus method that > >>currently does > >>not exist and needs to be implemented) > >> > >> 9. certmonger executes the specified CA helper to handle the request > >> > >> 10. the CA helper requests the certificate from the CA and returns > >>either the > >>certificate, wait delay or error > >> > >> 11. certmonger returns the result back to cert-request > > > >These steps are subject to Fraser's question (and I am curious too), i.e.: > > > >- how is authentication done? certmonger runs with FreeIPA server host > >principal. > > We are on the server, so the RA agent cert is used to authenticate to Dogtag > as usual, and whatever authentication is configured for other CAs is used > for other CAs. > > >- how will we handle 3-step certificate request, i.e.: > > - certificate is requested and in moderation/wait queue > > - request have to be acked by Dogtag administrator (we do not have > >API yet) > > - client should be able to ask for generated certificate > > This is not really related to my proposal, since we have to figure this out > for our Dogtag IPA CA anyway, but the CA helper can return a wait delay in > this case, so certmonger can poll the request until it is approved. > > > > >> 12. cert-request returns the result back to IPA CA helper on the client > >> > >> 13. the IPA CA helper on the client returns the result back to > >>certmonger > >> > >> 14. if the result was wait delay, certmonger waits and then retries the > >>request from step 4, otherwise it stores the certificate or sets error > >>status > >> > > > >Right, 12-14 is again the standard flow. Good summary of the steps! > > > -- > Jan Cholasta From mkosek at redhat.com Wed Dec 16 11:24:25 2015 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 16 Dec 2015 12:24:25 +0100 Subject: [Freeipa-devel] [PATCH 0069] Add 'review' target for make In-Reply-To: <5671448D.1050102@redhat.com> References: <56683B03.40703@redhat.com> <5669B09B.9000001@redhat.com> <566AE2B3.3010201@redhat.com> <56712496.5060907@redhat.com> <5671268F.8020302@redhat.com> <567128B2.80009@redhat.com> <567139D0.7050807@redhat.com> <5671448D.1050102@redhat.com> Message-ID: <567149E9.1040506@redhat.com> On 12/16/2015 12:01 PM, Petr Spacek wrote: > On 16.12.2015 11:15, Martin Kosek wrote: >> On 12/16/2015 10:02 AM, Petr Spacek wrote: >>> On 16.12.2015 09:53, Jan Cholasta wrote: >>>> On 16.12.2015 09:45, Petr Spacek wrote: >>>>> On 11.12.2015 15:50, Jan Cholasta wrote: >>>>>> Hi, >>>>>> >>>>>> On 10.12.2015 18:04, Petr Spacek wrote: >>>>>>> On 9.12.2015 15:30, Petr Spacek wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> this patch automates some of sanity checks proposed by Petr Vobornik. >>>>>>>> >>>>>>>> 'make review' should be used in root of clean Git tree which has patches >>>>>>>> under >>>>>>>> review applied. >>>>>>>> >>>>>>>> Magic in review.sh attempts to detect nearest remote branch which can be >>>>>>>> used >>>>>>>> as diff base for review. Please see review.sh for further details. >>>>>>> >>>>>>> And here is the patch! :-) >>>>>> >>>>>> Nice, but I would rather see this in ipatool >>>>>> (). Or is there any obvious benefit >>>>>> in having this in freeipa itself that I'm missing? >>>>> >>>>> For me the obvious benefit is: >>>>> git clone >>>>> git am >>>>> make review >>>>> >>>>> Done. >>>>> >>>>> No need to find & learn other tool, no risk of using wrong version of the tool >>>>> to wrong version of source tree etc. >>>> >>>> AFAIK all IPA developers are supposed to use ipatool, and it already has a >>> >>> Good to know. How does a newcomer learn about it? Honestly I never used >>> ipatool (or not even cloned it). >> >> ipatool targets rather upstream members with write access, so they are hardly >> newcomers. > > I though that we want to make it easy to contribute, so why are you talking > about core developers? I was mostly refering to ipatool's "push" command that I mostly used, but it's true there is also "start-review" or "am" commands that could be useful to others too. > Shouldn't we make it easy to self-review own patches for everyone? Including > random people who want to submit few patches and go away? (Think how we can > apply usability principles to development.) We should, I hope my reply did not suggest otherwise. Martin From ftweedal at redhat.com Wed Dec 16 11:27:16 2015 From: ftweedal at redhat.com (Fraser Tweedale) Date: Wed, 16 Dec 2015 21:27:16 +1000 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <567138DE.3000401@redhat.com> References: <566FC72B.3050406@redhat.com> <56703075.7040108@redhat.com> <20151216004006.GK23644@dhcp-40-8.bne.redhat.com> <56710E17.9060706@redhat.com> <567118B2.9000601@redhat.com> <56711E05.5010102@redhat.com> <567138DE.3000401@redhat.com> Message-ID: <20151216112716.GN23644@dhcp-40-8.bne.redhat.com> On Wed, Dec 16, 2015 at 11:11:42AM +0100, Martin Kosek wrote: > On 12/16/2015 09:17 AM, Jan Cholasta wrote: > > On 16.12.2015 08:54, Martin Kosek wrote: > ... > >>> 7. cert-request fetches the configuration for the specified sub-CA, > >>> or the > >>> default sub-CA if none was specified, from LDAP > >>> > >>> 8. cert-request forwards the request to the certmonger CA helper > >>> specified in > >>> the LDAP configuration over D-Bus (this is the D-Bus method that > >>> currently does > >>> not exist and needs to be implemented) > >>> > >>> 9. certmonger executes the specified CA helper to handle the request > >>> > >>> 10. the CA helper requests the certificate from the CA and returns > >>> either the > >>> certificate, wait delay or error > >>> > >>> 11. certmonger returns the result back to cert-request > >> > >> These steps are subject to Fraser's question (and I am curious too), i.e.: > >> > >> - how is authentication done? certmonger runs with FreeIPA server host > >> principal. > > > > We are on the server, so the RA agent cert is used to authenticate to Dogtag as > > usual, and whatever authentication is configured for other CAs is used for > > other CAs. > > Right, this is how it works now. However, in FreeIPA 4.4 or later, we plan to > switch GSSAPI authentication with Dogtag to get better authorization capabilities: > > https://fedorahosted.org/freeipa/ticket/5011 > > But maybe this could be done via S4U2Proxy as Fraser suggested, although in > this case it would be more complicated as certmonger itself does not have > access to user HTTP/ipa.server ticket, like Apache does, given that Apache > would contact certmonger via DBUS. > If I am not mistaken, Certmonger already uses host credentials, so IPA framework can S4U2Proxy to get user ticket for Certmonger, then Certmonger can S4U2Proxy to get user ticket for Dogtag. Big +1 to the fact that we are pushing away from RA cert to GSS-API for authenticating to Dogtag. > > > >> - how will we handle 3-step certificate request, i.e.: > >> - certificate is requested and in moderation/wait queue > >> - request have to be acked by Dogtag administrator (we do not have > >> API yet) > >> - client should be able to ask for generated certificate > > > > This is not really related to my proposal, since we have to figure this out for > > our Dogtag IPA CA anyway, but the CA helper can return a wait delay in this > > case, so certmonger can poll the request until it is approved. > > Ok. > > >>> 12. cert-request returns the result back to IPA CA helper on the client > >>> > >>> 13. the IPA CA helper on the client returns the result back to > >>> certmonger > >>> > >>> 14. if the result was wait delay, certmonger waits and then retries the > >>> request from step 4, otherwise it stores the certificate or sets error > >>> status > >>> > >> > >> Right, 12-14 is again the standard flow. Good summary of the steps! > > > > > From pviktori at redhat.com Wed Dec 16 12:14:33 2015 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 16 Dec 2015 13:14:33 +0100 Subject: [Freeipa-devel] [PATCH] 0749 Package ipapython, ipalib, ipaplatform, ipatests for Python 3 In-Reply-To: <5665A44E.3060203@redhat.com> References: <563C9810.1050604@redhat.com> <5656E487.8050703@redhat.com> <5658508D.1020105@redhat.com> <565C0FF3.8040406@redhat.com> <565D80B0.9000500@redhat.com> <565DA287.6030403@redhat.com> <565EE63E.6070207@redhat.com> <5665A44E.3060203@redhat.com> Message-ID: <567155A9.7090101@redhat.com> On 12/07/2015 04:22 PM, Jan Cholasta wrote: > On 2.12.2015 13:38, Petr Viktorin wrote: >> On 12/01/2015 02:37 PM, Jan Cholasta wrote: [...] >>>>>> >>>>>> As for new provides, Fedora's Python packaging guidelines say: >>>>>> >>>>>> """ >>>>>> Using a fictional module named "example", the subpackage containing >>>>>> the python2 version must provide python2-example. This is of course >>>>>> always the case if the subpackage is named python2-example [...] >>>>>> If the subpackage has some other name then then Provides: >>>>>> python2-example >>>>>> must be added explicitly (but see the %python_provide macro below). >>>>>> >>>>>> The python3 subpackage must provide python3-example. However, as the >>>>>> naming guidelines mandate that the python3 subpackage be named >>>>>> python3-example, this will happen automatically. >>>>>> """ >>>>>> >>>>>> so I'm now adding Provides for the top-level modules. >>>>> >>>>> The goal of this work is to add support for Python 3, not to comply >>>>> with >>>>> Fedora packaging guidelines. FreeIPA on Fedora uses its own spec file >>>>> anyway. >>>> >>>> The goal of this patch is to add new packages that support Python 3. >>>> Yes, the Fedora spec is different, but it's heavily based on the >>>> upstream one, and this is a good thing. I consider the Fedora >>>> guidelines >>>> the standard in Python RPM packaging. If IPA uses different packaging >>>> guidelines, can you point me to them? >>> >>> FreeIPA never fully complied to Fedora packaging guidelines AFAIK and I >>> don't see any reason to start now, since nobody seemed to care so far. >>> Following them in just py3 sub-packages does not improve the state of >>> FreeIPA as a whole and only brings inconsistency into it, so there's no >>> benefit in doing it at all. >>> >>>>> Again, if you insist on doing this, do it for Python 2 as well. >> >> OK, when your patches are ACKed I'll send patches to both improve py2 >> packaging and add the new packages. Here is the patch rebased onto current master, which includes your packaging improvements. -- Petr Viktorin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0749.3-Package-ipapython-ipalib-ipaplatform-ipatests-for-Py.patch Type: text/x-patch Size: 19034 bytes Desc: not available URL: From ldoudova at redhat.com Wed Dec 16 12:51:28 2015 From: ldoudova at redhat.com (Lenka Doudova) Date: Wed, 16 Dec 2015 13:51:28 +0100 Subject: [Freeipa-devel] [TESTS][PATCH 0008] Fix tests for (stage)user plugin Message-ID: <56715E50.8010806@redhat.com> Hi, this patch fixes few Tracker methods for staged and 'normal' user, which were mistakenly modified by my patch 0006.3. Applies for ipa-4-2 branch only. Lenka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ldoudova-0008-Tests-Fix-tests-for-stage-user-plugin.patch Type: text/x-patch Size: 2096 bytes Desc: not available URL: From pspacek at redhat.com Wed Dec 16 13:23:22 2015 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 16 Dec 2015 14:23:22 +0100 Subject: [Freeipa-devel] [PATCH 0071] dns: Handle SERVFAIL in check if domain already exists Message-ID: <567165CA.4010206@redhat.com> Hello, dns: Handle SERVFAIL in check if domain already exists. In cases where domain is already delegated to IPA prior installation we might get timeout or SERVFAIL. The answer depends on the recursive server we are using for the check. -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0071-dns-Handle-SERVFAIL-in-check-if-domain-already-exist.patch Type: text/x-patch Size: 3718 bytes Desc: not available URL: From dkupka at redhat.com Wed Dec 16 13:31:51 2015 From: dkupka at redhat.com (David Kupka) Date: Wed, 16 Dec 2015 14:31:51 +0100 Subject: [Freeipa-devel] [PATCH 0075-0076] Fix installer regression Message-ID: <567167C7.8060605@redhat.com> https://www.redhat.com/archives/freeipa-users/2015-December/msg00203.html -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0075.0-ipa42+ipa43-installer-Propagate-option-values-from-components-in.patch Type: text/x-patch Size: 2982 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0075.0-master-installer-Propagate-option-values-from-components-in.patch Type: text/x-patch Size: 3156 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0076.0-ipa42+ipa43-installer-Fix-logic-of-reading-option-values-from-ca.patch Type: text/x-patch Size: 1732 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0076.0-master-installer-Fix-logic-of-reading-option-values-from-ca.patch Type: text/x-patch Size: 1728 bytes Desc: not available URL: From ndehadra at redhat.com Wed Dec 16 14:17:10 2015 From: ndehadra at redhat.com (Nikhil Dehadrai) Date: Wed, 16 Dec 2015 19:47:10 +0530 Subject: [Freeipa-devel] Fwd: [PUBLIC] Re: [IPAQE][REVIEW-REQUEST][TEST PLAN] Replica promotion In-Reply-To: <56713F1E.7060303@redhat.com> References: <56558E00.8030006@redhat.com> <566FDF19.4060605@redhat.com> <56713F1E.7060303@redhat.com> Message-ID: <56717266.1080800@redhat.com> Hi There, Based on the URL for REPLICA PROMOTION Test plan (http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan), I have following observations / queries. Observations: ------------------- 1. For "ipa-kra-install" and "ipa-ca-install" , expected result for step 1 should be FAILED, But it is marked as SUCCEED. (What is the correct behavior here). Questions: -------------- 1. As per current design (IPA 4.2) we can prepare replica server file on an existing replica server using "ipa-replica-prepare" and use this file to setup new replica host, how can we achieve this behavior under this new feature when MASTER domain level is set to 1.? 2. How will the "ipa topologysegment-add" work for a 3 repilca scenario? Since, under domain level 1 it uses "rightnode" and "leftnode" to form a new segment. Can I add 3rd replica to this already created "Newsegment" or we need to have it created separately? How will it work? We can add New Test Scenarios as well: ----------------------------------------------------- 1. Add scenario for domain level details at UI. Since it talks of reflecting the domain level changes at UI. 2. Add scenario to validate once domain level of master is raised to 1, after that it cannot be lowered. 3. Add scenario for "ipa-restore" using master and replica topology , so as to identify the effect of restoration on replica by using "ipa topologysegment-find realm" command.( What will be the expected behavior in this case) Let me know. Thanks and Best Regards, Nikhil Dehadrai From dkupka at redhat.com Wed Dec 16 14:18:46 2015 From: dkupka at redhat.com (David Kupka) Date: Wed, 16 Dec 2015 15:18:46 +0100 Subject: [Freeipa-devel] [PATCH 0071] dns: Handle SERVFAIL in check if domain already exists In-Reply-To: <567165CA.4010206@redhat.com> References: <567165CA.4010206@redhat.com> Message-ID: <567172C6.1060002@redhat.com> On 16/12/15 14:23, Petr Spacek wrote: > Hello, > > dns: Handle SERVFAIL in check if domain already exists. > > In cases where domain is already delegated to IPA prior installation > we might get timeout or SERVFAIL. The answer depends on the recursive > server we are using for the check. > > > Works for me, ACK. -- David Kupka From pvoborni at redhat.com Wed Dec 16 14:26:23 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 16 Dec 2015 15:26:23 +0100 Subject: [Freeipa-devel] [PATCH 0071] dns: Handle SERVFAIL in check if domain already exists In-Reply-To: <567172C6.1060002@redhat.com> References: <567165CA.4010206@redhat.com> <567172C6.1060002@redhat.com> Message-ID: <5671748F.7000508@redhat.com> On 12/16/2015 03:18 PM, David Kupka wrote: > On 16/12/15 14:23, Petr Spacek wrote: >> Hello, >> >> dns: Handle SERVFAIL in check if domain already exists. >> >> In cases where domain is already delegated to IPA prior installation >> we might get timeout or SERVFAIL. The answer depends on the recursive >> server we are using for the check. >> >> >> > Works for me, ACK. > Pushed to: master: 58331208a5ded367e521d42d99de2835f329fab7 ipa-4-3: d35067515e20de764e58d3ad8e343b3e2282a780 -- Petr Vobornik From ofayans at redhat.com Wed Dec 16 14:28:06 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Wed, 16 Dec 2015 15:28:06 +0100 Subject: [Freeipa-devel] [PATCH 535] ipautil: remove unused import causing cyclic import in tests In-Reply-To: <5670262A.9060304@redhat.com> References: <5670262A.9060304@redhat.com> Message-ID: <567174F6.8020509@redhat.com> ACK, the issue is gone On 12/15/2015 03:39 PM, Jan Cholasta wrote: > Hi, > > the attached patch fixes . > > Pushed to under the one-liner rule: > master: c265e8736e51d5b4fede94a414d83b3e0ada2853 > ipa-4-3: 2b28704f926d24e1562b481880b111d12d2020f2 > > Honza > > > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From ssorce at redhat.com Wed Dec 16 14:29:45 2015 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 16 Dec 2015 09:29:45 -0500 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <567101F2.30508@redhat.com> References: <566FC72B.3050406@redhat.com> <56703075.7040108@redhat.com> <1450200176.17418.173.camel@redhat.com> <567101F2.30508@redhat.com> Message-ID: <1450276185.17418.178.camel@redhat.com> On Wed, 2015-12-16 at 07:17 +0100, Jan Cholasta wrote: > On 15.12.2015 18:22, Simo Sorce wrote: > > On Tue, 2015-12-15 at 16:23 +0100, Martin Kosek wrote: > >> On 12/15/2015 08:54 AM, Jan Cholasta wrote: > >>> Hi, > >>> > >>> recently I and David discussed the direction of installers with regard to > >>> requesting certificates. Currently there are four (!) different ways of > >>> requesting certificates in the installer [1][2][3][4]. We would like to reduce > >>> it to one. > >>> > >>> Since all the certificates are tracked by certmonger and certmonger already > >>> knows how to request certificates from Dogtag (and other CAs), we believe that > >>> all certificates should be requested using certmonger. > >>> > >>> Taking our meditation further, we thought "Why not use certmonger for the > >>> cert-request command as well?" What is the benefit, do you ask? > >>> > >>> a) single code path for requesting certificates (seriously, the current state > >>> is ridiculous) > >>> > >>> b) use any CA supported by certmonger as the IPA CA (i.e. Let's Encrypt [5], > >>> once certmonger gains support for it) > >>> > >>> c) automate external CA install, using any CA supported by certmonger [6] > >>> > >>> d) support multiple different CAs at once (generalization of the Sub-CA feature) > >>> > >>> e) uniform configuration on clients (configure once, use forever, even for > >>> CA-less) > >>> > >>> The idea is to store configuration for the different CAs in LDAP and have > >>> cert-request redirect requests to a proper CA helper according to that > >>> configuration. This would require a new certmonger D-Bus method to call a CA > >>> helper without associated certificate storage, but that should be rather easy > >>> to add. In return, it would be possible to do all of the above. > >>> > >>> Note that this should not conflict with tighter integration with Dogtag > >>> (profiles, ACLs, etc.). > >>> > >>> Comments are welcome. > >>> > >>> Honza > >>> > >>> [1] > >>> > >>> [2] > >>> > >>> > >>> [3] > >>> > >>> > >>> [4] > >>> > >>> > >>> [5] > >>> [6] > >> > >> Interesting idea! I would be definitely interested to hear what Fraser, Rob or > >> Simo thinks here. > > > > Sounds great to me in principle. > > > > How do you handle CAs that do not have automatic workflows for csr > > handling ? > > > > That's the reason we did the 2 step process (in reference to [6]) > > This could be handled by a dummy "manual" CA helper in certmonger, which > would dump the CSR into the filesystem and wait for the user to provide > the signed certificate in the filesystem and resume the certmonger request. > > As you pointed out, we currently do the same, although manually, in > external CA install. It is also done when renewing the externally signed > CA certificate - this time it is done using certmonger, but it is > handled by the dogtag-ipa-ca-renewal-agent helper rather than a separate > helper. The reason why we did the 2 step process is that it can take days in some cases to get back a cert given a csr. I do not think it is safe to wait for days mid-install. At the very least we'll need to be able to stop the install and resume it when the certs are available. Simo. From dkupka at redhat.com Wed Dec 16 14:35:14 2015 From: dkupka at redhat.com (David Kupka) Date: Wed, 16 Dec 2015 15:35:14 +0100 Subject: [Freeipa-devel] [PATCH] ca-less tests updated - POC In-Reply-To: <563CA575.1030808@redhat.com> References: <56337745.109@redhat.com> <563B091B.3010501@redhat.com> <563B6A96.8090606@redhat.com> <563BABEC.8080304@redhat.com> <563C5B1A.1090803@redhat.com> <563C5E6D.4060307@redhat.com> <563CA575.1030808@redhat.com> Message-ID: <567176A2.1000901@redhat.com> On 06/11/15 14:04, Oleg Fayans wrote: > Hi Jan, > > On 11/06/2015 09:01 AM, Jan Cholasta wrote: >> Actually it might be better to keep them, but fix them to expect >> ipa-server-certinstall to success. > > Done. Updated patch attached. > Also in the patch 0013 I removed a trailing whitespace which caused lint > to complain > > Now with domain level 0 the test output looks like this: > > [11:40:51]ofayans at vm-076:~]$ ipa-run-tests test_integration/test_caless.py > ==================================================================================== > test session starts > ===================================================================================== > > platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.6.4 > plugins: multihost, sourceorder > collected 88 items > > test_integration/test_caless.py > ......xx......ss...............xx........ss....................xx......ss............... > > > ===================================================================== 76 > passed, 6 skipped, 6 xfailed in 7871.10 seconds > ===================================================================== > > >> >> On 6.11.2015 08:47, Jan Cholasta wrote: >>> Hi Oleg, >>> >>> I think you can just remove >>> TestCertinstall.test_{http,ds}_intermediate_ca, the certificates are >>> imported correctly in this case and I didn't see anything break. >>> >>> Honza >>> >>> On 5.11.2015 20:20, Oleg Fayans wrote: >>>> Patch 0014 updated and passes lint >>>> >>>> On 11/05/2015 03:41 PM, Oleg Fayans wrote: >>>>> Wait a bit, the patch has problems with pylint: it does not build :) >>>>> The updated version (without the setupmaster nonsense) is being tested >>>>> now. >>>>> >>>>> On 11/05/2015 08:45 AM, Oleg Fayans wrote: >>>>>> Hi Jan, >>>>>> >>>>>> Could you take a look at these, whenever you are free? >>>>>> >>>>>> On 10/30/2015 02:57 PM, Oleg Fayans wrote: >>>>>>> Hi, >>>>>>> >>>>>>> The following patches contain updates to ca-less integration tests. >>>>>>> It's still a proof of concept: 2 tests still fail seemingly due to >>>>>>> the >>>>>>> change in target system logic (marked as xfail with "ask jcholast >>>>>>> comment") >>>>>>> >>>>>>> The test output looks like this: >>>>>>> >>>>>>> $ ipa-run-tests test_integration/test_caless.py --pdb >>>>>>> ==================================================================================== >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> test session starts >>>>>>> ===================================================================================== >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.6.4 >>>>>>> plugins: multihost, sourceorder >>>>>>> collected 88 items >>>>>>> >>>>>>> test_integration/test_caless.py >>>>>>> ......xx......ss............sssssssssssssssssss.ssssss.........xx......ssxx............. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> ==================================================================== >>>>>>> 53 >>>>>>> >>>>>>> passed, 29 skipped, 6 xfailed in 5620.17 seconds >>>>>>> ===================================================================== >>>>>>> >>>>>>> >>>>>>> Numerous skips correspond to the tests related to >>>>>>> ipa-replica-prepare >>>>>>> (unsupported under domain level 1) >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>>> >>> >>> >> >> > > > > This body part will be downloaded on demand. > Hello, thanks for updated patches. I'm really sorry it took so long before I got to them. There was change in ipapython.ipautil.run that happened after you sent the patches. Feel free to squash attached patch that fixes it. Unfortunately I see a lot of test failing with domain-level 0: http://fpaste.org/301657/50275682/ domain-level 1 (domain-level 1: http://fpaste.org/301658/02757191/) seems better. There are 2 failing test that you're probably mentioning in commit message plus one that I think is bug in code rather than bug in tests. Do you have any proposal for fixing the two failing tests? One nitpick: Please use mail for notes like "need further consulting ..." rather that commit message. When the patch gets accepted it will still need modification before push just because inappropriate commit message. Thank you! -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Addapt-CA-less-test-to-new-ipapython.ipautil.run.patch Type: text/x-patch Size: 1131 bytes Desc: not available URL: From mbasti at redhat.com Wed Dec 16 14:39:01 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 16 Dec 2015 15:39:01 +0100 Subject: [Freeipa-devel] [PATCH 016 - 017] First part of the replica promotion tests + testplan In-Reply-To: <566FDD80.7030805@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> <565DB7D3.5040500@redhat.com> <565F0BDD.8080206@redhat.com> <565F17EE.3020701@redhat.com> <5666DC18.2040906@redhat.com> <56675E31.3050203@redhat.com> <5667F49B.5020709@redhat.com> <5667FEFB.6090904@redhat.com> <56680C2E.7060805@redhat.com> <56680DDD.2000106@redhat.com> <566FDD80.7030805@redhat.com> Message-ID: <56717785.4020304@redhat.com> On 15.12.2015 10:29, Oleg Fayans wrote: > Hi Martin, > > The updated patches are attached. Patch 0017 includes all changes from > patch 0018, so, if you approve this one, there would be no need to > continue with the review of 0018. This one contains all changes related > to you remarks from 0018 review. Please see my explanation on the > stdout+stderr part in the thread from patch 0018. > With these two patches applied one of the tests fails due this bug: > https://fedorahosted.org/freeipa/ticket/5550 > > On 12/09/2015 12:17 PM, Martin Basti wrote: >> >> On 09.12.2015 12:10, Martin Basti wrote: >>> >>> On 09.12.2015 11:14, Oleg Fayans wrote: >>>> Hi Martin >>>> >>>> On 12/09/2015 10:30 AM, Martin Basti wrote: >>>>> On 08.12.2015 23:48, Oleg Fayans wrote: >>>>>> Substituted a hardcoded suffix name with a constant DOMAIN_SUFFIX_NAME >>>>>> >>>>>> On 12/08/2015 02:33 PM, Oleg Fayans wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> >>>>>>> The patches are rebased against the current master. >>>>>>> >>>>>>> On 12/02/2015 05:10 PM, Martin Basti wrote: >>>>>>>> On 02.12.2015 16:18, Oleg Fayans wrote: >>>>>>>>> Hi Martin, >>>>>>>>> >>>>>>>>> On 12/01/2015 04:08 PM, Martin Basti wrote: >>>>>>>>>> On 27.11.2015 16:26, Oleg Fayans wrote: >>>>>>>>>>> And patch N 16 passes lint too: >>>>>>>>>>> >>>>>>>>>>> On 11/27/2015 04:03 PM, Oleg Fayans wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> On 11/27/2015 03:26 PM, Martin Basti wrote: >>>>>>>>>>>>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>>> >>>>>>>>>>>>>> All your suggestions were taken into account. Both patches are >>>>>>>>>>>>>> updated. Thank you for your help! >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>>>>>>>>>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I agree to all your points but one. please, see my comment >>>>>>>>>>>>>>>> below >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 0) Note >>>>>>>>>>>>>>>>> Please be aware of >>>>>>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5469 >>>>>>>>>>>>>>>>> during >>>>>>>>>>>>>>>>> KRA testing >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, >>>>>>>>>>>>>>>>> this may >>>>>>>>>>>>>>>>> change >>>>>>>>>>>>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for domain >>>>>>>>>>>>>>>>> level 0 >>>>>>>>>>>>>>>>> and 1 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>>>> Why uninstall KRA then server, is not enough just uninstall >>>>>>>>>>>>>>>>> server >>>>>>>>>>>>>>>>> which >>>>>>>>>>>>>>>>> covers KRA uninstall? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> + def teardown_method(self, method): >>>>>>>>>>>>>>>>> + for host in self.replicas: >>>>>>>>>>>>>>>>> + host.run_command(self.kra_uninstall, >>>>>>>>>>>>>>>>> raiseonerr=False) >>>>>>>>>>>>>>>>> + tasks.uninstall_master(host) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>>>> Can be this function more generic? It should allow specify >>>>>>>>>>>>>>>>> host >>>>>>>>>>>>>>>>> where >>>>>>>>>>>>>>>>> KRA should be installed not just master >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> + def test_kra_install_master(self): >>>>>>>>>>>>>>>>> + self.master.run_command(self.kra_install) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> TestLevel0(Dummy): >>>>>>>>>>>>>>>>> Can be the test name more specific, something like >>>>>>>>>>>>>>>>> TestReplicaPromotionLevel0 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 5) >>>>>>>>>>>>>>>>> please remove this, the patch is on review and it will be >>>>>>>>>>>>>>>>> pushed >>>>>>>>>>>>>>>>> sooner >>>>>>>>>>>>>>>>> than tests >>>>>>>>>>>>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> and as I mentioned in ticket #5455, I cannot reproduce >>>>>>>>>>>>>>>>> it with >>>>>>>>>>>>>>>>> ipa-kra-install, so please provide steps to reproduce if >>>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>> insist >>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>> this still does not work as expected with KRA. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 6) This is completely wrong, it removes everything that we >>>>>>>>>>>>>>>>> tried to >>>>>>>>>>>>>>>>> achieve with previous patches with domain level in CI >>>>>>>>>>>>>>>> Actually, being able to configure domain level per class >>>>>>>>>>>>>>>> is WAY >>>>>>>>>>>>>>>> more >>>>>>>>>>>>>>>> convenient, than to always have to think which domain >>>>>>>>>>>>>>>> level is >>>>>>>>>>>>>>>> appropriate for which particular test during jenkins job >>>>>>>>>>>>>>>> configuration. In fact, I should have thought about it >>>>>>>>>>>>>>>> from the >>>>>>>>>>>>>>>> very >>>>>>>>>>>>>>>> beginning. For example, in test_replica_promotion.py we >>>>>>>>>>>>>>>> have on >>>>>>>>>>>>>>>> class, >>>>>>>>>>>>>>>> which intiates with domain level = 1, while others - with >>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>> 0. With config-based approach, we would have to implement a >>>>>>>>>>>>>>>> separate >>>>>>>>>>>>>>>> step that raises domain level. Overall, I am against the >>>>>>>>>>>>>>>> approach, >>>>>>>>>>>>>>>> when you have to remember to set certain domain level in >>>>>>>>>>>>>>>> config >>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>> any particular test. The tests themselves should be aware of >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>> domain level they need. >>>>>>>>>>>>>>> I do not say that we should not have something that overrides >>>>>>>>>>>>>>> settings >>>>>>>>>>>>>>> in from config in a particular test case, I say your patch is >>>>>>>>>>>>>>> doing it >>>>>>>>>>>>>>> wrong. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I agree it is useful to have param domain_level in >>>>>>>>>>>>>>> install_master, >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by >>>>>>>>>>>>>>> default, >>>>>>>>>>>>>>> because with your current patch the domain_level in config is >>>>>>>>>>>>>>> not >>>>>>>>>>>>>>> used >>>>>>>>>>>>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> For example I want to achieve this goal: >>>>>>>>>>>>>>> test_vault.py, this test suite can run on domain level1 >>>>>>>>>>>>>>> and on >>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>> level0, so with one test we can test 2 domain levels just >>>>>>>>>>>>>>> with >>>>>>>>>>>>>>> putting >>>>>>>>>>>>>>> domain level into config file. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I agree that with extraordinary test like replica >>>>>>>>>>>>>>> promotion test >>>>>>>>>>>>>>> is, we >>>>>>>>>>>>>>> need something that allows override the config file. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> As I said bellow, domain_level default value should be >>>>>>>>>>>>>>> None in >>>>>>>>>>>>>>> install_master and install_topo plugin. If domain level was >>>>>>>>>>>>>>> specified >>>>>>>>>>>>>>> use the specified one, if not (value is None) use the domain >>>>>>>>>>>>>>> level >>>>>>>>>>>>>>> from >>>>>>>>>>>>>>> config file. >>>>>>>>>>>>>> Agreed :) >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Martin >>>>>>>>>>>>>>>>> [PATCH] Enabled setting domain_level per class derived from >>>>>>>>>>>>>>>>> TestIntegration >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> When I configure domain level 0 in yaml config, how is this >>>>>>>>>>>>>>>>> supposed to >>>>>>>>>>>>>>>>> get into install methods when you removed that code? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>>>>>>>>>>>> + "--domain-level=%i" % domain_level >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> You always use MAX_DOMAIN_LEVEL in this case or whatever is >>>>>>>>>>>>>>>>> specified in >>>>>>>>>>>>>>>>> domain_level option. >>>>>>>>>>>>>>>>> I suggest to use domain_level=None, and when it is None use >>>>>>>>>>>>>>>>> 'host.config.domain_level', if it is not None, use >>>>>>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> With this we can specify domain level in config file for >>>>>>>>>>>>>>>>> test >>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>> be used for both domain levels and you can manually specify >>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>>> for test that requires specific domain level. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Also this should go away >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> @classmethod >>>>>>>>>>>>>>>>> def install(cls, mh): >>>>>>>>>>>>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>>>>>>>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>>>>>>>>>>>> if cls.topology is None: >>>>>>>>>>>>>>>>> return >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I do not see reason why test should override >>>>>>>>>>>>>>>>> configuration in >>>>>>>>>>>>>>>>> config in >>>>>>>>>>>>>>>>> this case. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Martin >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Here is the updated version of the patch (more tests + >>>>>>>>>>>>>>>>>> fixed the >>>>>>>>>>>>>>>>>> issues of the first one) + patch 0017, that implements the >>>>>>>>>>>>>>>>>> necessary >>>>>>>>>>>>>>>>>> changes in the background code, i. e. patch 16 does not >>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>> without >>>>>>>>>>>>>>>>>> patch 17 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>>>>>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>>>>>>>>>>>> Hi guys, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Here are first two automated testcases from this (so far >>>>>>>>>>>>>>>>>>>> incomplete) >>>>>>>>>>>>>>>>>>>> testplan: >>>>>>>>>>>>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> PATCH 16: NACK >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>>>>>>>>>>>> 'domain_level' to >>>>>>>>>>>>>>>>>>> install_topo()? >>>>>>>>>>>>>>>>>>> Also it is good practise to add new option as the last >>>>>>>>>>>>>>>>>>> parameter. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>>>>>> cab you in both tests specify a domain level with >>>>>>>>>>>>>>>>>>> constant >>>>>>>>>>>>>>>>>>> instead of >>>>>>>>>>>>>>>>>>> number literal? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>>>>>> both test call install_topo with custom domain level, >>>>>>>>>>>>>>>>>>> but it >>>>>>>>>>>>>>>>>>> cannot >>>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>>> because 1) (did you run the test?) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>>>>>>>>>>>> Respectively why there is call of install_topo() that >>>>>>>>>>>>>>>>>>> installs >>>>>>>>>>>>>>>>>>> replica. >>>>>>>>>>>>>>>>>>> As this test just tests that ipa-replica-prepare is not >>>>>>>>>>>>>>>>>>> working >>>>>>>>>>>>>>>>>>> anymore, >>>>>>>>>>>>>>>>>>> is it worth to spend 20 minutes with installing >>>>>>>>>>>>>>>>>>> replica and >>>>>>>>>>>>>>>>>>> then >>>>>>>>>>>>>>>>>>> just no >>>>>>>>>>>>>>>>>>> tot use it? IMO to install master in install step is >>>>>>>>>>>>>>>>>>> enough. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Martin^2 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>> ./make-lint >>>>>>>>>>>>> ************* Module ipatests.test_integration.base >>>>>>>>>>>>> ipatests/test_integration/base.py:66: [E1101(no-member), >>>>>>>>>>>>> IntegrationTest.install] Class 'IntegrationTest' has no >>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>> member) >>>>>>>>>>>>> ************* Module >>>>>>>>>>>>> ipatests.test_integration.test_replica_promotion >>>>>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:16: >>>>>>>>>>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no >>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>> member) >>>>>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:117: >>>>>>>>>>>>> [E1101(no-member), >>>>>>>>>>>>> TestCAInstall.test_ca_install_without_replica_file] >>>>>>>>>>>>> Module 'ipatests.test_integration.tasks' has no 'setup_replica' >>>>>>>>>>>>> member) >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Is it so hard to run pylint before patch is posted for review? >>>>>>>>>>>> Sorry, my bad. Fixed. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> 1) >>>>>>>>>> Why is this change in the patch? >>>>>>>>>> - # Clean up the test directory >>>>>>>>>> - host.run_command(['rm', '-rvf', host.config.test_dir]) >>>>>>>>> Otherwise 2 out of 8 tests fail with IOError at line 78 of >>>>>>>>> ipatests/test_integration/tasks.py >>>>>>>>> >>>>>>>>> I do not understand yet how does this happen, but if you remove >>>>>>>>> ipatests folder once, it then fails to be created again. >>>>>>>>> >>>>>>>> So this should be in separated patch and investigated properly. >>>>>>> Agree. Removed >>>>>>>>>> - >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2) >>>>>>>>>> is enough to have this check only in install_master, >>>>>>>>>> install_topo can >>>>>>>>>> pass None to install_master >>>>>>>>>> + if domain_level is None: >>>>>>>>>> + domain_level = master.config.domain_level >>>>>>>>> Done >>>>>>>>> >>>>>>>>>> 3) >>>>>>>>>> IMO replica-manage del should cleanup hosts entry, so following >>>>>>>>>> code >>>>>>>>>> should not be needed. >>>>>>>>>> + if cleanhost: >>>>>>>>>> + kinit_admin(master) >>>>>>>>>> + master.run_command(["ipa", "host-del", "--updatedns", >>>>>>>>>> replica.hostname], >>>>>>>>>> + raiseonerr=False) >>>>>>>>> Well, in fact it does not. At least the corresponding dns record >>>>>>>>> stays >>>>>>>>> and causes the subsequent ipa-client-install to fail. Probably >>>>>>>>> it's a >>>>>>>>> bug. On the other hand, if I promote an existing client to >>>>>>>>> replica and >>>>>>>>> then delete this replica, then, I probably want the host record >>>>>>>>> (that >>>>>>>>> was created during client-install) to stay in the system. So, >>>>>>>>> does not >>>>>>>>> look like a bug to me. >>>>>>>> No you don't, because replica uninstallation also removes the >>>>>>>> client. >>>>>>>> >>>>>>>> I tried it with ipa42, ipa-replica-manage del removes host entry, >>>>>>>> and >>>>>>>> DNS A records, only SSHFP records stay there (I'm not sure if it >>>>>>>> is bug >>>>>>>> or feature) >>>>>>>> >>>>>>>> Also I received this message >>>>>>>> """ >>>>>>>> Failed to cleanup replica1.ipa.test DNS entries: no matching entry >>>>>>>> found >>>>>>>> You may need to manually remove them from the tree >>>>>>>> """ >>>>>>>> But, A record has been removed, so this is probably false >>>>>>>> positive and >>>>>>>> it needs to have a ticket. >>>>>>> Agree, that was an issue with my setup. >>>>>>> Removed >>>>>>> >>>>>>>>>> 4) >>>>>>>>>> This variable is not used >>>>>>>>>> + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] >>>>>>>>> Removed >>>>>>>>> >>>>>>>>>> 5) >>>>>>>>>> Why do you need this >>>>>>>>>> + kra_install = ["ipa-kra-install", "-U", "-p", >>>>>>>>>> config.dirman_password] >>>>>>>>>> when you implemented tasks.install_kra that returns the exactly >>>>>>>>>> the same >>>>>>>>>> result? >>>>>>>>> Right. Removed >>>>>>>>> >>>>>>>>>> 6) >>>>>>>>>> PEP8 >>>>>>>>>> ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 blank >>>>>>>>>> lines, >>>>>>>>>> found 1 >>>>>>>>>> ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 blank >>>>>>>>>> lines, >>>>>>>>>> found 1 >>>>>>>>>> ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 blank >>>>>>>>>> lines, >>>>>>>>>> found 1 >>>>>>>>>> ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 blank >>>>>>>>>> lines, >>>>>>>>>> found 1 >>>>>>>>>> ./ipatests/test_integration/tasks.py:950:80: E501 line too long >>>>>>>>>> (80 > 79 >>>>>>>>>> characters) >>>>>>>>>> >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:29:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (85 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:64:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (85 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:67:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (88 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:93:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (80 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:94:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (83 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (81 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (80 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (82 > 79 characters) >>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 >>>>>>>>>> line >>>>>>>>>> too long (80 > 79 characters) >>>>>>>>> Most of these complaints are unrelated to the current patches. >>>>>>>>> It's better to create a separate patch addressing PEP8 errors. >>>>>>>>> >>>>>>>> I beg for your pardon, those PEP8 errors have been introduced by >>>>>>>> your >>>>>>>> patches. >>>>>>> Fixed >>>>>>> >>>>>>>>>> 7) >>>>>>>>>> Why this must be stored in instance? IMO to have it stored as >>>>>>>>>> local >>>>>>>>>> variable is perfect >>>>>>>>>> TestKRAInstall, TestCAInstall >>>>>>>>>> self.replica1_filename = >>>>>>>>>> tasks.get_replica_filename(replica1) >>>>>>>>>> self.replica2_filename = >>>>>>>>>> tasks.get_replica_filename(replica2) >>>>>>>>> Agree. Fixed >>>>>>>>> >>>>> This patch is missing something. >>>> I am sorry, I forgot to revert my previous change. The correct patch is >>>> attached >>>> >>> ************* Module ipatests.test_integration.test_replica_promotion >>> ipatests/test_integration/test_replica_promotion.py:15: >>> [E1123(unexpected-keyword-arg), Dummy.install] Unexpected keyword >>> argument 'domain_level' in function call) >>> ipatests/test_integration/test_replica_promotion.py:15: >>> [E1101(no-member), Dummy.install] Class 'Dummy' has no 'domain_level' >>> member) >>> ipatests/test_integration/test_replica_promotion.py:19: >>> [E1101(no-member), Dummy.teardown_method] Module >>> 'ipatests.test_integration.tasks' has no 'uninstall_replica' member) >>> ipatests/test_integration/test_replica_promotion.py:67: >>> [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] >>> Module 'ipatests.test_integration.tasks' has no 'ipa_backup' member) >>> ipatests/test_integration/test_replica_promotion.py:72: >>> [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] >>> Module 'ipatests.test_integration.tasks' has no 'ipa_restore' member) >>> ipatests/test_integration/test_replica_promotion.py:120: >>> [E1123(unexpected-keyword-arg), TestCAInstall.install] Unexpected >>> keyword argument 'domain_level' in function call) >>> >> Sorry I forgot to apply patch 17, my bad, I'm continuing with review LGTM, I haven't had time to test it, but if you are sure that test is working, we may push this. From jcholast at redhat.com Wed Dec 16 14:41:22 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 16 Dec 2015 15:41:22 +0100 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <1450276185.17418.178.camel@redhat.com> References: <566FC72B.3050406@redhat.com> <56703075.7040108@redhat.com> <1450200176.17418.173.camel@redhat.com> <567101F2.30508@redhat.com> <1450276185.17418.178.camel@redhat.com> Message-ID: <56717812.40304@redhat.com> On 16.12.2015 15:29, Simo Sorce wrote: > On Wed, 2015-12-16 at 07:17 +0100, Jan Cholasta wrote: >> On 15.12.2015 18:22, Simo Sorce wrote: >>> On Tue, 2015-12-15 at 16:23 +0100, Martin Kosek wrote: >>>> On 12/15/2015 08:54 AM, Jan Cholasta wrote: >>>>> Hi, >>>>> >>>>> recently I and David discussed the direction of installers with regard to >>>>> requesting certificates. Currently there are four (!) different ways of >>>>> requesting certificates in the installer [1][2][3][4]. We would like to reduce >>>>> it to one. >>>>> >>>>> Since all the certificates are tracked by certmonger and certmonger already >>>>> knows how to request certificates from Dogtag (and other CAs), we believe that >>>>> all certificates should be requested using certmonger. >>>>> >>>>> Taking our meditation further, we thought "Why not use certmonger for the >>>>> cert-request command as well?" What is the benefit, do you ask? >>>>> >>>>> a) single code path for requesting certificates (seriously, the current state >>>>> is ridiculous) >>>>> >>>>> b) use any CA supported by certmonger as the IPA CA (i.e. Let's Encrypt [5], >>>>> once certmonger gains support for it) >>>>> >>>>> c) automate external CA install, using any CA supported by certmonger [6] >>>>> >>>>> d) support multiple different CAs at once (generalization of the Sub-CA feature) >>>>> >>>>> e) uniform configuration on clients (configure once, use forever, even for >>>>> CA-less) >>>>> >>>>> The idea is to store configuration for the different CAs in LDAP and have >>>>> cert-request redirect requests to a proper CA helper according to that >>>>> configuration. This would require a new certmonger D-Bus method to call a CA >>>>> helper without associated certificate storage, but that should be rather easy >>>>> to add. In return, it would be possible to do all of the above. >>>>> >>>>> Note that this should not conflict with tighter integration with Dogtag >>>>> (profiles, ACLs, etc.). >>>>> >>>>> Comments are welcome. >>>>> >>>>> Honza >>>>> >>>>> [1] >>>>> >>>>> [2] >>>>> >>>>> >>>>> [3] >>>>> >>>>> >>>>> [4] >>>>> >>>>> >>>>> [5] >>>>> [6] >>>> >>>> Interesting idea! I would be definitely interested to hear what Fraser, Rob or >>>> Simo thinks here. >>> >>> Sounds great to me in principle. >>> >>> How do you handle CAs that do not have automatic workflows for csr >>> handling ? >>> >>> That's the reason we did the 2 step process (in reference to [6]) >> >> This could be handled by a dummy "manual" CA helper in certmonger, which >> would dump the CSR into the filesystem and wait for the user to provide >> the signed certificate in the filesystem and resume the certmonger request. >> >> As you pointed out, we currently do the same, although manually, in >> external CA install. It is also done when renewing the externally signed >> CA certificate - this time it is done using certmonger, but it is >> handled by the dogtag-ipa-ca-renewal-agent helper rather than a separate >> helper. > > The reason why we did the 2 step process is that it can take days in > some cases to get back a cert given a csr. > > I do not think it is safe to wait for days mid-install. > At the very least we'll need to be able to stop the install and resume > it when the certs are available. I'm not suggesting to wait, we don't wait in ipa-cacert-manage when renewing the CA certificate either. We can monitor the status of the certmonger request and interrupt the install when it reaches NEED_GUIDANCE (which means user intervention is needed). -- Jan Cholasta From mbabinsk at redhat.com Wed Dec 16 14:41:24 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Wed, 16 Dec 2015 15:41:24 +0100 Subject: [Freeipa-devel] Fwd: [PUBLIC] Re: [IPAQE][REVIEW-REQUEST][TEST PLAN] Replica promotion In-Reply-To: <56717266.1080800@redhat.com> References: <56558E00.8030006@redhat.com> <566FDF19.4060605@redhat.com> <56713F1E.7060303@redhat.com> <56717266.1080800@redhat.com> Message-ID: <56717814.1050604@redhat.com> On 12/16/2015 03:17 PM, Nikhil Dehadrai wrote: > Hi There, > > Based on the URL for REPLICA PROMOTION Test plan > (http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan), I have > following observations / queries. > > Observations: > ------------------- > 1. For "ipa-kra-install" and "ipa-ca-install" , expected result for step > 1 should be FAILED, But it is marked as SUCCEED. (What is the correct > behavior here). > > Questions: > -------------- > 1. As per current design (IPA 4.2) we can prepare replica server file > on an existing replica server using "ipa-replica-prepare" and use this > file to setup new replica host, how can we achieve this behavior under > this new feature when MASTER domain level is set to 1.? You can't. The whole point of replica installation by promotion is to eliminate the need to lug around files to set up new replicas. In domain level 1 the ipa-replica-prepare command is disabled and running it raises an error (See test case 2 in http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan). Also there is no MASTER domain level, the domain level is uniform across the whole topology > 2. How will the "ipa topologysegment-add" work for a 3 repilca scenario? > Since, under domain level 1 it uses "rightnode" and "leftnode" to form a > new segment. Can I add 3rd replica to this already created "Newsegment" > or we need to have it created separately? How will it work? > Topology segments describe replication agreements between *two* endpoints (think about them as graph edges connecting two nodes). So you set up a segment between replica1 and replica2. To connect third replica to the topology, you have to create new segment connecting either replica1 and replica3, or replica2 and replica3. > We can add New Test Scenarios as well: > ----------------------------------------------------- > 1. Add scenario for domain level details at UI. Since it talks of > reflecting the domain level changes at UI. > 2. Add scenario to validate once domain level of master is raised to 1, > after that it cannot be lowered. > 3. Add scenario for "ipa-restore" using master and replica topology , so > as to identify the effect of restoration on replica by using "ipa > topologysegment-find realm" command.( What will be the expected behavior > in this case) > I'm not quite sure about point 3 either. I guess that you have to reenroll the replica after restore, but maybe someone more knowledgeable about the feature will chime in. > > Let me know. > > Thanks and Best Regards, > Nikhil Dehadrai > > -- Martin^3 Babinsky From ssorce at redhat.com Wed Dec 16 14:44:44 2015 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 16 Dec 2015 09:44:44 -0500 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <56717812.40304@redhat.com> References: <566FC72B.3050406@redhat.com> <56703075.7040108@redhat.com> <1450200176.17418.173.camel@redhat.com> <567101F2.30508@redhat.com> <1450276185.17418.178.camel@redhat.com> <56717812.40304@redhat.com> Message-ID: <1450277084.17418.183.camel@redhat.com> On Wed, 2015-12-16 at 15:41 +0100, Jan Cholasta wrote: > On 16.12.2015 15:29, Simo Sorce wrote: > > On Wed, 2015-12-16 at 07:17 +0100, Jan Cholasta wrote: > >> On 15.12.2015 18:22, Simo Sorce wrote: > >>> On Tue, 2015-12-15 at 16:23 +0100, Martin Kosek wrote: > >>>> On 12/15/2015 08:54 AM, Jan Cholasta wrote: > >>>>> Hi, > >>>>> > >>>>> recently I and David discussed the direction of installers with regard to > >>>>> requesting certificates. Currently there are four (!) different ways of > >>>>> requesting certificates in the installer [1][2][3][4]. We would like to reduce > >>>>> it to one. > >>>>> > >>>>> Since all the certificates are tracked by certmonger and certmonger already > >>>>> knows how to request certificates from Dogtag (and other CAs), we believe that > >>>>> all certificates should be requested using certmonger. > >>>>> > >>>>> Taking our meditation further, we thought "Why not use certmonger for the > >>>>> cert-request command as well?" What is the benefit, do you ask? > >>>>> > >>>>> a) single code path for requesting certificates (seriously, the current state > >>>>> is ridiculous) > >>>>> > >>>>> b) use any CA supported by certmonger as the IPA CA (i.e. Let's Encrypt [5], > >>>>> once certmonger gains support for it) > >>>>> > >>>>> c) automate external CA install, using any CA supported by certmonger [6] > >>>>> > >>>>> d) support multiple different CAs at once (generalization of the Sub-CA feature) > >>>>> > >>>>> e) uniform configuration on clients (configure once, use forever, even for > >>>>> CA-less) > >>>>> > >>>>> The idea is to store configuration for the different CAs in LDAP and have > >>>>> cert-request redirect requests to a proper CA helper according to that > >>>>> configuration. This would require a new certmonger D-Bus method to call a CA > >>>>> helper without associated certificate storage, but that should be rather easy > >>>>> to add. In return, it would be possible to do all of the above. > >>>>> > >>>>> Note that this should not conflict with tighter integration with Dogtag > >>>>> (profiles, ACLs, etc.). > >>>>> > >>>>> Comments are welcome. > >>>>> > >>>>> Honza > >>>>> > >>>>> [1] > >>>>> > >>>>> [2] > >>>>> > >>>>> > >>>>> [3] > >>>>> > >>>>> > >>>>> [4] > >>>>> > >>>>> > >>>>> [5] > >>>>> [6] > >>>> > >>>> Interesting idea! I would be definitely interested to hear what Fraser, Rob or > >>>> Simo thinks here. > >>> > >>> Sounds great to me in principle. > >>> > >>> How do you handle CAs that do not have automatic workflows for csr > >>> handling ? > >>> > >>> That's the reason we did the 2 step process (in reference to [6]) > >> > >> This could be handled by a dummy "manual" CA helper in certmonger, which > >> would dump the CSR into the filesystem and wait for the user to provide > >> the signed certificate in the filesystem and resume the certmonger request. > >> > >> As you pointed out, we currently do the same, although manually, in > >> external CA install. It is also done when renewing the externally signed > >> CA certificate - this time it is done using certmonger, but it is > >> handled by the dogtag-ipa-ca-renewal-agent helper rather than a separate > >> helper. > > > > The reason why we did the 2 step process is that it can take days in > > some cases to get back a cert given a csr. > > > > I do not think it is safe to wait for days mid-install. > > At the very least we'll need to be able to stop the install and resume > > it when the certs are available. > > I'm not suggesting to wait, we don't wait in ipa-cacert-manage when > renewing the CA certificate either. We can monitor the status of the > certmonger request and interrupt the install when it reaches > NEED_GUIDANCE (which means user intervention is needed). > \ Ok so we'll keep the 2 stages install for this case, fine with me and +1 to reducing parallel paths. Simo. From mbabinsk at redhat.com Wed Dec 16 15:03:22 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Wed, 16 Dec 2015 16:03:22 +0100 Subject: [Freeipa-devel] [TESTS][PATCH 0008] Fix tests for (stage)user plugin In-Reply-To: <56715E50.8010806@redhat.com> References: <56715E50.8010806@redhat.com> Message-ID: <56717D3A.2070106@redhat.com> On 12/16/2015 01:51 PM, Lenka Doudova wrote: > Hi, > > this patch fixes few Tracker methods for staged and 'normal' user, which > were mistakenly modified by my patch 0006.3. Applies for ipa-4-2 branch > only. > > Lenka > > ACK. -- Martin^3 Babinsky From mbasti at redhat.com Wed Dec 16 15:30:10 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 16 Dec 2015 16:30:10 +0100 Subject: [Freeipa-devel] [TESTS][PATCH 0008] Fix tests for (stage)user plugin In-Reply-To: <56717D3A.2070106@redhat.com> References: <56715E50.8010806@redhat.com> <56717D3A.2070106@redhat.com> Message-ID: <56718382.6050101@redhat.com> On 16.12.2015 16:03, Martin Babinsky wrote: > On 12/16/2015 01:51 PM, Lenka Doudova wrote: >> Hi, >> >> this patch fixes few Tracker methods for staged and 'normal' user, which >> were mistakenly modified by my patch 0006.3. Applies for ipa-4-2 branch >> only. >> >> Lenka >> >> > ACK. > Pushed to ipa-4-2: d62f023d75a659fa22bfb52de94b70eb084b7ac4 From abokovoy at redhat.com Wed Dec 16 15:39:28 2015 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 16 Dec 2015 17:39:28 +0200 Subject: [Freeipa-devel] certmonger everywhere In-Reply-To: <1450277084.17418.183.camel@redhat.com> References: <566FC72B.3050406@redhat.com> <56703075.7040108@redhat.com> <1450200176.17418.173.camel@redhat.com> <567101F2.30508@redhat.com> <1450276185.17418.178.camel@redhat.com> <56717812.40304@redhat.com> <1450277084.17418.183.camel@redhat.com> Message-ID: <20151216153928.GQ4620@redhat.com> On Wed, 16 Dec 2015, Simo Sorce wrote: >On Wed, 2015-12-16 at 15:41 +0100, Jan Cholasta wrote: >> On 16.12.2015 15:29, Simo Sorce wrote: >> > On Wed, 2015-12-16 at 07:17 +0100, Jan Cholasta wrote: >> >> On 15.12.2015 18:22, Simo Sorce wrote: >> >>> On Tue, 2015-12-15 at 16:23 +0100, Martin Kosek wrote: >> >>>> On 12/15/2015 08:54 AM, Jan Cholasta wrote: >> >>>>> Hi, >> >>>>> >> >>>>> recently I and David discussed the direction of installers with regard to >> >>>>> requesting certificates. Currently there are four (!) different ways of >> >>>>> requesting certificates in the installer [1][2][3][4]. We would like to reduce >> >>>>> it to one. >> >>>>> >> >>>>> Since all the certificates are tracked by certmonger and certmonger already >> >>>>> knows how to request certificates from Dogtag (and other CAs), we believe that >> >>>>> all certificates should be requested using certmonger. >> >>>>> >> >>>>> Taking our meditation further, we thought "Why not use certmonger for the >> >>>>> cert-request command as well?" What is the benefit, do you ask? >> >>>>> >> >>>>> a) single code path for requesting certificates (seriously, the current state >> >>>>> is ridiculous) >> >>>>> >> >>>>> b) use any CA supported by certmonger as the IPA CA (i.e. Let's Encrypt [5], >> >>>>> once certmonger gains support for it) >> >>>>> >> >>>>> c) automate external CA install, using any CA supported by certmonger [6] >> >>>>> >> >>>>> d) support multiple different CAs at once (generalization of the Sub-CA feature) >> >>>>> >> >>>>> e) uniform configuration on clients (configure once, use forever, even for >> >>>>> CA-less) >> >>>>> >> >>>>> The idea is to store configuration for the different CAs in LDAP and have >> >>>>> cert-request redirect requests to a proper CA helper according to that >> >>>>> configuration. This would require a new certmonger D-Bus method to call a CA >> >>>>> helper without associated certificate storage, but that should be rather easy >> >>>>> to add. In return, it would be possible to do all of the above. >> >>>>> >> >>>>> Note that this should not conflict with tighter integration with Dogtag >> >>>>> (profiles, ACLs, etc.). >> >>>>> >> >>>>> Comments are welcome. >> >>>>> >> >>>>> Honza >> >>>>> >> >>>>> [1] >> >>>>> >> >>>>> [2] >> >>>>> >> >>>>> >> >>>>> [3] >> >>>>> >> >>>>> >> >>>>> [4] >> >>>>> >> >>>>> >> >>>>> [5] >> >>>>> [6] >> >>>> >> >>>> Interesting idea! I would be definitely interested to hear what Fraser, Rob or >> >>>> Simo thinks here. >> >>> >> >>> Sounds great to me in principle. >> >>> >> >>> How do you handle CAs that do not have automatic workflows for csr >> >>> handling ? >> >>> >> >>> That's the reason we did the 2 step process (in reference to [6]) >> >> >> >> This could be handled by a dummy "manual" CA helper in certmonger, which >> >> would dump the CSR into the filesystem and wait for the user to provide >> >> the signed certificate in the filesystem and resume the certmonger request. >> >> >> >> As you pointed out, we currently do the same, although manually, in >> >> external CA install. It is also done when renewing the externally signed >> >> CA certificate - this time it is done using certmonger, but it is >> >> handled by the dogtag-ipa-ca-renewal-agent helper rather than a separate >> >> helper. >> > >> > The reason why we did the 2 step process is that it can take days in >> > some cases to get back a cert given a csr. >> > >> > I do not think it is safe to wait for days mid-install. >> > At the very least we'll need to be able to stop the install and resume >> > it when the certs are available. >> >> I'm not suggesting to wait, we don't wait in ipa-cacert-manage when >> renewing the CA certificate either. We can monitor the status of the >> certmonger request and interrupt the install when it reaches >> NEED_GUIDANCE (which means user intervention is needed). >> \ >Ok so we'll keep the 2 stages install for this case, fine with me and +1 >to reducing parallel paths. I guess this still wouldn't help automated installs as there are problems in running certmonger in Anaconda's chroot but it would probably be unlikely that someone would do automated installs with two-stage process. -- / Alexander Bokovoy From ofayans at redhat.com Wed Dec 16 15:50:15 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Wed, 16 Dec 2015 16:50:15 +0100 Subject: [Freeipa-devel] [PATCH] ca-less tests updated - POC In-Reply-To: <567176A2.1000901@redhat.com> References: <56337745.109@redhat.com> <563B091B.3010501@redhat.com> <563B6A96.8090606@redhat.com> <563BABEC.8080304@redhat.com> <563C5B1A.1090803@redhat.com> <563C5E6D.4060307@redhat.com> <563CA575.1030808@redhat.com> <567176A2.1000901@redhat.com> Message-ID: <56718837.9010707@redhat.com> Hi David, On 12/16/2015 03:35 PM, David Kupka wrote: > On 06/11/15 14:04, Oleg Fayans wrote: >> Hi Jan, >> >> On 11/06/2015 09:01 AM, Jan Cholasta wrote: >>> Actually it might be better to keep them, but fix them to expect >>> ipa-server-certinstall to success. >> >> Done. Updated patch attached. >> Also in the patch 0013 I removed a trailing whitespace which caused lint >> to complain >> >> Now with domain level 0 the test output looks like this: >> >> [11:40:51]ofayans at vm-076:~]$ ipa-run-tests >> test_integration/test_caless.py >> ==================================================================================== >> >> test session starts >> ===================================================================================== >> >> >> platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.6.4 >> plugins: multihost, sourceorder >> collected 88 items >> >> test_integration/test_caless.py >> ......xx......ss...............xx........ss....................xx......ss............... >> >> >> >> ===================================================================== 76 >> passed, 6 skipped, 6 xfailed in 7871.10 seconds >> ===================================================================== >> >> >>> >>> On 6.11.2015 08:47, Jan Cholasta wrote: >>>> Hi Oleg, >>>> >>>> I think you can just remove >>>> TestCertinstall.test_{http,ds}_intermediate_ca, the certificates are >>>> imported correctly in this case and I didn't see anything break. >>>> >>>> Honza >>>> >>>> On 5.11.2015 20:20, Oleg Fayans wrote: >>>>> Patch 0014 updated and passes lint >>>>> >>>>> On 11/05/2015 03:41 PM, Oleg Fayans wrote: >>>>>> Wait a bit, the patch has problems with pylint: it does not build :) >>>>>> The updated version (without the setupmaster nonsense) is being >>>>>> tested >>>>>> now. >>>>>> >>>>>> On 11/05/2015 08:45 AM, Oleg Fayans wrote: >>>>>>> Hi Jan, >>>>>>> >>>>>>> Could you take a look at these, whenever you are free? >>>>>>> >>>>>>> On 10/30/2015 02:57 PM, Oleg Fayans wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> The following patches contain updates to ca-less integration tests. >>>>>>>> It's still a proof of concept: 2 tests still fail seemingly due to >>>>>>>> the >>>>>>>> change in target system logic (marked as xfail with "ask jcholast >>>>>>>> comment") >>>>>>>> >>>>>>>> The test output looks like this: >>>>>>>> >>>>>>>> $ ipa-run-tests test_integration/test_caless.py --pdb >>>>>>>> ==================================================================================== >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> test session starts >>>>>>>> ===================================================================================== >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.6.4 >>>>>>>> plugins: multihost, sourceorder >>>>>>>> collected 88 items >>>>>>>> >>>>>>>> test_integration/test_caless.py >>>>>>>> ......xx......ss............sssssssssssssssssss.ssssss.........xx......ssxx............. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ==================================================================== >>>>>>>> >>>>>>>> 53 >>>>>>>> >>>>>>>> passed, 29 skipped, 6 xfailed in 5620.17 seconds >>>>>>>> ===================================================================== >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Numerous skips correspond to the tests related to >>>>>>>> ipa-replica-prepare >>>>>>>> (unsupported under domain level 1) >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> >>>> >>>> >>> >>> >> >> >> >> This body part will be downloaded on demand. >> > Hello, thanks for updated patches. I'm really sorry it took so long > before I got to them. > There was change in ipapython.ipautil.run that happened after you sent > the patches. Feel free to squash attached patch that fixes it. Already noticed this and made the similar fix. > > Unfortunately I see a lot of test failing with domain-level 0: > http://fpaste.org/301657/50275682/ > > domain-level 1 (domain-level 1: http://fpaste.org/301658/02757191/) > seems better. There are 2 failing test that you're probably mentioning > in commit message plus one that I think is bug in code rather than bug > in tests. > Do you have any proposal for fixing the two failing tests? I am working on it right now. > > One nitpick: Please use mail for notes like "need further consulting > ..." rather that commit message. When the patch gets accepted it will > still need modification before push just because inappropriate commit > message. Good to know, thank you! > > Thank you! -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From mbasti at redhat.com Wed Dec 16 15:55:04 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 16 Dec 2015 16:55:04 +0100 Subject: [Freeipa-devel] [PATCH 0364, 0367] ipa-kra-install: allow first KRA to be installed on replica In-Reply-To: <56658A2D.2010803@redhat.com> References: <56587E92.7060008@redhat.com> <565C4C31.4070509@redhat.com> <565C877B.3050405@redhat.com> <565EF7A8.7050304@redhat.com> <565F1B3B.5080905@redhat.com> <56658A2D.2010803@redhat.com> Message-ID: <56718958.8020300@redhat.com> On 07.12.2015 14:31, Martin Babinsky wrote: > On 12/02/2015 05:24 PM, Martin Basti wrote: >> >> >> On 02.12.2015 14:52, Martin Babinsky wrote: >>> On 11/30/2015 06:29 PM, Martin Basti wrote: >>>> >>>> >>>> On 30.11.2015 14:16, Martin Babinsky wrote: >>>>> On 11/27/2015 05:02 PM, Martin Basti wrote: >>>>>> https://fedorahosted.org/freeipa/ticket/5460 >>>>>> >>>>>> I tested just master, I will test ipa-4-2 later. >>>>>> patch attached. >>>>>> >>>>>> >>>>> >>>>> ACK for the master branch. >>>>> >>>> Thanks, additional patch improves error message when >>>> ipa-replica-install >>>> --setup-ca --setup-kra is executed and KRA is not installed anywhere >>>> yet. >>>> >>>> I'm working on patches for ipa-4-2 branch >>>> >>>> Martin >>> >>> ACK for patch 367. >>> >> Pushed to master: bbbe411f357b7fbad533b5211a90bb0558b1abbe >> >> IPA 4.2 patches attached. > > ACK. > Pushed to ipa-4-2: cacca7bade36d5b01dd8c3568e41abb2b183aa50 From mbasti at redhat.com Wed Dec 16 16:47:53 2015 From: mbasti at redhat.com (Martin Basti) Date: Wed, 16 Dec 2015 17:47:53 +0100 Subject: [Freeipa-devel] [PATCH 016 - 017] First part of the replica promotion tests + testplan In-Reply-To: <56717785.4020304@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> <565DB7D3.5040500@redhat.com> <565F0BDD.8080206@redhat.com> <565F17EE.3020701@redhat.com> <5666DC18.2040906@redhat.com> <56675E31.3050203@redhat.com> <5667F49B.5020709@redhat.com> <5667FEFB.6090904@redhat.com> <56680C2E.7060805@redhat.com> <56680DDD.2000106@redhat.com> <566FDD80.7030805@redhat.com> <56717785.4020304@redhat.com> Message-ID: <567195B9.9070104@redhat.com> On 16.12.2015 15:39, Martin Basti wrote: > > > On 15.12.2015 10:29, Oleg Fayans wrote: >> Hi Martin, >> >> The updated patches are attached. Patch 0017 includes all changes from >> patch 0018, so, if you approve this one, there would be no need to >> continue with the review of 0018. This one contains all changes related >> to you remarks from 0018 review. Please see my explanation on the >> stdout+stderr part in the thread from patch 0018. >> With these two patches applied one of the tests fails due this bug: >> https://fedorahosted.org/freeipa/ticket/5550 >> >> On 12/09/2015 12:17 PM, Martin Basti wrote: >>> >>> On 09.12.2015 12:10, Martin Basti wrote: >>>> >>>> On 09.12.2015 11:14, Oleg Fayans wrote: >>>>> Hi Martin >>>>> >>>>> On 12/09/2015 10:30 AM, Martin Basti wrote: >>>>>> On 08.12.2015 23:48, Oleg Fayans wrote: >>>>>>> Substituted a hardcoded suffix name with a constant >>>>>>> DOMAIN_SUFFIX_NAME >>>>>>> >>>>>>> On 12/08/2015 02:33 PM, Oleg Fayans wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> >>>>>>>> The patches are rebased against the current master. >>>>>>>> >>>>>>>> On 12/02/2015 05:10 PM, Martin Basti wrote: >>>>>>>>> On 02.12.2015 16:18, Oleg Fayans wrote: >>>>>>>>>> Hi Martin, >>>>>>>>>> >>>>>>>>>> On 12/01/2015 04:08 PM, Martin Basti wrote: >>>>>>>>>>> On 27.11.2015 16:26, Oleg Fayans wrote: >>>>>>>>>>>> And patch N 16 passes lint too: >>>>>>>>>>>> >>>>>>>>>>>> On 11/27/2015 04:03 PM, Oleg Fayans wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> On 11/27/2015 03:26 PM, Martin Basti wrote: >>>>>>>>>>>>>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> All your suggestions were taken into account. Both >>>>>>>>>>>>>>> patches are >>>>>>>>>>>>>>> updated. Thank you for your help! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>>>>>>>>>>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I agree to all your points but one. please, see my >>>>>>>>>>>>>>>>> comment >>>>>>>>>>>>>>>>> below >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 0) Note >>>>>>>>>>>>>>>>>> Please be aware of >>>>>>>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5469 >>>>>>>>>>>>>>>>>> during >>>>>>>>>>>>>>>>>> KRA testing >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, >>>>>>>>>>>>>>>>>> this may >>>>>>>>>>>>>>>>>> change >>>>>>>>>>>>>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for >>>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>>> level 0 >>>>>>>>>>>>>>>>>> and 1 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>>>>> Why uninstall KRA then server, is not enough just >>>>>>>>>>>>>>>>>> uninstall >>>>>>>>>>>>>>>>>> server >>>>>>>>>>>>>>>>>> which >>>>>>>>>>>>>>>>>> covers KRA uninstall? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> + def teardown_method(self, method): >>>>>>>>>>>>>>>>>> + for host in self.replicas: >>>>>>>>>>>>>>>>>> + host.run_command(self.kra_uninstall, >>>>>>>>>>>>>>>>>> raiseonerr=False) >>>>>>>>>>>>>>>>>> + tasks.uninstall_master(host) >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>>>>> Can be this function more generic? It should allow >>>>>>>>>>>>>>>>>> specify >>>>>>>>>>>>>>>>>> host >>>>>>>>>>>>>>>>>> where >>>>>>>>>>>>>>>>>> KRA should be installed not just master >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> + def test_kra_install_master(self): >>>>>>>>>>>>>>>>>> + self.master.run_command(self.kra_install) >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> TestLevel0(Dummy): >>>>>>>>>>>>>>>>>> Can be the test name more specific, something like >>>>>>>>>>>>>>>>>> TestReplicaPromotionLevel0 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 5) >>>>>>>>>>>>>>>>>> please remove this, the patch is on review and it >>>>>>>>>>>>>>>>>> will be >>>>>>>>>>>>>>>>>> pushed >>>>>>>>>>>>>>>>>> sooner >>>>>>>>>>>>>>>>>> than tests >>>>>>>>>>>>>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> and as I mentioned in ticket #5455, I cannot reproduce >>>>>>>>>>>>>>>>>> it with >>>>>>>>>>>>>>>>>> ipa-kra-install, so please provide steps to reproduce if >>>>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>>> insist >>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>> this still does not work as expected with KRA. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 6) This is completely wrong, it removes everything >>>>>>>>>>>>>>>>>> that we >>>>>>>>>>>>>>>>>> tried to >>>>>>>>>>>>>>>>>> achieve with previous patches with domain level in CI >>>>>>>>>>>>>>>>> Actually, being able to configure domain level per class >>>>>>>>>>>>>>>>> is WAY >>>>>>>>>>>>>>>>> more >>>>>>>>>>>>>>>>> convenient, than to always have to think which domain >>>>>>>>>>>>>>>>> level is >>>>>>>>>>>>>>>>> appropriate for which particular test during jenkins job >>>>>>>>>>>>>>>>> configuration. In fact, I should have thought about it >>>>>>>>>>>>>>>>> from the >>>>>>>>>>>>>>>>> very >>>>>>>>>>>>>>>>> beginning. For example, in test_replica_promotion.py we >>>>>>>>>>>>>>>>> have on >>>>>>>>>>>>>>>>> class, >>>>>>>>>>>>>>>>> which intiates with domain level = 1, while others - with >>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>>> 0. With config-based approach, we would have to >>>>>>>>>>>>>>>>> implement a >>>>>>>>>>>>>>>>> separate >>>>>>>>>>>>>>>>> step that raises domain level. Overall, I am against the >>>>>>>>>>>>>>>>> approach, >>>>>>>>>>>>>>>>> when you have to remember to set certain domain level in >>>>>>>>>>>>>>>>> config >>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>> any particular test. The tests themselves should be >>>>>>>>>>>>>>>>> aware of >>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> domain level they need. >>>>>>>>>>>>>>>> I do not say that we should not have something that >>>>>>>>>>>>>>>> overrides >>>>>>>>>>>>>>>> settings >>>>>>>>>>>>>>>> in from config in a particular test case, I say your >>>>>>>>>>>>>>>> patch is >>>>>>>>>>>>>>>> doing it >>>>>>>>>>>>>>>> wrong. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I agree it is useful to have param domain_level in >>>>>>>>>>>>>>>> install_master, >>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by >>>>>>>>>>>>>>>> default, >>>>>>>>>>>>>>>> because with your current patch the domain_level in >>>>>>>>>>>>>>>> config is >>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>> used >>>>>>>>>>>>>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> For example I want to achieve this goal: >>>>>>>>>>>>>>>> test_vault.py, this test suite can run on domain level1 >>>>>>>>>>>>>>>> and on >>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>> level0, so with one test we can test 2 domain levels just >>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>> putting >>>>>>>>>>>>>>>> domain level into config file. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I agree that with extraordinary test like replica >>>>>>>>>>>>>>>> promotion test >>>>>>>>>>>>>>>> is, we >>>>>>>>>>>>>>>> need something that allows override the config file. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> As I said bellow, domain_level default value should be >>>>>>>>>>>>>>>> None in >>>>>>>>>>>>>>>> install_master and install_topo plugin. If domain level >>>>>>>>>>>>>>>> was >>>>>>>>>>>>>>>> specified >>>>>>>>>>>>>>>> use the specified one, if not (value is None) use the >>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>> from >>>>>>>>>>>>>>>> config file. >>>>>>>>>>>>>>> Agreed :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Martin >>>>>>>>>>>>>>>>>> [PATCH] Enabled setting domain_level per class >>>>>>>>>>>>>>>>>> derived from >>>>>>>>>>>>>>>>>> TestIntegration >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> When I configure domain level 0 in yaml config, how >>>>>>>>>>>>>>>>>> is this >>>>>>>>>>>>>>>>>> supposed to >>>>>>>>>>>>>>>>>> get into install methods when you removed that code? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>>>>>>>>>>>>> + "--domain-level=%i" % domain_level >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> You always use MAX_DOMAIN_LEVEL in this case or >>>>>>>>>>>>>>>>>> whatever is >>>>>>>>>>>>>>>>>> specified in >>>>>>>>>>>>>>>>>> domain_level option. >>>>>>>>>>>>>>>>>> I suggest to use domain_level=None, and when it is >>>>>>>>>>>>>>>>>> None use >>>>>>>>>>>>>>>>>> 'host.config.domain_level', if it is not None, use >>>>>>>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> With this we can specify domain level in config file for >>>>>>>>>>>>>>>>>> test >>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>> be used for both domain levels and you can manually >>>>>>>>>>>>>>>>>> specify >>>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>>>> for test that requires specific domain level. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Also this should go away >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> @classmethod >>>>>>>>>>>>>>>>>> def install(cls, mh): >>>>>>>>>>>>>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>>>>>>>>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>>>>>>>>>>>>> if cls.topology is None: >>>>>>>>>>>>>>>>>> return >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I do not see reason why test should override >>>>>>>>>>>>>>>>>> configuration in >>>>>>>>>>>>>>>>>> config in >>>>>>>>>>>>>>>>>> this case. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Martin >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Here is the updated version of the patch (more tests + >>>>>>>>>>>>>>>>>>> fixed the >>>>>>>>>>>>>>>>>>> issues of the first one) + patch 0017, that >>>>>>>>>>>>>>>>>>> implements the >>>>>>>>>>>>>>>>>>> necessary >>>>>>>>>>>>>>>>>>> changes in the background code, i. e. patch 16 does not >>>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>>> without >>>>>>>>>>>>>>>>>>> patch 17 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>>>>>>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>>>>>>>>>>>>> Hi guys, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Here are first two automated testcases from this >>>>>>>>>>>>>>>>>>>>> (so far >>>>>>>>>>>>>>>>>>>>> incomplete) >>>>>>>>>>>>>>>>>>>>> testplan: >>>>>>>>>>>>>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> PATCH 16: NACK >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>>>>>>>>>>>>> 'domain_level' to >>>>>>>>>>>>>>>>>>>> install_topo()? >>>>>>>>>>>>>>>>>>>> Also it is good practise to add new option as the last >>>>>>>>>>>>>>>>>>>> parameter. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>>>>>>> cab you in both tests specify a domain level with >>>>>>>>>>>>>>>>>>>> constant >>>>>>>>>>>>>>>>>>>> instead of >>>>>>>>>>>>>>>>>>>> number literal? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>>>>>>> both test call install_topo with custom domain level, >>>>>>>>>>>>>>>>>>>> but it >>>>>>>>>>>>>>>>>>>> cannot >>>>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>>>> because 1) (did you run the test?) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>>>>>>>>>>>>> Respectively why there is call of install_topo() that >>>>>>>>>>>>>>>>>>>> installs >>>>>>>>>>>>>>>>>>>> replica. >>>>>>>>>>>>>>>>>>>> As this test just tests that ipa-replica-prepare is >>>>>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>> working >>>>>>>>>>>>>>>>>>>> anymore, >>>>>>>>>>>>>>>>>>>> is it worth to spend 20 minutes with installing >>>>>>>>>>>>>>>>>>>> replica and >>>>>>>>>>>>>>>>>>>> then >>>>>>>>>>>>>>>>>>>> just no >>>>>>>>>>>>>>>>>>>> tot use it? IMO to install master in install step is >>>>>>>>>>>>>>>>>>>> enough. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Martin^2 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> ./make-lint >>>>>>>>>>>>>> ************* Module ipatests.test_integration.base >>>>>>>>>>>>>> ipatests/test_integration/base.py:66: [E1101(no-member), >>>>>>>>>>>>>> IntegrationTest.install] Class 'IntegrationTest' has no >>>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>>> member) >>>>>>>>>>>>>> ************* Module >>>>>>>>>>>>>> ipatests.test_integration.test_replica_promotion >>>>>>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:16: >>>>>>>>>>>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no >>>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>>> member) >>>>>>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:117: >>>>>>>>>>>>>> [E1101(no-member), >>>>>>>>>>>>>> TestCAInstall.test_ca_install_without_replica_file] >>>>>>>>>>>>>> Module 'ipatests.test_integration.tasks' has no >>>>>>>>>>>>>> 'setup_replica' >>>>>>>>>>>>>> member) >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Is it so hard to run pylint before patch is posted for >>>>>>>>>>>>>> review? >>>>>>>>>>>>> Sorry, my bad. Fixed. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> 1) >>>>>>>>>>> Why is this change in the patch? >>>>>>>>>>> - # Clean up the test directory >>>>>>>>>>> - host.run_command(['rm', '-rvf', host.config.test_dir]) >>>>>>>>>> Otherwise 2 out of 8 tests fail with IOError at line 78 of >>>>>>>>>> ipatests/test_integration/tasks.py >>>>>>>>>> >>>>>>>>>> I do not understand yet how does this happen, but if you remove >>>>>>>>>> ipatests folder once, it then fails to be created again. >>>>>>>>>> >>>>>>>>> So this should be in separated patch and investigated properly. >>>>>>>> Agree. Removed >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 2) >>>>>>>>>>> is enough to have this check only in install_master, >>>>>>>>>>> install_topo can >>>>>>>>>>> pass None to install_master >>>>>>>>>>> + if domain_level is None: >>>>>>>>>>> + domain_level = master.config.domain_level >>>>>>>>>> Done >>>>>>>>>> >>>>>>>>>>> 3) >>>>>>>>>>> IMO replica-manage del should cleanup hosts entry, so following >>>>>>>>>>> code >>>>>>>>>>> should not be needed. >>>>>>>>>>> + if cleanhost: >>>>>>>>>>> + kinit_admin(master) >>>>>>>>>>> + master.run_command(["ipa", "host-del", "--updatedns", >>>>>>>>>>> replica.hostname], >>>>>>>>>>> + raiseonerr=False) >>>>>>>>>> Well, in fact it does not. At least the corresponding dns record >>>>>>>>>> stays >>>>>>>>>> and causes the subsequent ipa-client-install to fail. Probably >>>>>>>>>> it's a >>>>>>>>>> bug. On the other hand, if I promote an existing client to >>>>>>>>>> replica and >>>>>>>>>> then delete this replica, then, I probably want the host record >>>>>>>>>> (that >>>>>>>>>> was created during client-install) to stay in the system. So, >>>>>>>>>> does not >>>>>>>>>> look like a bug to me. >>>>>>>>> No you don't, because replica uninstallation also removes the >>>>>>>>> client. >>>>>>>>> >>>>>>>>> I tried it with ipa42, ipa-replica-manage del removes host entry, >>>>>>>>> and >>>>>>>>> DNS A records, only SSHFP records stay there (I'm not sure if it >>>>>>>>> is bug >>>>>>>>> or feature) >>>>>>>>> >>>>>>>>> Also I received this message >>>>>>>>> """ >>>>>>>>> Failed to cleanup replica1.ipa.test DNS entries: no matching >>>>>>>>> entry >>>>>>>>> found >>>>>>>>> You may need to manually remove them from the tree >>>>>>>>> """ >>>>>>>>> But, A record has been removed, so this is probably false >>>>>>>>> positive and >>>>>>>>> it needs to have a ticket. >>>>>>>> Agree, that was an issue with my setup. >>>>>>>> Removed >>>>>>>> >>>>>>>>>>> 4) >>>>>>>>>>> This variable is not used >>>>>>>>>>> + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] >>>>>>>>>> Removed >>>>>>>>>> >>>>>>>>>>> 5) >>>>>>>>>>> Why do you need this >>>>>>>>>>> + kra_install = ["ipa-kra-install", "-U", "-p", >>>>>>>>>>> config.dirman_password] >>>>>>>>>>> when you implemented tasks.install_kra that returns the exactly >>>>>>>>>>> the same >>>>>>>>>>> result? >>>>>>>>>> Right. Removed >>>>>>>>>> >>>>>>>>>>> 6) >>>>>>>>>>> PEP8 >>>>>>>>>>> ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 >>>>>>>>>>> blank >>>>>>>>>>> lines, >>>>>>>>>>> found 1 >>>>>>>>>>> ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 >>>>>>>>>>> blank >>>>>>>>>>> lines, >>>>>>>>>>> found 1 >>>>>>>>>>> ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 >>>>>>>>>>> blank >>>>>>>>>>> lines, >>>>>>>>>>> found 1 >>>>>>>>>>> ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 >>>>>>>>>>> blank >>>>>>>>>>> lines, >>>>>>>>>>> found 1 >>>>>>>>>>> ./ipatests/test_integration/tasks.py:950:80: E501 line too long >>>>>>>>>>> (80 > 79 >>>>>>>>>>> characters) >>>>>>>>>>> >>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:29:80: >>>>>>>>>>> E501 >>>>>>>>>>> line >>>>>>>>>>> too long (85 > 79 characters) >>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:64:80: >>>>>>>>>>> E501 >>>>>>>>>>> line >>>>>>>>>>> too long (85 > 79 characters) >>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:67:80: >>>>>>>>>>> E501 >>>>>>>>>>> line >>>>>>>>>>> too long (88 > 79 characters) >>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:93:80: >>>>>>>>>>> E501 >>>>>>>>>>> line >>>>>>>>>>> too long (80 > 79 characters) >>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:94:80: >>>>>>>>>>> E501 >>>>>>>>>>> line >>>>>>>>>>> too long (83 > 79 characters) >>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 >>>>>>>>>>> >>>>>>>>>>> line >>>>>>>>>>> too long (81 > 79 characters) >>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 >>>>>>>>>>> >>>>>>>>>>> line >>>>>>>>>>> too long (80 > 79 characters) >>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 >>>>>>>>>>> >>>>>>>>>>> line >>>>>>>>>>> too long (82 > 79 characters) >>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 >>>>>>>>>>> >>>>>>>>>>> line >>>>>>>>>>> too long (80 > 79 characters) >>>>>>>>>> Most of these complaints are unrelated to the current patches. >>>>>>>>>> It's better to create a separate patch addressing PEP8 errors. >>>>>>>>>> >>>>>>>>> I beg for your pardon, those PEP8 errors have been introduced by >>>>>>>>> your >>>>>>>>> patches. >>>>>>>> Fixed >>>>>>>> >>>>>>>>>>> 7) >>>>>>>>>>> Why this must be stored in instance? IMO to have it stored as >>>>>>>>>>> local >>>>>>>>>>> variable is perfect >>>>>>>>>>> TestKRAInstall, TestCAInstall >>>>>>>>>>> self.replica1_filename = >>>>>>>>>>> tasks.get_replica_filename(replica1) >>>>>>>>>>> self.replica2_filename = >>>>>>>>>>> tasks.get_replica_filename(replica2) >>>>>>>>>> Agree. Fixed >>>>>>>>>> >>>>>> This patch is missing something. >>>>> I am sorry, I forgot to revert my previous change. The correct >>>>> patch is >>>>> attached >>>>> >>>> ************* Module ipatests.test_integration.test_replica_promotion >>>> ipatests/test_integration/test_replica_promotion.py:15: >>>> [E1123(unexpected-keyword-arg), Dummy.install] Unexpected keyword >>>> argument 'domain_level' in function call) >>>> ipatests/test_integration/test_replica_promotion.py:15: >>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no 'domain_level' >>>> member) >>>> ipatests/test_integration/test_replica_promotion.py:19: >>>> [E1101(no-member), Dummy.teardown_method] Module >>>> 'ipatests.test_integration.tasks' has no 'uninstall_replica' member) >>>> ipatests/test_integration/test_replica_promotion.py:67: >>>> [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] >>>> Module 'ipatests.test_integration.tasks' has no 'ipa_backup' member) >>>> ipatests/test_integration/test_replica_promotion.py:72: >>>> [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] >>>> Module 'ipatests.test_integration.tasks' has no 'ipa_restore' member) >>>> ipatests/test_integration/test_replica_promotion.py:120: >>>> [E1123(unexpected-keyword-arg), TestCAInstall.install] Unexpected >>>> keyword argument 'domain_level' in function call) >>>> >>> Sorry I forgot to apply patch 17, my bad, I'm continuing with review > LGTM, I haven't had time to test it, but if you are sure that test is > working, we may push this. > Is this expected due the bug you mentioned? _____ __________________________________________________________________________ TestReplicaPromotionLevel0.test_kra_install_master ________________________________________________________________________________ self = def test_kra_install_master(self): result1 = tasks.install_kra(self.master, raiseonerr=False) > assert result1.returncode == 0, result1.stderr_text E AssertionError: Usage: ipa-kra-install [options] [replica_file] E E ipa-kra-install: error: Replica file /root/ipatests/replica-info.gpg does not exist E The ipa-kra-install command failed. See /var/log/ipaserver-kra-install.log for more information E E assert 2 == 0 E + where 2 = .returncode From mkubik at redhat.com Wed Dec 16 18:14:25 2015 From: mkubik at redhat.com (=?UTF-8?Q?Milan_Kub=c3=adk?=) Date: Wed, 16 Dec 2015 19:14:25 +0100 Subject: [Freeipa-devel] [patch 0026] ipatests: replace the test-example.com domain in tests Message-ID: <5671AA01.80408@redhat.com> Applies to ipa-4-3 and master. Reason in commit message. -- Milan Kubik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkubik-0026-ipatests-replace-the-test-example.com-domain-in-test.patch Type: text/x-patch Size: 8616 bytes Desc: not available URL: From mbasti at redhat.com Thu Dec 17 01:11:40 2015 From: mbasti at redhat.com (Martin Basti) Date: Thu, 17 Dec 2015 02:11:40 +0100 Subject: [Freeipa-devel] [PATCH 0379 - 0393] Winter code cleanup Message-ID: <56720BCC.1050504@redhat.com> Hello, following patches cleanup the code and add checks to pylint to avoid the mess again Patches: 379-381: the most important patches, they cleanup imports Patch 382: enables various pylint checks, we may reduce the list of check if needed. Patches 383 - 393: remove minor issues from code, and enable particular checks Feel free to nack patches/checks that should not be there. Please apply patches in order. Martin^2 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0379-Remove-empty-test-file.patch Type: text/x-patch Size: 1537 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0380-Remove-unused-imports.patch Type: text/x-patch Size: 104888 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0381-Remove-wildcard-imports.patch Type: text/x-patch Size: 44245 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0382-Enable-multiple-warnings-checks-in-Pylint.patch Type: text/x-patch Size: 2386 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0383-Enable-pylint-lost-exception-check.patch Type: text/x-patch Size: 1685 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0384-Enable-pylint-duplicated-key-check.patch Type: text/x-patch Size: 1954 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0385-Enable-pylint-trailing-whitespace-check.patch Type: text/x-patch Size: 3905 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0386-Enable-pylint-missing-final-newline-check.patch Type: text/x-patch Size: 1664 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0387-Enable-pylint-unused-format-string-key-check.patch Type: text/x-patch Size: 2760 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0388-Enable-pylint-expression-not-assigned-check.patch Type: text/x-patch Size: 8097 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0389-Enable-pylint-pointless-statement-check.patch Type: text/x-patch Size: 5280 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0390-Enable-pylint-empty-docstring-check.patch Type: text/x-patch Size: 2630 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0391-Enable-pylint-unnecessary-lambda-check.patch Type: text/x-patch Size: 5916 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0392-Enable-pylint-eval-and-exec-usage-checks.patch Type: text/x-patch Size: 2227 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0393-Enable-pylint-unnecessary-pass-check.patch Type: text/x-patch Size: 11013 bytes Desc: not available URL: From jcholast at redhat.com Thu Dec 17 06:54:44 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 17 Dec 2015 07:54:44 +0100 Subject: [Freeipa-devel] [PATCH 0379 - 0393] Winter code cleanup In-Reply-To: <56720BCC.1050504@redhat.com> References: <56720BCC.1050504@redhat.com> Message-ID: <56725C34.5060400@redhat.com> Hi, On 17.12.2015 02:11, Martin Basti wrote: > Hello, > following patches cleanup the code and add checks to pylint to avoid the > mess again > > Patches: 379-381: > the most important patches, they cleanup imports > > Patch 382: > enables various pylint checks, we may reduce the list of check if needed. Would it be possible to turn this into a blacklist of disabled warnings rather than a whitelist of enabled warnings? > > Patches 383 - 393: > remove minor issues from code, and enable particular checks > Feel free to nack patches/checks that should not be there. > > Please apply patches in order. > > Martin^2 > > Honza -- Jan Cholasta From ofayans at redhat.com Thu Dec 17 09:04:39 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Thu, 17 Dec 2015 10:04:39 +0100 Subject: [Freeipa-devel] [PATCH 016 - 017] First part of the replica promotion tests + testplan In-Reply-To: <567195B9.9070104@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> <565DB7D3.5040500@redhat.com> <565F0BDD.8080206@redhat.com> <565F17EE.3020701@redhat.com> <5666DC18.2040906@redhat.com> <56675E31.3050203@redhat.com> <5667F49B.5020709@redhat.com> <5667FEFB.6090904@redhat.com> <56680C2E.7060805@redhat.com> <56680DDD.2000106@redhat.com> <566FDD80.7030805@redhat.com> <56717785.4020304@redhat.com> <567195B9.9070104@redhat.com> Message-ID: <56727AA7.2030809@redhat.com> Hi Martin, I am sorry, in my previous email I attached the old version of patch 0016. The correct on is attached. On 12/16/2015 05:47 PM, Martin Basti wrote: > > > On 16.12.2015 15:39, Martin Basti wrote: >> >> >> On 15.12.2015 10:29, Oleg Fayans wrote: >>> Hi Martin, >>> >>> The updated patches are attached. Patch 0017 includes all changes from >>> patch 0018, so, if you approve this one, there would be no need to >>> continue with the review of 0018. This one contains all changes related >>> to you remarks from 0018 review. Please see my explanation on the >>> stdout+stderr part in the thread from patch 0018. >>> With these two patches applied one of the tests fails due this bug: >>> https://fedorahosted.org/freeipa/ticket/5550 >>> >>> On 12/09/2015 12:17 PM, Martin Basti wrote: >>>> >>>> On 09.12.2015 12:10, Martin Basti wrote: >>>>> >>>>> On 09.12.2015 11:14, Oleg Fayans wrote: >>>>>> Hi Martin >>>>>> >>>>>> On 12/09/2015 10:30 AM, Martin Basti wrote: >>>>>>> On 08.12.2015 23:48, Oleg Fayans wrote: >>>>>>>> Substituted a hardcoded suffix name with a constant >>>>>>>> DOMAIN_SUFFIX_NAME >>>>>>>> >>>>>>>> On 12/08/2015 02:33 PM, Oleg Fayans wrote: >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> >>>>>>>>> The patches are rebased against the current master. >>>>>>>>> >>>>>>>>> On 12/02/2015 05:10 PM, Martin Basti wrote: >>>>>>>>>> On 02.12.2015 16:18, Oleg Fayans wrote: >>>>>>>>>>> Hi Martin, >>>>>>>>>>> >>>>>>>>>>> On 12/01/2015 04:08 PM, Martin Basti wrote: >>>>>>>>>>>> On 27.11.2015 16:26, Oleg Fayans wrote: >>>>>>>>>>>>> And patch N 16 passes lint too: >>>>>>>>>>>>> >>>>>>>>>>>>> On 11/27/2015 04:03 PM, Oleg Fayans wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 11/27/2015 03:26 PM, Martin Basti wrote: >>>>>>>>>>>>>>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>>>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> All your suggestions were taken into account. Both >>>>>>>>>>>>>>>> patches are >>>>>>>>>>>>>>>> updated. Thank you for your help! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>>>>>>>>>>>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>>>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I agree to all your points but one. please, see my >>>>>>>>>>>>>>>>>> comment >>>>>>>>>>>>>>>>>> below >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 0) Note >>>>>>>>>>>>>>>>>>> Please be aware of >>>>>>>>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5469 >>>>>>>>>>>>>>>>>>> during >>>>>>>>>>>>>>>>>>> KRA testing >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, >>>>>>>>>>>>>>>>>>> this may >>>>>>>>>>>>>>>>>>> change >>>>>>>>>>>>>>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for >>>>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>>>> level 0 >>>>>>>>>>>>>>>>>>> and 1 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>>>>>> Why uninstall KRA then server, is not enough just >>>>>>>>>>>>>>>>>>> uninstall >>>>>>>>>>>>>>>>>>> server >>>>>>>>>>>>>>>>>>> which >>>>>>>>>>>>>>>>>>> covers KRA uninstall? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> + def teardown_method(self, method): >>>>>>>>>>>>>>>>>>> + for host in self.replicas: >>>>>>>>>>>>>>>>>>> + host.run_command(self.kra_uninstall, >>>>>>>>>>>>>>>>>>> raiseonerr=False) >>>>>>>>>>>>>>>>>>> + tasks.uninstall_master(host) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>>>>>> Can be this function more generic? It should allow >>>>>>>>>>>>>>>>>>> specify >>>>>>>>>>>>>>>>>>> host >>>>>>>>>>>>>>>>>>> where >>>>>>>>>>>>>>>>>>> KRA should be installed not just master >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> + def test_kra_install_master(self): >>>>>>>>>>>>>>>>>>> + self.master.run_command(self.kra_install) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> TestLevel0(Dummy): >>>>>>>>>>>>>>>>>>> Can be the test name more specific, something like >>>>>>>>>>>>>>>>>>> TestReplicaPromotionLevel0 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 5) >>>>>>>>>>>>>>>>>>> please remove this, the patch is on review and it >>>>>>>>>>>>>>>>>>> will be >>>>>>>>>>>>>>>>>>> pushed >>>>>>>>>>>>>>>>>>> sooner >>>>>>>>>>>>>>>>>>> than tests >>>>>>>>>>>>>>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> and as I mentioned in ticket #5455, I cannot reproduce >>>>>>>>>>>>>>>>>>> it with >>>>>>>>>>>>>>>>>>> ipa-kra-install, so please provide steps to reproduce if >>>>>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>>>> insist >>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>> this still does not work as expected with KRA. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 6) This is completely wrong, it removes everything >>>>>>>>>>>>>>>>>>> that we >>>>>>>>>>>>>>>>>>> tried to >>>>>>>>>>>>>>>>>>> achieve with previous patches with domain level in CI >>>>>>>>>>>>>>>>>> Actually, being able to configure domain level per class >>>>>>>>>>>>>>>>>> is WAY >>>>>>>>>>>>>>>>>> more >>>>>>>>>>>>>>>>>> convenient, than to always have to think which domain >>>>>>>>>>>>>>>>>> level is >>>>>>>>>>>>>>>>>> appropriate for which particular test during jenkins job >>>>>>>>>>>>>>>>>> configuration. In fact, I should have thought about it >>>>>>>>>>>>>>>>>> from the >>>>>>>>>>>>>>>>>> very >>>>>>>>>>>>>>>>>> beginning. For example, in test_replica_promotion.py we >>>>>>>>>>>>>>>>>> have on >>>>>>>>>>>>>>>>>> class, >>>>>>>>>>>>>>>>>> which intiates with domain level = 1, while others - with >>>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>>>> 0. With config-based approach, we would have to >>>>>>>>>>>>>>>>>> implement a >>>>>>>>>>>>>>>>>> separate >>>>>>>>>>>>>>>>>> step that raises domain level. Overall, I am against the >>>>>>>>>>>>>>>>>> approach, >>>>>>>>>>>>>>>>>> when you have to remember to set certain domain level in >>>>>>>>>>>>>>>>>> config >>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>> any particular test. The tests themselves should be >>>>>>>>>>>>>>>>>> aware of >>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>> domain level they need. >>>>>>>>>>>>>>>>> I do not say that we should not have something that >>>>>>>>>>>>>>>>> overrides >>>>>>>>>>>>>>>>> settings >>>>>>>>>>>>>>>>> in from config in a particular test case, I say your >>>>>>>>>>>>>>>>> patch is >>>>>>>>>>>>>>>>> doing it >>>>>>>>>>>>>>>>> wrong. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I agree it is useful to have param domain_level in >>>>>>>>>>>>>>>>> install_master, >>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by >>>>>>>>>>>>>>>>> default, >>>>>>>>>>>>>>>>> because with your current patch the domain_level in >>>>>>>>>>>>>>>>> config is >>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>> used >>>>>>>>>>>>>>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> For example I want to achieve this goal: >>>>>>>>>>>>>>>>> test_vault.py, this test suite can run on domain level1 >>>>>>>>>>>>>>>>> and on >>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>> level0, so with one test we can test 2 domain levels just >>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>> putting >>>>>>>>>>>>>>>>> domain level into config file. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I agree that with extraordinary test like replica >>>>>>>>>>>>>>>>> promotion test >>>>>>>>>>>>>>>>> is, we >>>>>>>>>>>>>>>>> need something that allows override the config file. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> As I said bellow, domain_level default value should be >>>>>>>>>>>>>>>>> None in >>>>>>>>>>>>>>>>> install_master and install_topo plugin. If domain level >>>>>>>>>>>>>>>>> was >>>>>>>>>>>>>>>>> specified >>>>>>>>>>>>>>>>> use the specified one, if not (value is None) use the >>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>>> from >>>>>>>>>>>>>>>>> config file. >>>>>>>>>>>>>>>> Agreed :) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Martin >>>>>>>>>>>>>>>>>>> [PATCH] Enabled setting domain_level per class >>>>>>>>>>>>>>>>>>> derived from >>>>>>>>>>>>>>>>>>> TestIntegration >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> When I configure domain level 0 in yaml config, how >>>>>>>>>>>>>>>>>>> is this >>>>>>>>>>>>>>>>>>> supposed to >>>>>>>>>>>>>>>>>>> get into install methods when you removed that code? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>>>>>>>>>>>>>> + "--domain-level=%i" % domain_level >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> You always use MAX_DOMAIN_LEVEL in this case or >>>>>>>>>>>>>>>>>>> whatever is >>>>>>>>>>>>>>>>>>> specified in >>>>>>>>>>>>>>>>>>> domain_level option. >>>>>>>>>>>>>>>>>>> I suggest to use domain_level=None, and when it is >>>>>>>>>>>>>>>>>>> None use >>>>>>>>>>>>>>>>>>> 'host.config.domain_level', if it is not None, use >>>>>>>>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> With this we can specify domain level in config file for >>>>>>>>>>>>>>>>>>> test >>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>> be used for both domain levels and you can manually >>>>>>>>>>>>>>>>>>> specify >>>>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>>>>> for test that requires specific domain level. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Also this should go away >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> @classmethod >>>>>>>>>>>>>>>>>>> def install(cls, mh): >>>>>>>>>>>>>>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>>>>>>>>>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>>>>>>>>>>>>>> if cls.topology is None: >>>>>>>>>>>>>>>>>>> return >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I do not see reason why test should override >>>>>>>>>>>>>>>>>>> configuration in >>>>>>>>>>>>>>>>>>> config in >>>>>>>>>>>>>>>>>>> this case. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Martin >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Here is the updated version of the patch (more tests + >>>>>>>>>>>>>>>>>>>> fixed the >>>>>>>>>>>>>>>>>>>> issues of the first one) + patch 0017, that >>>>>>>>>>>>>>>>>>>> implements the >>>>>>>>>>>>>>>>>>>> necessary >>>>>>>>>>>>>>>>>>>> changes in the background code, i. e. patch 16 does not >>>>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>>>> without >>>>>>>>>>>>>>>>>>>> patch 17 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>>>>>>>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>>>>>>>>>>>>>> Hi guys, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Here are first two automated testcases from this >>>>>>>>>>>>>>>>>>>>>> (so far >>>>>>>>>>>>>>>>>>>>>> incomplete) >>>>>>>>>>>>>>>>>>>>>> testplan: >>>>>>>>>>>>>>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> PATCH 16: NACK >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>>>>>>>>>>>>>> 'domain_level' to >>>>>>>>>>>>>>>>>>>>> install_topo()? >>>>>>>>>>>>>>>>>>>>> Also it is good practise to add new option as the last >>>>>>>>>>>>>>>>>>>>> parameter. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>>>>>>>> cab you in both tests specify a domain level with >>>>>>>>>>>>>>>>>>>>> constant >>>>>>>>>>>>>>>>>>>>> instead of >>>>>>>>>>>>>>>>>>>>> number literal? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>>>>>>>> both test call install_topo with custom domain level, >>>>>>>>>>>>>>>>>>>>> but it >>>>>>>>>>>>>>>>>>>>> cannot >>>>>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>>>>> because 1) (did you run the test?) >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>>>>>>>>>>>>>> Respectively why there is call of install_topo() that >>>>>>>>>>>>>>>>>>>>> installs >>>>>>>>>>>>>>>>>>>>> replica. >>>>>>>>>>>>>>>>>>>>> As this test just tests that ipa-replica-prepare is >>>>>>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>> working >>>>>>>>>>>>>>>>>>>>> anymore, >>>>>>>>>>>>>>>>>>>>> is it worth to spend 20 minutes with installing >>>>>>>>>>>>>>>>>>>>> replica and >>>>>>>>>>>>>>>>>>>>> then >>>>>>>>>>>>>>>>>>>>> just no >>>>>>>>>>>>>>>>>>>>> tot use it? IMO to install master in install step is >>>>>>>>>>>>>>>>>>>>> enough. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Martin^2 >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ./make-lint >>>>>>>>>>>>>>> ************* Module ipatests.test_integration.base >>>>>>>>>>>>>>> ipatests/test_integration/base.py:66: [E1101(no-member), >>>>>>>>>>>>>>> IntegrationTest.install] Class 'IntegrationTest' has no >>>>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>>>> member) >>>>>>>>>>>>>>> ************* Module >>>>>>>>>>>>>>> ipatests.test_integration.test_replica_promotion >>>>>>>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:16: >>>>>>>>>>>>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no >>>>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>>>> member) >>>>>>>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:117: >>>>>>>>>>>>>>> [E1101(no-member), >>>>>>>>>>>>>>> TestCAInstall.test_ca_install_without_replica_file] >>>>>>>>>>>>>>> Module 'ipatests.test_integration.tasks' has no >>>>>>>>>>>>>>> 'setup_replica' >>>>>>>>>>>>>>> member) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Is it so hard to run pylint before patch is posted for >>>>>>>>>>>>>>> review? >>>>>>>>>>>>>> Sorry, my bad. Fixed. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> 1) >>>>>>>>>>>> Why is this change in the patch? >>>>>>>>>>>> - # Clean up the test directory >>>>>>>>>>>> - host.run_command(['rm', '-rvf', host.config.test_dir]) >>>>>>>>>>> Otherwise 2 out of 8 tests fail with IOError at line 78 of >>>>>>>>>>> ipatests/test_integration/tasks.py >>>>>>>>>>> >>>>>>>>>>> I do not understand yet how does this happen, but if you remove >>>>>>>>>>> ipatests folder once, it then fails to be created again. >>>>>>>>>>> >>>>>>>>>> So this should be in separated patch and investigated properly. >>>>>>>>> Agree. Removed >>>>>>>>>>>> - >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 2) >>>>>>>>>>>> is enough to have this check only in install_master, >>>>>>>>>>>> install_topo can >>>>>>>>>>>> pass None to install_master >>>>>>>>>>>> + if domain_level is None: >>>>>>>>>>>> + domain_level = master.config.domain_level >>>>>>>>>>> Done >>>>>>>>>>> >>>>>>>>>>>> 3) >>>>>>>>>>>> IMO replica-manage del should cleanup hosts entry, so following >>>>>>>>>>>> code >>>>>>>>>>>> should not be needed. >>>>>>>>>>>> + if cleanhost: >>>>>>>>>>>> + kinit_admin(master) >>>>>>>>>>>> + master.run_command(["ipa", "host-del", "--updatedns", >>>>>>>>>>>> replica.hostname], >>>>>>>>>>>> + raiseonerr=False) >>>>>>>>>>> Well, in fact it does not. At least the corresponding dns record >>>>>>>>>>> stays >>>>>>>>>>> and causes the subsequent ipa-client-install to fail. Probably >>>>>>>>>>> it's a >>>>>>>>>>> bug. On the other hand, if I promote an existing client to >>>>>>>>>>> replica and >>>>>>>>>>> then delete this replica, then, I probably want the host record >>>>>>>>>>> (that >>>>>>>>>>> was created during client-install) to stay in the system. So, >>>>>>>>>>> does not >>>>>>>>>>> look like a bug to me. >>>>>>>>>> No you don't, because replica uninstallation also removes the >>>>>>>>>> client. >>>>>>>>>> >>>>>>>>>> I tried it with ipa42, ipa-replica-manage del removes host entry, >>>>>>>>>> and >>>>>>>>>> DNS A records, only SSHFP records stay there (I'm not sure if it >>>>>>>>>> is bug >>>>>>>>>> or feature) >>>>>>>>>> >>>>>>>>>> Also I received this message >>>>>>>>>> """ >>>>>>>>>> Failed to cleanup replica1.ipa.test DNS entries: no matching >>>>>>>>>> entry >>>>>>>>>> found >>>>>>>>>> You may need to manually remove them from the tree >>>>>>>>>> """ >>>>>>>>>> But, A record has been removed, so this is probably false >>>>>>>>>> positive and >>>>>>>>>> it needs to have a ticket. >>>>>>>>> Agree, that was an issue with my setup. >>>>>>>>> Removed >>>>>>>>> >>>>>>>>>>>> 4) >>>>>>>>>>>> This variable is not used >>>>>>>>>>>> + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] >>>>>>>>>>> Removed >>>>>>>>>>> >>>>>>>>>>>> 5) >>>>>>>>>>>> Why do you need this >>>>>>>>>>>> + kra_install = ["ipa-kra-install", "-U", "-p", >>>>>>>>>>>> config.dirman_password] >>>>>>>>>>>> when you implemented tasks.install_kra that returns the exactly >>>>>>>>>>>> the same >>>>>>>>>>>> result? >>>>>>>>>>> Right. Removed >>>>>>>>>>> >>>>>>>>>>>> 6) >>>>>>>>>>>> PEP8 >>>>>>>>>>>> ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 >>>>>>>>>>>> blank >>>>>>>>>>>> lines, >>>>>>>>>>>> found 1 >>>>>>>>>>>> ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 >>>>>>>>>>>> blank >>>>>>>>>>>> lines, >>>>>>>>>>>> found 1 >>>>>>>>>>>> ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 >>>>>>>>>>>> blank >>>>>>>>>>>> lines, >>>>>>>>>>>> found 1 >>>>>>>>>>>> ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 >>>>>>>>>>>> blank >>>>>>>>>>>> lines, >>>>>>>>>>>> found 1 >>>>>>>>>>>> ./ipatests/test_integration/tasks.py:950:80: E501 line too long >>>>>>>>>>>> (80 > 79 >>>>>>>>>>>> characters) >>>>>>>>>>>> >>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:29:80: >>>>>>>>>>>> E501 >>>>>>>>>>>> line >>>>>>>>>>>> too long (85 > 79 characters) >>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:64:80: >>>>>>>>>>>> E501 >>>>>>>>>>>> line >>>>>>>>>>>> too long (85 > 79 characters) >>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:67:80: >>>>>>>>>>>> E501 >>>>>>>>>>>> line >>>>>>>>>>>> too long (88 > 79 characters) >>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:93:80: >>>>>>>>>>>> E501 >>>>>>>>>>>> line >>>>>>>>>>>> too long (80 > 79 characters) >>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:94:80: >>>>>>>>>>>> E501 >>>>>>>>>>>> line >>>>>>>>>>>> too long (83 > 79 characters) >>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 >>>>>>>>>>>> >>>>>>>>>>>> line >>>>>>>>>>>> too long (81 > 79 characters) >>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 >>>>>>>>>>>> >>>>>>>>>>>> line >>>>>>>>>>>> too long (80 > 79 characters) >>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 >>>>>>>>>>>> >>>>>>>>>>>> line >>>>>>>>>>>> too long (82 > 79 characters) >>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 >>>>>>>>>>>> >>>>>>>>>>>> line >>>>>>>>>>>> too long (80 > 79 characters) >>>>>>>>>>> Most of these complaints are unrelated to the current patches. >>>>>>>>>>> It's better to create a separate patch addressing PEP8 errors. >>>>>>>>>>> >>>>>>>>>> I beg for your pardon, those PEP8 errors have been introduced by >>>>>>>>>> your >>>>>>>>>> patches. >>>>>>>>> Fixed >>>>>>>>> >>>>>>>>>>>> 7) >>>>>>>>>>>> Why this must be stored in instance? IMO to have it stored as >>>>>>>>>>>> local >>>>>>>>>>>> variable is perfect >>>>>>>>>>>> TestKRAInstall, TestCAInstall >>>>>>>>>>>> self.replica1_filename = >>>>>>>>>>>> tasks.get_replica_filename(replica1) >>>>>>>>>>>> self.replica2_filename = >>>>>>>>>>>> tasks.get_replica_filename(replica2) >>>>>>>>>>> Agree. Fixed >>>>>>>>>>> >>>>>>> This patch is missing something. >>>>>> I am sorry, I forgot to revert my previous change. The correct >>>>>> patch is >>>>>> attached >>>>>> >>>>> ************* Module ipatests.test_integration.test_replica_promotion >>>>> ipatests/test_integration/test_replica_promotion.py:15: >>>>> [E1123(unexpected-keyword-arg), Dummy.install] Unexpected keyword >>>>> argument 'domain_level' in function call) >>>>> ipatests/test_integration/test_replica_promotion.py:15: >>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no 'domain_level' >>>>> member) >>>>> ipatests/test_integration/test_replica_promotion.py:19: >>>>> [E1101(no-member), Dummy.teardown_method] Module >>>>> 'ipatests.test_integration.tasks' has no 'uninstall_replica' member) >>>>> ipatests/test_integration/test_replica_promotion.py:67: >>>>> [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] >>>>> Module 'ipatests.test_integration.tasks' has no 'ipa_backup' member) >>>>> ipatests/test_integration/test_replica_promotion.py:72: >>>>> [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] >>>>> Module 'ipatests.test_integration.tasks' has no 'ipa_restore' member) >>>>> ipatests/test_integration/test_replica_promotion.py:120: >>>>> [E1123(unexpected-keyword-arg), TestCAInstall.install] Unexpected >>>>> keyword argument 'domain_level' in function call) >>>>> >>>> Sorry I forgot to apply patch 17, my bad, I'm continuing with review >> LGTM, I haven't had time to test it, but if you are sure that test is >> working, we may push this. >> > Is this expected due the bug you mentioned? > _____ > __________________________________________________________________________ > TestReplicaPromotionLevel0.test_kra_install_master > ________________________________________________________________________________ > > > self = > object at 0x7f5071a59e50> > > def test_kra_install_master(self): > result1 = tasks.install_kra(self.master, raiseonerr=False) >> assert result1.returncode == 0, result1.stderr_text > E AssertionError: Usage: ipa-kra-install [options] [replica_file] > E > E ipa-kra-install: error: Replica file > /root/ipatests/replica-info.gpg does not exist > E The ipa-kra-install command failed. See > /var/log/ipaserver-kra-install.log for more information > E > E assert 2 == 0 > E + where 2 = 0x7f5071adbd50>.returncode > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0016.8-First-part-of-replica-promotion-tests.patch Type: text/x-patch Size: 10262 bytes Desc: not available URL: From ofayans at redhat.com Thu Dec 17 09:41:22 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Thu, 17 Dec 2015 10:41:22 +0100 Subject: [Freeipa-devel] [patch 0026] ipatests: replace the test-example.com domain in tests In-Reply-To: <5671AA01.80408@redhat.com> References: <5671AA01.80408@redhat.com> Message-ID: <56728342.3060501@redhat.com> ACK On 12/16/2015 07:14 PM, Milan Kub?k wrote: > Applies to ipa-4-3 and master. Reason in commit message. > > > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From jcholast at redhat.com Thu Dec 17 09:46:15 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 17 Dec 2015 10:46:15 +0100 Subject: [Freeipa-devel] [PATCH] 0749 Package ipapython, ipalib, ipaplatform, ipatests for Python 3 In-Reply-To: <567155A9.7090101@redhat.com> References: <563C9810.1050604@redhat.com> <5656E487.8050703@redhat.com> <5658508D.1020105@redhat.com> <565C0FF3.8040406@redhat.com> <565D80B0.9000500@redhat.com> <565DA287.6030403@redhat.com> <565EE63E.6070207@redhat.com> <5665A44E.3060203@redhat.com> <567155A9.7090101@redhat.com> Message-ID: <56728467.5030109@redhat.com> On 16.12.2015 13:14, Petr Viktorin wrote: > On 12/07/2015 04:22 PM, Jan Cholasta wrote: >> On 2.12.2015 13:38, Petr Viktorin wrote: >>> On 12/01/2015 02:37 PM, Jan Cholasta wrote: > [...] >>>>>>> >>>>>>> As for new provides, Fedora's Python packaging guidelines say: >>>>>>> >>>>>>> """ >>>>>>> Using a fictional module named "example", the subpackage containing >>>>>>> the python2 version must provide python2-example. This is of course >>>>>>> always the case if the subpackage is named python2-example [...] >>>>>>> If the subpackage has some other name then then Provides: >>>>>>> python2-example >>>>>>> must be added explicitly (but see the %python_provide macro below). >>>>>>> >>>>>>> The python3 subpackage must provide python3-example. However, as the >>>>>>> naming guidelines mandate that the python3 subpackage be named >>>>>>> python3-example, this will happen automatically. >>>>>>> """ >>>>>>> >>>>>>> so I'm now adding Provides for the top-level modules. >>>>>> >>>>>> The goal of this work is to add support for Python 3, not to comply >>>>>> with >>>>>> Fedora packaging guidelines. FreeIPA on Fedora uses its own spec file >>>>>> anyway. >>>>> >>>>> The goal of this patch is to add new packages that support Python 3. >>>>> Yes, the Fedora spec is different, but it's heavily based on the >>>>> upstream one, and this is a good thing. I consider the Fedora >>>>> guidelines >>>>> the standard in Python RPM packaging. If IPA uses different packaging >>>>> guidelines, can you point me to them? >>>> >>>> FreeIPA never fully complied to Fedora packaging guidelines AFAIK and I >>>> don't see any reason to start now, since nobody seemed to care so far. >>>> Following them in just py3 sub-packages does not improve the state of >>>> FreeIPA as a whole and only brings inconsistency into it, so there's no >>>> benefit in doing it at all. >>>> >>>>>> Again, if you insist on doing this, do it for Python 2 as well. >>> >>> OK, when your patches are ACKed I'll send patches to both improve py2 >>> packaging and add the new packages. > > Here is the patch rebased onto current master, which includes your > packaging improvements. 1) python3-ipatests is missing requires on freeipa-client-common. Also the python3-ipalib Requires should use %_isa, so that only the package for the right architecture is pulled on install. 2) Please keep the layout of the python3-* %packages as close as possible to python2-*. 3) Nitpick: please keep 2 empty lines between subpackage definitions. Otherwise ACK. I fixed all the issues, see attachment. If you are OK with the changes, I will push the patch. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0749.4-Package-ipapython-ipalib-ipaplatform-ipatests-for-Py.patch Type: text/x-patch Size: 18446 bytes Desc: not available URL: From pviktori at redhat.com Thu Dec 17 09:50:57 2015 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 17 Dec 2015 10:50:57 +0100 Subject: [Freeipa-devel] [PATCH] 0749 Package ipapython, ipalib, ipaplatform, ipatests for Python 3 In-Reply-To: <56728467.5030109@redhat.com> References: <563C9810.1050604@redhat.com> <5656E487.8050703@redhat.com> <5658508D.1020105@redhat.com> <565C0FF3.8040406@redhat.com> <565D80B0.9000500@redhat.com> <565DA287.6030403@redhat.com> <565EE63E.6070207@redhat.com> <5665A44E.3060203@redhat.com> <567155A9.7090101@redhat.com> <56728467.5030109@redhat.com> Message-ID: <56728581.2090104@redhat.com> On 12/17/2015 10:46 AM, Jan Cholasta wrote: > On 16.12.2015 13:14, Petr Viktorin wrote: >> On 12/07/2015 04:22 PM, Jan Cholasta wrote: >>> On 2.12.2015 13:38, Petr Viktorin wrote: >>>> On 12/01/2015 02:37 PM, Jan Cholasta wrote: >> [...] >>>>>>>> >>>>>>>> As for new provides, Fedora's Python packaging guidelines say: >>>>>>>> >>>>>>>> """ >>>>>>>> Using a fictional module named "example", the subpackage containing >>>>>>>> the python2 version must provide python2-example. This is of course >>>>>>>> always the case if the subpackage is named python2-example [...] >>>>>>>> If the subpackage has some other name then then Provides: >>>>>>>> python2-example >>>>>>>> must be added explicitly (but see the %python_provide macro below). >>>>>>>> >>>>>>>> The python3 subpackage must provide python3-example. However, as >>>>>>>> the >>>>>>>> naming guidelines mandate that the python3 subpackage be named >>>>>>>> python3-example, this will happen automatically. >>>>>>>> """ >>>>>>>> >>>>>>>> so I'm now adding Provides for the top-level modules. >>>>>>> >>>>>>> The goal of this work is to add support for Python 3, not to comply >>>>>>> with >>>>>>> Fedora packaging guidelines. FreeIPA on Fedora uses its own spec >>>>>>> file >>>>>>> anyway. >>>>>> >>>>>> The goal of this patch is to add new packages that support Python 3. >>>>>> Yes, the Fedora spec is different, but it's heavily based on the >>>>>> upstream one, and this is a good thing. I consider the Fedora >>>>>> guidelines >>>>>> the standard in Python RPM packaging. If IPA uses different packaging >>>>>> guidelines, can you point me to them? >>>>> >>>>> FreeIPA never fully complied to Fedora packaging guidelines AFAIK >>>>> and I >>>>> don't see any reason to start now, since nobody seemed to care so far. >>>>> Following them in just py3 sub-packages does not improve the state of >>>>> FreeIPA as a whole and only brings inconsistency into it, so >>>>> there's no >>>>> benefit in doing it at all. >>>>> >>>>>>> Again, if you insist on doing this, do it for Python 2 as well. >>>> >>>> OK, when your patches are ACKed I'll send patches to both improve py2 >>>> packaging and add the new packages. >> >> Here is the patch rebased onto current master, which includes your >> packaging improvements. > > 1) python3-ipatests is missing requires on freeipa-client-common. Also > the python3-ipalib Requires should use %_isa, so that only the package > for the right architecture is pulled on install. > > 2) Please keep the layout of the python3-* %packages as close as > possible to python2-*. > > 3) Nitpick: please keep 2 empty lines between subpackage definitions. > > Otherwise ACK. I fixed all the issues, see attachment. If you are OK > with the changes, I will push the patch. Thank you, ACK from me. -- Petr Viktorin From jcholast at redhat.com Thu Dec 17 09:53:21 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 17 Dec 2015 10:53:21 +0100 Subject: [Freeipa-devel] [PATCH] 0749 Package ipapython, ipalib, ipaplatform, ipatests for Python 3 In-Reply-To: <56728581.2090104@redhat.com> References: <563C9810.1050604@redhat.com> <5656E487.8050703@redhat.com> <5658508D.1020105@redhat.com> <565C0FF3.8040406@redhat.com> <565D80B0.9000500@redhat.com> <565DA287.6030403@redhat.com> <565EE63E.6070207@redhat.com> <5665A44E.3060203@redhat.com> <567155A9.7090101@redhat.com> <56728467.5030109@redhat.com> <56728581.2090104@redhat.com> Message-ID: <56728611.4090600@redhat.com> On 17.12.2015 10:50, Petr Viktorin wrote: > On 12/17/2015 10:46 AM, Jan Cholasta wrote: >> On 16.12.2015 13:14, Petr Viktorin wrote: >>> On 12/07/2015 04:22 PM, Jan Cholasta wrote: >>>> On 2.12.2015 13:38, Petr Viktorin wrote: >>>>> On 12/01/2015 02:37 PM, Jan Cholasta wrote: >>> [...] >>>>>>>>> >>>>>>>>> As for new provides, Fedora's Python packaging guidelines say: >>>>>>>>> >>>>>>>>> """ >>>>>>>>> Using a fictional module named "example", the subpackage containing >>>>>>>>> the python2 version must provide python2-example. This is of course >>>>>>>>> always the case if the subpackage is named python2-example [...] >>>>>>>>> If the subpackage has some other name then then Provides: >>>>>>>>> python2-example >>>>>>>>> must be added explicitly (but see the %python_provide macro below). >>>>>>>>> >>>>>>>>> The python3 subpackage must provide python3-example. However, as >>>>>>>>> the >>>>>>>>> naming guidelines mandate that the python3 subpackage be named >>>>>>>>> python3-example, this will happen automatically. >>>>>>>>> """ >>>>>>>>> >>>>>>>>> so I'm now adding Provides for the top-level modules. >>>>>>>> >>>>>>>> The goal of this work is to add support for Python 3, not to comply >>>>>>>> with >>>>>>>> Fedora packaging guidelines. FreeIPA on Fedora uses its own spec >>>>>>>> file >>>>>>>> anyway. >>>>>>> >>>>>>> The goal of this patch is to add new packages that support Python 3. >>>>>>> Yes, the Fedora spec is different, but it's heavily based on the >>>>>>> upstream one, and this is a good thing. I consider the Fedora >>>>>>> guidelines >>>>>>> the standard in Python RPM packaging. If IPA uses different packaging >>>>>>> guidelines, can you point me to them? >>>>>> >>>>>> FreeIPA never fully complied to Fedora packaging guidelines AFAIK >>>>>> and I >>>>>> don't see any reason to start now, since nobody seemed to care so far. >>>>>> Following them in just py3 sub-packages does not improve the state of >>>>>> FreeIPA as a whole and only brings inconsistency into it, so >>>>>> there's no >>>>>> benefit in doing it at all. >>>>>> >>>>>>>> Again, if you insist on doing this, do it for Python 2 as well. >>>>> >>>>> OK, when your patches are ACKed I'll send patches to both improve py2 >>>>> packaging and add the new packages. >>> >>> Here is the patch rebased onto current master, which includes your >>> packaging improvements. >> >> 1) python3-ipatests is missing requires on freeipa-client-common. Also >> the python3-ipalib Requires should use %_isa, so that only the package >> for the right architecture is pulled on install. >> >> 2) Please keep the layout of the python3-* %packages as close as >> possible to python2-*. >> >> 3) Nitpick: please keep 2 empty lines between subpackage definitions. >> >> Otherwise ACK. I fixed all the issues, see attachment. If you are OK >> with the changes, I will push the patch. > > Thank you, ACK from me. Pushed to: master: 53b60546e43caecdfb0d9e1b91cd5f309d08cf52 ipa-4-3: 8dbae7d4edb5edccdd118e0779bdeed2c40344e0 -- Jan Cholasta From ndehadra at redhat.com Thu Dec 17 10:33:28 2015 From: ndehadra at redhat.com (Nikhil Dehadrai) Date: Thu, 17 Dec 2015 16:03:28 +0530 Subject: [Freeipa-devel] Fwd: [PUBLIC] Re: [IPAQE][REVIEW-REQUEST][TEST PLAN] Replica promotion In-Reply-To: <56717814.1050604@redhat.com> References: <56558E00.8030006@redhat.com> <566FDF19.4060605@redhat.com> <56713F1E.7060303@redhat.com> <56717266.1080800@redhat.com> <56717814.1050604@redhat.com> Message-ID: <56728F78.4080304@redhat.com> Hi Martin, Ok. Understood. But how about the observations, what do you think of that? As far as new test scenarios is concerned, will wait for an update from other members too who can help clarify the behavior under such scenarios. Thanks and Best Regards, Nikhil Dehadrai On 12/16/2015 08:11 PM, Martin Babinsky wrote: > On 12/16/2015 03:17 PM, Nikhil Dehadrai wrote: >> Hi There, >> >> Based on the URL for REPLICA PROMOTION Test plan >> (http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan), I have >> following observations / queries. >> >> Observations: >> ------------------- >> 1. For "ipa-kra-install" and "ipa-ca-install" , expected result for step >> 1 should be FAILED, But it is marked as SUCCEED. (What is the correct >> behavior here). >> >> Questions: >> -------------- >> 1. As per current design (IPA 4.2) we can prepare replica server file >> on an existing replica server using "ipa-replica-prepare" and use this >> file to setup new replica host, how can we achieve this behavior under >> this new feature when MASTER domain level is set to 1.? > > You can't. The whole point of replica installation by promotion is to > eliminate the need to lug around files to set up new replicas. > > In domain level 1 the ipa-replica-prepare command is disabled and > running it raises an error (See test case 2 in > http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan). Also > there is no MASTER domain level, the domain level is uniform across > the whole topology > >> 2. How will the "ipa topologysegment-add" work for a 3 repilca scenario? >> Since, under domain level 1 it uses "rightnode" and "leftnode" to form a >> new segment. Can I add 3rd replica to this already created "Newsegment" >> or we need to have it created separately? How will it work? >> > Topology segments describe replication agreements between *two* > endpoints (think about them as graph edges connecting two nodes). So > you set up a segment between replica1 and replica2. To connect third > replica to the topology, you have to create new segment connecting > either replica1 and replica3, or replica2 and replica3. > >> We can add New Test Scenarios as well: >> ----------------------------------------------------- >> 1. Add scenario for domain level details at UI. Since it talks of >> reflecting the domain level changes at UI. >> 2. Add scenario to validate once domain level of master is raised to 1, >> after that it cannot be lowered. >> 3. Add scenario for "ipa-restore" using master and replica topology , so >> as to identify the effect of restoration on replica by using "ipa >> topologysegment-find realm" command.( What will be the expected behavior >> in this case) >> > I'm not quite sure about point 3 either. I guess that you have to > reenroll the replica after restore, but maybe someone more > knowledgeable about the feature will chime in. >> >> Let me know. >> >> Thanks and Best Regards, >> Nikhil Dehadrai >> >> > > From pvoborni at redhat.com Thu Dec 17 11:14:40 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 17 Dec 2015 12:14:40 +0100 Subject: [Freeipa-devel] [PATCH 0075-0076] Fix installer regression In-Reply-To: <567167C7.8060605@redhat.com> References: <567167C7.8060605@redhat.com> Message-ID: <56729920.3040500@redhat.com> On 12/16/2015 02:31 PM, David Kupka wrote: > https://www.redhat.com/archives/freeipa-users/2015-December/msg00203.html > > please link the patch to https://fedorahosted.org/freeipa/ticket/5556 -- Petr Vobornik From ofayans at redhat.com Thu Dec 17 11:30:23 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Thu, 17 Dec 2015 12:30:23 +0100 Subject: [Freeipa-devel] Fwd: [PUBLIC] Re: [IPAQE][REVIEW-REQUEST][TEST PLAN] Replica promotion In-Reply-To: <56728F78.4080304@redhat.com> References: <56558E00.8030006@redhat.com> <566FDF19.4060605@redhat.com> <56713F1E.7060303@redhat.com> <56717266.1080800@redhat.com> <56717814.1050604@redhat.com> <56728F78.4080304@redhat.com> Message-ID: <56729CCF.6020204@redhat.com> Hi Nikhil, Your observations are correct. Testplan was updated. On 12/17/2015 11:33 AM, Nikhil Dehadrai wrote: > Hi Martin, > > Ok. Understood. > > But how about the observations, what do you think of that? > > As far as new test scenarios is concerned, will wait for an update from > other members too who can help clarify the behavior under such scenarios. > > Thanks and Best Regards, > Nikhil Dehadrai > > On 12/16/2015 08:11 PM, Martin Babinsky wrote: >> On 12/16/2015 03:17 PM, Nikhil Dehadrai wrote: >>> Hi There, >>> >>> Based on the URL for REPLICA PROMOTION Test plan >>> (http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan), I have >>> following observations / queries. >>> >>> Observations: >>> ------------------- >>> 1. For "ipa-kra-install" and "ipa-ca-install" , expected result for step >>> 1 should be FAILED, But it is marked as SUCCEED. (What is the correct >>> behavior here). >>> >>> Questions: >>> -------------- >>> 1. As per current design (IPA 4.2) we can prepare replica server file >>> on an existing replica server using "ipa-replica-prepare" and use this >>> file to setup new replica host, how can we achieve this behavior under >>> this new feature when MASTER domain level is set to 1.? >> >> You can't. The whole point of replica installation by promotion is to >> eliminate the need to lug around files to set up new replicas. >> >> In domain level 1 the ipa-replica-prepare command is disabled and >> running it raises an error (See test case 2 in >> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan). Also >> there is no MASTER domain level, the domain level is uniform across >> the whole topology >> >>> 2. How will the "ipa topologysegment-add" work for a 3 repilca scenario? >>> Since, under domain level 1 it uses "rightnode" and "leftnode" to form a >>> new segment. Can I add 3rd replica to this already created "Newsegment" >>> or we need to have it created separately? How will it work? >>> >> Topology segments describe replication agreements between *two* >> endpoints (think about them as graph edges connecting two nodes). So >> you set up a segment between replica1 and replica2. To connect third >> replica to the topology, you have to create new segment connecting >> either replica1 and replica3, or replica2 and replica3. >> >>> We can add New Test Scenarios as well: >>> ----------------------------------------------------- >>> 1. Add scenario for domain level details at UI. Since it talks of >>> reflecting the domain level changes at UI. >>> 2. Add scenario to validate once domain level of master is raised to 1, >>> after that it cannot be lowered. >>> 3. Add scenario for "ipa-restore" using master and replica topology , so >>> as to identify the effect of restoration on replica by using "ipa >>> topologysegment-find realm" command.( What will be the expected behavior >>> in this case) >>> >> I'm not quite sure about point 3 either. I guess that you have to >> reenroll the replica after restore, but maybe someone more >> knowledgeable about the feature will chime in. >>> >>> Let me know. >>> >>> Thanks and Best Regards, >>> Nikhil Dehadrai >>> >>> >> >> > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From dkupka at redhat.com Thu Dec 17 12:26:28 2015 From: dkupka at redhat.com (David Kupka) Date: Thu, 17 Dec 2015 13:26:28 +0100 Subject: [Freeipa-devel] [PATCH 0075-0076] Fix installer regression In-Reply-To: <56729920.3040500@redhat.com> References: <567167C7.8060605@redhat.com> <56729920.3040500@redhat.com> Message-ID: <5672A9F4.40205@redhat.com> On 17/12/15 12:14, Petr Vobornik wrote: > On 12/16/2015 02:31 PM, David Kupka wrote: >> https://www.redhat.com/archives/freeipa-users/2015-December/msg00203.html >> >> > please link the patch to https://fedorahosted.org/freeipa/ticket/5556 Updated patches attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0075.1-ipa42-installer-Propagate-option-values-from-components-in.patch Type: text/x-patch Size: 3159 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0075.1-master+ipa43-installer-Propagate-option-values-from-components-in.patch Type: text/x-patch Size: 3333 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0076.1-ipa42-installer-Fix-logic-of-reading-option-values-from-ca.patch Type: text/x-patch Size: 1844 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0076.1-master+ipa43-installer-Fix-logic-of-reading-option-values-from-ca.patch Type: text/x-patch Size: 1840 bytes Desc: not available URL: From jcholast at redhat.com Thu Dec 17 12:44:30 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 17 Dec 2015 13:44:30 +0100 Subject: [Freeipa-devel] [PATCH 0075-0076] Fix installer regression In-Reply-To: <5672A9F4.40205@redhat.com> References: <567167C7.8060605@redhat.com> <56729920.3040500@redhat.com> <5672A9F4.40205@redhat.com> Message-ID: <5672AE2E.2020701@redhat.com> On 17.12.2015 13:26, David Kupka wrote: > On 17/12/15 12:14, Petr Vobornik wrote: >> On 12/16/2015 02:31 PM, David Kupka wrote: >>> https://www.redhat.com/archives/freeipa-users/2015-December/msg00203.html >>> >>> >>> >> please link the patch to https://fedorahosted.org/freeipa/ticket/5556 > > Updated patches attached. NACK, this is the correct procedure for the __getattr__(), sorry for confusing you earlier: def __getattr__(self, name): for owner_cls, knob_name in self.knobs(): if knob_name == name: break else: raise AttributeError(name) for component in self.__components: if isinstance(component, owner_cls): break else: raise AttributeError(name) return getattr(component, name) Honza -- Jan Cholasta From ofayans at redhat.com Thu Dec 17 12:53:34 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Thu, 17 Dec 2015 13:53:34 +0100 Subject: [Freeipa-devel] [PATCH-0019][Tests]Enabled --auto-reverse by default for master installation Message-ID: <5672B04E.9040602@redhat.com> -- Oleg Fayans Quality Engineer FreeIPA team RedHat. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0019-Enabled-automatic-creation-of-reverse-zone.patch Type: text/x-patch Size: 974 bytes Desc: not available URL: From lslebodn at redhat.com Thu Dec 17 13:05:12 2015 From: lslebodn at redhat.com (Lukas Slebodnik) Date: Thu, 17 Dec 2015 14:05:12 +0100 Subject: [Freeipa-devel] [PATCH-0019][Tests]Enabled --auto-reverse by default for master installation In-Reply-To: <5672B04E.9040602@redhat.com> References: <5672B04E.9040602@redhat.com> Message-ID: <20151217130511.GA16380@mail.corp.redhat.com> On (17/12/15 13:53), Oleg Fayans wrote: > >-- >Oleg Fayans >Quality Engineer >FreeIPA team >RedHat. >From ed4630140386c1043e36733eb42ec402cc276bee Mon Sep 17 00:00:00 2001 >From: Oleg Fayans >Date: Thu, 17 Dec 2015 13:50:19 +0100 >Subject: [PATCH] Enabled automatic creation of reverse zone during master > installation > The commit message does not contain ticket. It is also not explained why this change was done. Could you update commit message with verbose explanation? Proper commit message might prevent removal of this change in future based on "git blame" LS From mbasti at redhat.com Thu Dec 17 13:18:41 2015 From: mbasti at redhat.com (Martin Basti) Date: Thu, 17 Dec 2015 14:18:41 +0100 Subject: [Freeipa-devel] [PATCH 0379 - 0393] Winter code cleanup In-Reply-To: <56725C34.5060400@redhat.com> References: <56720BCC.1050504@redhat.com> <56725C34.5060400@redhat.com> Message-ID: <5672B631.7030709@redhat.com> On 17.12.2015 07:54, Jan Cholasta wrote: > Hi, > > On 17.12.2015 02:11, Martin Basti wrote: >> Hello, >> following patches cleanup the code and add checks to pylint to avoid the >> mess again >> >> Patches: 379-381: >> the most important patches, they cleanup imports >> >> Patch 382: >> enables various pylint checks, we may reduce the list of check if >> needed. > > Would it be possible to turn this into a blacklist of disabled > warnings rather than a whitelist of enabled warnings? > >> >> Patches 383 - 393: >> remove minor issues from code, and enable particular checks >> Feel free to nack patches/checks that should not be there. >> >> Please apply patches in order. >> >> Martin^2 >> >> > > Honza > Updated patches attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0393.2-Enable-pylint-unnecessary-pass-check.patch Type: text/x-patch Size: 11023 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0392.2-Enable-pylint-eval-and-exec-usage-checks.patch Type: text/x-patch Size: 2213 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0391.2-Enable-pylint-unnecessary-lambda-check.patch Type: text/x-patch Size: 5925 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0390.2-Enable-pylint-empty-docstring-check.patch Type: text/x-patch Size: 2596 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0389.2-Enable-pylint-pointless-statement-check.patch Type: text/x-patch Size: 5265 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0388.2-Enable-pylint-expression-not-assigned-check.patch Type: text/x-patch Size: 8082 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0387.2-Enable-pylint-unused-format-string-key-check.patch Type: text/x-patch Size: 2774 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0386.2-Enable-pylint-missing-final-newline-check.patch Type: text/x-patch Size: 1637 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0385.2-Enable-pylint-trailing-whitespace-check.patch Type: text/x-patch Size: 3825 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0384.2-Enable-pylint-duplicated-key-check.patch Type: text/x-patch Size: 1918 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0383.2-Enable-pylint-lost-exception-check.patch Type: text/x-patch Size: 1665 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0382.2-Enable-multiple-warnings-checks-in-Pylint.patch Type: text/x-patch Size: 4827 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0381.2-Remove-wildcard-imports.patch Type: text/x-patch Size: 45573 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0380.2-Remove-unused-imports.patch Type: text/x-patch Size: 104888 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0379.2-Remove-empty-test-file.patch Type: text/x-patch Size: 1537 bytes Desc: not available URL: From ofayans at redhat.com Thu Dec 17 13:27:24 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Thu, 17 Dec 2015 14:27:24 +0100 Subject: [Freeipa-devel] [PATCH-0019][Tests]Enabled --auto-reverse by default for master installation In-Reply-To: <20151217130511.GA16380@mail.corp.redhat.com> References: <5672B04E.9040602@redhat.com> <20151217130511.GA16380@mail.corp.redhat.com> Message-ID: <5672B83C.3030700@redhat.com> Commit message was updated. Thanks! On 12/17/2015 02:05 PM, Lukas Slebodnik wrote: > On (17/12/15 13:53), Oleg Fayans wrote: >> >> -- >> Oleg Fayans >> Quality Engineer >> FreeIPA team >> RedHat. > >>From ed4630140386c1043e36733eb42ec402cc276bee Mon Sep 17 00:00:00 2001 >> From: Oleg Fayans >> Date: Thu, 17 Dec 2015 13:50:19 +0100 >> Subject: [PATCH] Enabled automatic creation of reverse zone during master >> installation >> > The commit message does not contain ticket. > It is also not explained why this change was done. > Could you update commit message with verbose explanation? > > Proper commit message might prevent removal of this change in future > based on "git blame" > > LS > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0019.1-Enabled-automatic-creation-of-reverse-zone.patch Type: text/x-patch Size: 1195 bytes Desc: not available URL: From mkubik at redhat.com Thu Dec 17 13:35:14 2015 From: mkubik at redhat.com (=?UTF-8?Q?Milan_Kub=c3=adk?=) Date: Thu, 17 Dec 2015 14:35:14 +0100 Subject: [Freeipa-devel] [patch 0027] ipatests: Roll back the forwarder config after a test case Message-ID: <5672BA12.4070608@redhat.com> Patch attached. -- Milan Kubik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkubik-0027-ipatests-Roll-back-the-forwarder-config-after-a-test.patch Type: text/x-patch Size: 1108 bytes Desc: not available URL: From mbasti at redhat.com Thu Dec 17 14:15:51 2015 From: mbasti at redhat.com (Martin Basti) Date: Thu, 17 Dec 2015 15:15:51 +0100 Subject: [Freeipa-devel] [patch 0026] ipatests: replace the test-example.com domain in tests In-Reply-To: <56728342.3060501@redhat.com> References: <5671AA01.80408@redhat.com> <56728342.3060501@redhat.com> Message-ID: <5672C397.8070704@redhat.com> On 17.12.2015 10:41, Oleg Fayans wrote: > ACK > > On 12/16/2015 07:14 PM, Milan Kub?k wrote: >> Applies to ipa-4-3 and master. Reason in commit message. >> >> >> Pushed to: master: aa648bcedcdb06162f1e578731ecf7fba299b709 ipa-4-3: b402a8dc0fe0ff567a61c7ddde26f66e4f470e24 From ofayans at redhat.com Thu Dec 17 15:28:21 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Thu, 17 Dec 2015 16:28:21 +0100 Subject: [Freeipa-devel] [PATCH 0020][Tests] Added --realm parameter to master installation procedure in integration tests Message-ID: <5672D495.1010302@redhat.com> -- Oleg Fayans Quality Engineer FreeIPA team RedHat. -------------- next part -------------- A non-text attachment was scrubbed... Name: .freeipa-ofayans-0020-Added-domain-realm-as-a-parameter-to-master-installation.patch.swp Type: application/octet-stream Size: 12288 bytes Desc: not available URL: From mbabinsk at redhat.com Thu Dec 17 15:44:51 2015 From: mbabinsk at redhat.com (Martin Babinsky) Date: Thu, 17 Dec 2015 16:44:51 +0100 Subject: [Freeipa-devel] [PATCH 0020][Tests] Added --realm parameter to master installation procedure in integration tests In-Reply-To: <5672D495.1010302@redhat.com> References: <5672D495.1010302@redhat.com> Message-ID: <5672D873.3030706@redhat.com> On 12/17/2015 04:28 PM, Oleg Fayans wrote: > > > Oleg, you sent a vim swap file instead of the patch. -- Martin^3 Babinsky From pvoborni at redhat.com Thu Dec 17 15:53:22 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 17 Dec 2015 16:53:22 +0100 Subject: [Freeipa-devel] Drafting FreeIPA 4.3 release notes Message-ID: <5672DA72.6010802@redhat.com> Hello all, draft: http://www.freeipa.org/page/Releases/4.3.0 I've added big sections about changes in replica installation, topology management and DNS zone checks(thanks Petr for release notes in ticket #3681). Comments welcome! Note: list of bugfixes and minor enhancements needs to be added. -- Petr Vobornik From pspacek at redhat.com Thu Dec 17 19:16:07 2015 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 17 Dec 2015 20:16:07 +0100 Subject: [Freeipa-devel] [patch 0027] ipatests: Roll back the forwarder config after a test case In-Reply-To: <5672BA12.4070608@redhat.com> References: <5672BA12.4070608@redhat.com> Message-ID: <567309F7.3060507@redhat.com> On 17.12.2015 14:35, Milan Kub?k wrote: > Patch attached. > > -- > Milan Kubik > > > freeipa-mkubik-0027-ipatests-Roll-back-the-forwarder-config-after-a-test.patch > > > From 66db35cf2a315f0a58e002b06f0fcc5bd915a5c5 Mon Sep 17 00:00:00 2001 > From: Milan Kubik > Date: Thu, 17 Dec 2015 14:17:22 +0100 > Subject: [PATCH] ipatests: Roll back the forwarder config after a test case > > --- > ipatests/test_xmlrpc/test_dns_plugin.py | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/ipatests/test_xmlrpc/test_dns_plugin.py b/ipatests/test_xmlrpc/test_dns_plugin.py > index 5f692528eb9a5ae0dc488c663ab43cc47ffd29b3..cba6b001053e6a5c708e9d4359119ad071d70280 100644 > --- a/ipatests/test_xmlrpc/test_dns_plugin.py > +++ b/ipatests/test_xmlrpc/test_dns_plugin.py > @@ -1768,6 +1768,15 @@ class test_dns(Declarative): > }, > ), > > + dict( > + desc='Update global DNS settings - rollback', > + command=('dnsconfig_mod', [], {'idnsforwarders' : None,}), > + expected={ > + 'value': None, > + 'summary': u'Global DNS configuration is empty', > + 'result': {}, > + }, > + ), > > dict( > desc='Try to add invalid allow-query to zone %r' % zone1, I have seen this code to work on David's VM. ACK. -- Petr^2 Spacek From pspacek at redhat.com Thu Dec 17 19:19:51 2015 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 17 Dec 2015 20:19:51 +0100 Subject: [Freeipa-devel] [PATCH-0019][Tests]Enabled --auto-reverse by default for master installation In-Reply-To: <5672B83C.3030700@redhat.com> References: <5672B04E.9040602@redhat.com> <20151217130511.GA16380@mail.corp.redhat.com> <5672B83C.3030700@redhat.com> Message-ID: <56730AD7.4090106@redhat.com> On 17.12.2015 14:27, Oleg Fayans wrote: > Commit message was updated. Thanks! > > On 12/17/2015 02:05 PM, Lukas Slebodnik wrote: >> On (17/12/15 13:53), Oleg Fayans wrote: >>> >>> -- >>> Oleg Fayans >>> Quality Engineer >>> FreeIPA team >>> RedHat. >> >> >From ed4630140386c1043e36733eb42ec402cc276bee Mon Sep 17 00:00:00 2001 >>> From: Oleg Fayans >>> Date: Thu, 17 Dec 2015 13:50:19 +0100 >>> Subject: [PATCH] Enabled automatic creation of reverse zone during master >>> installation >>> >> The commit message does not contain ticket. >> It is also not explained why this change was done. >> Could you update commit message with verbose explanation? >> >> Proper commit message might prevent removal of this change in future >> based on "git blame" Seems reasonable in general but this should not happen in BRQ or ABCDE labs. Where do you see the problem? Otherwise ACK. -- Petr^2 Spacek From ofayans at redhat.com Fri Dec 18 08:35:25 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Fri, 18 Dec 2015 09:35:25 +0100 Subject: [Freeipa-devel] [PATCH-0019][Tests]Enabled --auto-reverse by default for master installation In-Reply-To: <56730AD7.4090106@redhat.com> References: <5672B04E.9040602@redhat.com> <20151217130511.GA16380@mail.corp.redhat.com> <5672B83C.3030700@redhat.com> <56730AD7.4090106@redhat.com> Message-ID: <5673C54D.50805@redhat.com> Hi Petr, On 12/17/2015 08:19 PM, Petr Spacek wrote: > On 17.12.2015 14:27, Oleg Fayans wrote: >> Commit message was updated. Thanks! >> >> On 12/17/2015 02:05 PM, Lukas Slebodnik wrote: >>> On (17/12/15 13:53), Oleg Fayans wrote: >>>> >>>> -- >>>> Oleg Fayans >>>> Quality Engineer >>>> FreeIPA team >>>> RedHat. >>> >>> >From ed4630140386c1043e36733eb42ec402cc276bee Mon Sep 17 00:00:00 2001 >>>> From: Oleg Fayans >>>> Date: Thu, 17 Dec 2015 13:50:19 +0100 >>>> Subject: [PATCH] Enabled automatic creation of reverse zone during master >>>> installation >>>> >>> The commit message does not contain ticket. >>> It is also not explained why this change was done. >>> Could you update commit message with verbose explanation? >>> >>> Proper commit message might prevent removal of this change in future >>> based on "git blame" > > Seems reasonable in general but this should not happen in BRQ or ABCDE labs. > Where do you see the problem? > > Otherwise ACK. The problem occurs in local libvirt-powered VMs. Also it may (or may not - I did not try it yet) be met in Beaker. -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From ofayans at redhat.com Fri Dec 18 08:37:17 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Fri, 18 Dec 2015 09:37:17 +0100 Subject: [Freeipa-devel] [PATCH 0020][Tests] Added --realm parameter to master installation procedure in integration tests In-Reply-To: <5672D873.3030706@redhat.com> References: <5672D495.1010302@redhat.com> <5672D873.3030706@redhat.com> Message-ID: <5673C5BD.7090203@redhat.com> Sorry, my bad :) On 12/17/2015 04:44 PM, Martin Babinsky wrote: > On 12/17/2015 04:28 PM, Oleg Fayans wrote: >> >> >> > Oleg, > > you sent a vim swap file instead of the patch. > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0020-Added-domain-realm-as-a-parameter-to-master-installation.patch Type: text/x-patch Size: 1218 bytes Desc: not available URL: From dkupka at redhat.com Fri Dec 18 10:26:30 2015 From: dkupka at redhat.com (David Kupka) Date: Fri, 18 Dec 2015 11:26:30 +0100 Subject: [Freeipa-devel] [PATCH 0077] ipa-dns-install: Do not check for zone overlap when DNS, installed. Message-ID: <5673DF56.8060906@redhat.com> Standalone DNS installer always performed overlap check effectively preventing installation on replica when other DNS instance was already installed in topology. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0077.0-ipa-dns-install-Do-not-check-for-zone-overlap-when-D.patch Type: text/x-patch Size: 4707 bytes Desc: not available URL: From pvoborni at redhat.com Fri Dec 18 11:04:18 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 18 Dec 2015 12:04:18 +0100 Subject: [Freeipa-devel] [PATCH 0077] ipa-dns-install: Do not check for zone overlap when DNS, installed. In-Reply-To: <5673DF56.8060906@redhat.com> References: <5673DF56.8060906@redhat.com> Message-ID: <5673E832.3020609@redhat.com> On 12/18/2015 11:26 AM, David Kupka wrote: > Standalone DNS installer always performed overlap check effectively > preventing installation on replica when other DNS instance was already > installed in topology. > > I don't like the position of api argument in the install_check. It is not consistent with install_check in KRA plus it's between two related options: standalone and replica. Is there a reason behind it which I don't see? -- Petr Vobornik From slaznick at redhat.com Fri Dec 18 11:46:01 2015 From: slaznick at redhat.com (Stanislav Laznicka) Date: Fri, 18 Dec 2015 12:46:01 +0100 Subject: [Freeipa-devel] [PATCH 0011-0012][RFE] ipa-replica-manage: automatically clean dangling RUVs Message-ID: <5673F1F9.2030403@redhat.com> Hi, Attached are the patches for auto-find and clean of dangling (cs)ruvs. Currently, the cleaning of an RUV waits for all replicas to be online, even on --force. If that were an issue, I can make the command fail before trying to clean any of RUVs. However, the user is shown a replica is offline and is prompted to confirm the cleaning so the possible wait should not be a problem I believe. Standa L. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-stlaz-0011-Listing-and-cleaning-RUV-extended-for-CA-suffix.patch Type: text/x-patch Size: 4787 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-stlaz-0012-Automatically-detect-and-remove-dangling-RUVs.patch Type: text/x-patch Size: 8084 bytes Desc: not available URL: From dkupka at redhat.com Fri Dec 18 12:17:27 2015 From: dkupka at redhat.com (David Kupka) Date: Fri, 18 Dec 2015 13:17:27 +0100 Subject: [Freeipa-devel] [PATCH 0077] ipa-dns-install: Do not check for zone overlap when DNS, installed. In-Reply-To: <5673E832.3020609@redhat.com> References: <5673DF56.8060906@redhat.com> <5673E832.3020609@redhat.com> Message-ID: <5673F957.40009@redhat.com> On 18/12/15 12:04, Petr Vobornik wrote: > On 12/18/2015 11:26 AM, David Kupka wrote: >> Standalone DNS installer always performed overlap check effectively >> preventing installation on replica when other DNS instance was already >> installed in topology. >> >> > > I don't like the position of api argument in the install_check. It is > not consistent with install_check in KRA plus it's between two related > options: standalone and replica. > > Is there a reason behind it which I don't see? Right now (with my patch applied) I see: ca.install_check(standalone, replica_config, options) kra.install_check(api, replica_config, options) dns.install_check(standalone, api, replica, options, hostname) And if I needed to get common signature of the functions I would probably end up with: install_check(standalone, api, replica_config, options, ...) In KRA there is no use for option standalone so it is left out. In DNS replica_config is not needed, only information we are running on replica has some value. So it's replaced by boolean. But if you think this is important please tell me where api option should go and I'll change it... -- David Kupka From dkupka at redhat.com Fri Dec 18 12:57:56 2015 From: dkupka at redhat.com (David Kupka) Date: Fri, 18 Dec 2015 13:57:56 +0100 Subject: [Freeipa-devel] [PATCH 0075-0076] Fix installer regression In-Reply-To: <5672AE2E.2020701@redhat.com> References: <567167C7.8060605@redhat.com> <56729920.3040500@redhat.com> <5672A9F4.40205@redhat.com> <5672AE2E.2020701@redhat.com> Message-ID: <567402D4.8010605@redhat.com> On 17/12/15 13:44, Jan Cholasta wrote: > On 17.12.2015 13:26, David Kupka wrote: >> On 17/12/15 12:14, Petr Vobornik wrote: >>> On 12/16/2015 02:31 PM, David Kupka wrote: >>>> https://www.redhat.com/archives/freeipa-users/2015-December/msg00203.html >>>> >>>> >>>> >>>> >>> please link the patch to https://fedorahosted.org/freeipa/ticket/5556 >> >> Updated patches attached. > > NACK, this is the correct procedure for the __getattr__(), sorry for > confusing you earlier: > > def __getattr__(self, name): > for owner_cls, knob_name in self.knobs(): > if knob_name == name: > break > else: > raise AttributeError(name) > > for component in self.__components: > if isinstance(component, owner_cls): > break > else: > raise AttributeError(name) > > return getattr(component, name) > > Honza > Updated patches attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0075.2-ipa42-installer-Propagate-option-values-from-components-in.patch Type: text/x-patch Size: 3927 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0075.2-master+ipa43-installer-Propagate-option-values-from-components-in.patch Type: text/x-patch Size: 4100 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0076.2-ipa42-installer-Fix-logic-of-reading-option-values-from-ca.patch Type: text/x-patch Size: 1844 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0076.2-master+ipa43-installer-Fix-logic-of-reading-option-values-from-ca.patch Type: text/x-patch Size: 1840 bytes Desc: not available URL: From dkupka at redhat.com Fri Dec 18 13:11:05 2015 From: dkupka at redhat.com (David Kupka) Date: Fri, 18 Dec 2015 14:11:05 +0100 Subject: [Freeipa-devel] [PATCH 0075-0076] Fix installer regression In-Reply-To: <567402D4.8010605@redhat.com> References: <567167C7.8060605@redhat.com> <56729920.3040500@redhat.com> <5672A9F4.40205@redhat.com> <5672AE2E.2020701@redhat.com> <567402D4.8010605@redhat.com> Message-ID: <567405E9.2070909@redhat.com> On 18/12/15 13:57, David Kupka wrote: > On 17/12/15 13:44, Jan Cholasta wrote: >> On 17.12.2015 13:26, David Kupka wrote: >>> On 17/12/15 12:14, Petr Vobornik wrote: >>>> On 12/16/2015 02:31 PM, David Kupka wrote: >>>>> https://www.redhat.com/archives/freeipa-users/2015-December/msg00203.html >>>>> >>>>> >>>>> >>>>> >>>>> >>>> please link the patch to https://fedorahosted.org/freeipa/ticket/5556 >>> >>> Updated patches attached. >> >> NACK, this is the correct procedure for the __getattr__(), sorry for >> confusing you earlier: >> >> def __getattr__(self, name): >> for owner_cls, knob_name in self.knobs(): >> if knob_name == name: >> break >> else: >> raise AttributeError(name) >> >> for component in self.__components: >> if isinstance(component, owner_cls): >> break >> else: >> raise AttributeError(name) >> >> return getattr(component, name) >> >> Honza >> > Updated patches attached. > Completing the update. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0075.3-ipa42-installer-Propagate-option-values-from-components-in.patch Type: text/x-patch Size: 4065 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0075.3-master+ipa43-installer-Propagate-option-values-from-components-in.patch Type: text/x-patch Size: 4238 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0076.2-ipa42-installer-Fix-logic-of-reading-option-values-from-ca.patch Type: text/x-patch Size: 1844 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0076.2-master+ipa43-installer-Fix-logic-of-reading-option-values-from-ca.patch Type: text/x-patch Size: 1840 bytes Desc: not available URL: From jcholast at redhat.com Fri Dec 18 14:13:04 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 18 Dec 2015 15:13:04 +0100 Subject: [Freeipa-devel] [PATCH 0075-0076] Fix installer regression In-Reply-To: <567405E9.2070909@redhat.com> References: <567167C7.8060605@redhat.com> <56729920.3040500@redhat.com> <5672A9F4.40205@redhat.com> <5672AE2E.2020701@redhat.com> <567402D4.8010605@redhat.com> <567405E9.2070909@redhat.com> Message-ID: <56741470.9040609@redhat.com> On 18.12.2015 14:11, David Kupka wrote: > On 18/12/15 13:57, David Kupka wrote: >> On 17/12/15 13:44, Jan Cholasta wrote: >>> On 17.12.2015 13:26, David Kupka wrote: >>>> On 17/12/15 12:14, Petr Vobornik wrote: >>>>> On 12/16/2015 02:31 PM, David Kupka wrote: >>>>>> https://www.redhat.com/archives/freeipa-users/2015-December/msg00203.html >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> please link the patch to https://fedorahosted.org/freeipa/ticket/5556 >>>> >>>> Updated patches attached. >>> >>> NACK, this is the correct procedure for the __getattr__(), sorry for >>> confusing you earlier: >>> >>> def __getattr__(self, name): >>> for owner_cls, knob_name in self.knobs(): >>> if knob_name == name: >>> break >>> else: >>> raise AttributeError(name) >>> >>> for component in self.__components: >>> if isinstance(component, owner_cls): >>> break >>> else: >>> raise AttributeError(name) >>> >>> return getattr(component, name) >>> >>> Honza >>> >> Updated patches attached. >> > Completing the update. Thanks, ACK. Not pushed yet because ipa-4-3 is cursed. Honza -- Jan Cholasta From pvoborni at redhat.com Fri Dec 18 14:39:36 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 18 Dec 2015 15:39:36 +0100 Subject: [Freeipa-devel] [PATCH 0077] ipa-dns-install: Do not check for zone overlap when DNS, installed. In-Reply-To: <5673E832.3020609@redhat.com> References: <5673DF56.8060906@redhat.com> <5673E832.3020609@redhat.com> Message-ID: <56741AA8.5060807@redhat.com> On 12/18/2015 12:04 PM, Petr Vobornik wrote: > On 12/18/2015 11:26 AM, David Kupka wrote: >> Standalone DNS installer always performed overlap check effectively >> preventing installation on replica when other DNS instance was already >> installed in topology. >> >> > > I don't like the position of api argument in the install_check. It is > not consistent with install_check in KRA plus it's between two related > options: standalone and replica. > > Is there a reason behind it which I don't see? NACK, the API call is incorrect. It should be api.Command['dns_is_enabled']()['result'] or api.Command.dns_is_enabled()['result'] ipa-dns-install --forwarder=$DNS_FORWARDER The log file for this installation can be found in /var/log/ipaserver-install.log Unexpected error - see /var/log/ipaserver-install.log for details: TypeError: __call__() takes exactly 1 argument (2 given) log file: 2015-12-18T14:33:30Z DEBUG File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 736, in run_script return_value = main_function() File "/sbin/ipa-dns-install", line 137, in main dns_installer.install_check(True, api, False, options, hostname=api.env.host) File "/usr/lib/python2.7/site-packages/ipaserver/install/dns.py", line 113, in install_check already_enabled = api.Command('dns_is_enabled')['result'] 2015-12-18T14:33:30Z DEBUG The ipa-dns-install command failed, exception: TypeError: __call__() takes exactly 1 argument (2 given) -- Petr Vobornik From pvoborni at redhat.com Fri Dec 18 17:24:14 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 18 Dec 2015 18:24:14 +0100 Subject: [Freeipa-devel] Announcing FreeIPA 4.3.0 Message-ID: <5674413E.4000206@redhat.com> The FreeIPA team would like to announce FreeIPA v4.3.0 release! It can be downloaded from http://www.freeipa.org/page/Downloads. The builds are available for Fedora rawhide. Builds for Fedora 23 are available in the official COPR repository . This announcement is also available at . == Highlights in 4.3.0 == * Simplified management of replication topology - control and display your topology from CLI and UI * Simplified replica installation - install replica without ''replica package'' via OTP, keytab or privileged user credentials. The new method is called ''replica promotion'' as it adds FreeIPA server capability to existing or new client === Domain Level === Both feature sets are tight with introduction of new "server capability indicator" - a "domain level". Domain level indicates that server is capable of doing certain operations. Domain level 1 means that it supports replica promotion and topology management. Old servers and servers upgraded to 4.3 in existing environments have domain level 0. In order to use new functionality all servers needs to be updated to a version which supports the domain level, right now it is only version 4.3. Domain level is raised by command: $ ipa domainlevel-set 1 Current domain can be obtained by: $ ipa domainlevel-get Or supported levels of individual FreeIPA servers: $ ipa server-show $HOSTNAME === Replica installation === ==== Old method - domain level 0 ==== Prior FreeIPA 4.3 replica installation needed to perform actions on both master and future replica. First step on master: $ ipa-replica-prepare $REPLICA_HOSTNAME --ip-address $REPLICA_IP It created a replica file - an encrypted file containing secrets and other data needed for replica installation. Second step on replica: $ ipa-replica-install --various-options $REPLICA_FILE Disadvantage is that both 'ipa-replica-prepared' and 'ipa-replica-install' need directory manager password and that copying of the replica file is cumbersome. Old method is still available for environments with domain level 0. ==== New method - domain level 1 ==== New method transforms an IPA client into an IPA server. I.e., an IPA client can be installed first and then it can be "promoted" into an FreeIPA server - a new replica. Alternatively, replica installer can also install the client so it can be done in a single operation. New method doesn't require to run 'ipa-replica-prepare' and manipulate with replica file. There are multiple ways to install new replica: ===== 1. Promotion of existing client ===== On client which will become new FreeIPA server: $ kinit admin $ ipa-replica-install [--various-options, ...] ===== 2. Installation of replica on non-FreeIPA client machine ===== $ ipa-replica-install --principal admin -W [--various-options, ...] It will ask for admin password, install a client and then promote it to replica. It will use DNS auto-discovery to locate the master server. Alternatively the same discovery options as for 'ipa-client-install' can be provided: '--server', '--domain', '--realm'. ===== 3. Installation of replica using one time password(OTP) ===== On any host with 'ipa' command line utility available first prepare the host entry with One Time Password set and assign it to 'ipaservers' hostgroup to mark it as future IPA server. $ kinit admin $ ipa host-add $REPLICA_HOSTNAME --password $OTP $ ipa hostgroup-add-member ipaservers --hosts=$REPLICA_HOSTNAME On future replica: $ ipa-replica-install --password $OTP [--various-options, ...] ===== 4. Installation of replica using a host keytab ===== Steps are similar as in installation with OTP: On arbitrary FreeIPA client or server: $ kinit admin $ ipa host-add $REPLICA_HOSTNAME $ ipa hostgroup-add-member ipaservers --hosts=$REPLICA_HOSTNAME $ ipa-getkeytab --server=$IPASERVER_HOSTNAME --principal=host/$REPLICA_HOSTNAME@$REALM --keytab=replica_host.keytab $ # copy the replica_host.keytab to a replica on $REPLICA_KEYTAB_PATH (arbitrary) On future replica: $ ipa-replica-install --keytab $REPLICA_KEYTAB_PATH [--various-options, ...] === Managed Replication Topology === FreeIPA is a multi-master technology. Data changes on a server are replicated automatically to all other servers. Data is stored in Directory Server server in two so-called suffixes: a 'domain' suffix, e.g., 'dc=example,dc=com' which contains all domain related data(users, groups, hbac and sudo rules, ...) and, if the setup has CA, a 'ca' suffix('o=ipaca') which contains Certificate Server data. IPA servers, in general, are not connected with all other servers, but usually with only a few. It means the data is gradually propagated. The way is defined in Directory Server by so-called replication agreements. Replication agreements for each suffix need to be managed separately. Recommended maximum number of agreements on one server is 4 for each suffix. It is required to manage the topology of replication agreements correctly so a failure of one server would not disconnect the entire topology. FreeIPA 4.2 and older manages the agreements using 'ipa-replica-manage' and 'ipa-csreplica-manage' tools. The disadvantage of the tools are: * No single single server has data about the whole topology. * The tools needs to be run on an IPA server -> not possible in CLI or Web UI. * The lack of information prevents of proper disconnection checks, e.g., when a replica or a connection is removed. FreeIPA 4.3 introduces a managed topology. The topology is maintained as data and is replicated to all other servers. It is represented by two new IPA object types: topology suffixes and topology segments. Topology suffix represents a Directory Server suffix mentioned above. Topology segment represents replication agreements between 2 servers. See `ipa help topology` for more information about CLI commands. IPA servers changes their replication agreements automatically according to this configuration. It brings following benefits: * 'ipa' command line interface and Web UI(located under "IPA Server/Topology" menu item) can be used to manage the topology from any place * Modification of the topology performs a check to prevent disconnection (a server or a group of servers would not be connected with rest of the topology). * Uninstallation of replica using 'ipa-replica-manage del' and 'ipa-server-install --uninstall' tools checks if the uninstallation would disconnect the topology and refuses to do so. * Existing topology can be checked for errors using a new 'ipa topologysuffix-verify' command. * Web UI comes with new topology graph which visualizes the topology and allows interactive changes of the topology. * It will allow to monitor state of replication in a future. On domain level 1, managing of IPA replication agreements using 'ipa-replica-manage' and 'ipa-csreplica-manage' tools is no longer possible. But the tools can be still used for managing of winsync agreements, DNA ranges, RUVs and for reinitializing and force-synchronizing of replicas. Long term goal is to completely replace 'ipa-csreplica-manage' and leave 'ipa-replica-manage' only for managing of winsync agreements. === DNS zone creation checks === FreeIPA now checks if specified DNS domains exist prior installing the integrated DNS server and refuses to use DNS domain names which are already served by other DNS servers. This prevents problems caused by situation where multiple DNS servers wrongy act as authoritative servers for single DNS domain. This has multiple consequences: * To avoid conflicts, the unattended installation creates reverse zones only if option '--auto-reverse' is used. * Reverse DNS zones which already exist on some other DNS servers are not automatically created to avoid conflicts (even during interactive installation). * When reverse zones are not managed by FreeIPA DNS, the automatic empty zones (as specified in RFC 6303 ) are automatically created by BIND. In situations where these reverse zones are used and managed by other DNS servers, FreeIPA DNS servers should forward queries for these zones. In that case users must manually create 'forward zone' using `ipa dnsforwarzone-add` command to override automatic empty zone supplied by BIND. This change affects only new installations. === Known Issues === * Running 'ipa-dns-install' when some other IPA server has DNS installed will fail. Use '--force' option to workaround the issue. * FreeIPA 4.3 requires an update of SELinux policy, see bug 1289930 . To workaround the issue, disable SELinux - 'setenforce 0' - on master when installing a replica or a Certificate Server. * Re-installation of replica with CA or re-installation of KRA will fail without pki-core-10.2.6-13 , see bug #1704 === Bug fixes === * Contains all bugfixes and enhancements of 4.2.2 and 4.2.3 releases. * Automatic configuration for Firefox < 10 was dropped. #5144 * '--configure-firefox' is documented in 'ipa-client-install' man page. #5375 === Enhancements === * 'ipa-getkeytab' no longer requires to specify server when run on FreeIPA server. #2203 * Custom configuration for dse.ldif can be provided on replica installation. #4048 #4949 * Added support for Ed25519 SSH keys (RFC 7479). #5471 == Upgrading == Upgrade instructions are available on Upgrade page == Feedback == Please provide comments, bugs and other feedback via the freeipa-users mailing list (http://www.redhat.com/mailman/listinfo/freeipa-users) or #freeipa channel on Freenode. == Detailed Changelog since 4.2.1 == === Abhijeet Kasurde (4) === * Added try/except block for user_input in ipautil * Updated number of legacy permission in ipatests * Added user friendly error message for dnszone enable and disable * Fixed small typo in stage-user documentation === Alexander Bokovoy (7) === * selinux: enable httpd_run_ipa to allow communicating with oddjobd services * oddjob: avoid chown keytab to sssd if sssd user does not exist * Fix selector of protocol for LSA RPC binding string * trusts: harden trust-fetch-domains oddjobd-based script * trusts: format Kerberos principal properly when fetching trust topology * client referral support for trusted domain principals * spec file: depend on Dogtag 10.2.6-12 for tomcat 8 upgrade === Benjamin Drung (3) === * Fix hyphen-used-as-minus-sign warning (found by lintian) * Fix manpage-has-errors-from-man warning (found by Lintian) * default.conf.5: Fix a typo === Christian Heimes (18) === * Start dirsrv for kdcproxy upgrade * Remove tuple unpacking from except clause contrib/RHEL4/ipachangeconf.py * Remove tuple unpacking from except clause ipa-client/ipaclient/ipachangeconf.py * Remove tuple unpacking from except clause ipalib/plugins/hbactest.py * Remove tuple unpacking from except clause ipaserver/dcerpc.py * Replace file() with open() * Fix selinux denial during kdcproxy user creation * certprofile-import: improve profile format documentation * otptoken: use ipapython.nsslib instead of Python's ssl module * Require Dogtag PKI >= 10.2.6 * Replace M2Crypto RC4 with python-cryptography ARC4 * Validate vault's file parameters * certprofile-import: do not require profileId in profile data * Asymmetric vault: validate public key in client * Add flag to list all service and user vaults * Change internal rsa_(public|private)_key variable names * Handle timeout error in ipa-httpd-kdcproxy * mod_auth_gssapi: Remove ntlmssp support and restrict mechanism to krb5 === David Kupka (22) === * migration: Use api.env variables. * cermonger: Use private unix socket when DBus SystemBus is not available. * ipa-client-install: Do not (re)start certmonger and DBus daemons. * dbus: Create empty dbus.Array with specified signature * user-undel: Fix error messages. * client: Add support for multiple IP addresses during installation. * client: Add description of --ip-address and --all-ip-addresses to man page * Backup/resore authentication control configuration * vault: Limit size of data stored in vault * ipactl: Do not start/stop/restart single service multiple times * comment: Add Documentation string to deduplicate function * admintool: Add error message with path to log on failure. * ipa-cacert-renew: Fix connection to ldap. * ipa-otptoken-import: Fix connection to ldap. * ipa-replica-install support caless install with promotion. * install: Run all validators at once. * replica: Fix ipa-replica-install with replica file (domain level 0). * test: Temporarily increase timeout in vault test. * spec file: Add dbus-python to BuildRequires * dns: do not add (forward)zone if it is already resolvable. * dns: Check if domain already exists. * dns: Add --auto-reverse option. === Endi Sukma Dewata (6) === * Fixed missing KRA agent cert on replica. * Added CLI param and ACL for vault service operations. * Fixed vault container ownership. * Added support for changing vault encryption. * Removed clear text passwords from KRA install log. * Using LDAPI to setup CA and KRA agents. === Fran?ois Cami (1) === * ipa-client-install: Fix the "download the CA cert" query === Fraser Tweedale (19) === * user-show: add --out option to save certificates to file * Fix otptoken-remove-managedby command summary * Give more info on virtual command access denial * Allow SAN extension for cert-request self-service * Add profile for DNP3 / IEC 62351-8 certificates * Work around python-nss bug on unrecognised OIDs * Fix default CA ACL added during upgrade * Fix KRB5PrincipalName / UPN SAN comparison * certprofile: add profile format explanation * Add permission for bypassing CA ACL enforcement * Prohibit deletion of predefined profiles * cert-request: remove allowed extensions check * certprofile: prevent rename (modrdn) * certprofile: remove 'rename' option * TLS and Dogtag HTTPS request logging improvements * Avoid race condition caused by profile delete and recreate * Do not erroneously reinit NSS in Dogtag interface * Add profiles and default CA ACL on migration * dogtaginstance: remove unused function 'check_inst' === Gabe Alford (16) === * Fix client ca.crt to match the server's cert * Add Chromium configuration note to ssbrowser * Standardize minvalue for ipasearchrecordlimit and ipasesarchsizelimit for unlimited minvalue * dnssec option missing in ipa-dns-install man page * Update FreeIPA package description * Remove bind configuration detected question * Warn if no installation found when running ipa-server-install --uninstall * Add Firefox options to ipa-client-install man page * interactive installer does not ignore leading/trailing whitespace * Remove 50-lockout-policy.update file * Incomplete ports for IPA AD Trust * custodia: ipa-upgrade failed on replica * ipa-replica-manage del continues when host does not exist in domain level 1 * Check if IPA is configured before attempting a winsync migration * ipa-replica-install prints incorrect error message when replica is already installed * Migrate wget references and usage to curl === Jan Cholasta (70) === * spec file: Move /etc/ipa/kdcproxy to the server subpackage * spec file: Update minimum required version of krb5 * install: Fix server and replica install options * ULC: Prevent preserved users from being assigned membership * baseldap: Allow overriding member param label in LDAPModMember * vault: Fix param labels in output of vault owner commands * install: Fix replica install with custom certificates * vault: Fix vault-find with criteria * vault: Add container information to vault command results * spec file: Add Requires(post) on selinux-policy * cert renewal: Include KRA users in Dogtag LDAP update * cert renewal: Automatically update KRA agent PEM file * install: Fix SASL mappings not added in ipa-server-install * ldap: Make ldap2 connection management thread-safe again * Use six.with_metaclass to specify metaclasses * Use six.python_2_unicode_compatible * Decode script arguments using file system encoding * config: allow user/host attributes with tagging options * Alias "unicode" to "str" under Python 3 * Use bytes instead of str where appropriate * Use byte literals where appropriate * baseldap: make subtree deletion optional in LDAPDelete * vault: set owner to current user on container creation * vault: update access control * vault: add permissions and administrator privilege * install: support KRA update * install: Support overriding knobs in subclasses * install: Add common base class for server and replica install * install: Move unattended option to the general help section * install: create kdcproxy user during server install * platform: add option to create home directory when adding user * install: fix kdcproxy user home directory * install: fix invocation of KRAInstance.create_instance() * install: fix ipa-server-install fail on missing --forwarder * install: fix KRA agent PEM file permissions * install: always export KRA agent PEM file * vault: select a server with KRA for vault operations * schema: do not derive ipaVaultPublicKey from ipaPublicKey * upgrade: make sure ldap2 is connected in export_kra_agent_pem * vault: fix private service vault creation * install: fix command line option validation * install: export KRA agent PEM file in ipa-kra-install * cert renewal: make renewal of ipaCert atomic * client install: do not corrupt OpenSSH config with Match sections * install: drop support for Dogtag 9 * server: use topologysuffix name in iparepltopomanagedsuffix * topology: replace "suffices" with "suffixes" * aci: add IPA servers host group 'ipaservers' * aci: replace per-server ACIs with ipaserver-based ACIs * aci: allow members of ipaservers to set up replication * ipautil: use file in a temporary dir as ccache in private_ccache * replica promotion: use host credentials when setting up replication * replica promotion: automatically add the local host to ipaservers * custodia: do not modify memberPrincipal on key update * replica promotion: allow OTP bulk client enrollment * replica install: add ipaservers if it does not exist * replica promotion: check domain level before ipaservers membership * server uninstall: ignore --ignore-topology-disconnect in domain level 0 * spec file: remove config files from freeipa-python * spec file: put Python modules into standalone packages * build: put oddjob scripts into separate directory * replica install: add remote connection check over API * replica promotion: use host credentials for connection check * replica promotion: notify user about ignoring client enrollment options * aci: merge domain and CA suffix replication agreement ACIs * ca install: use host credentials in domain level 1 * ipautil: allow redirecting command output to standard output in run() * server install: redirect ipa-client-install output to standard output * replica promotion: let ipa-client-install validate enrollment options * ipautil: remove unused import causing cyclic import in tests === Jan Pazdziora (1) === * The delegation uris are not set, match message to code. === Lenka Doudova (3) === * Automated test for stageuser plugin * Fix user tracker to reflect new user-del message * Adding descriptive IDs to stageuser tests === Ludwig Krispenz (5) === * handle multiple managed suffixes * prevent operation on tombstones * handle cleaning of RUV in the topology plugin * reject agreement only if both ends are managed * update list of managed servers when a suffix becomes managed === Luk?? Slebodn?k (9) === * SPEC: Drop sssd from BuildRequires * ipa_kdb_tests: Remove unused variables * ipa_kdb_tests: Fix warning Wmissing-braces * topology: Fix warning Wshadow * ipa-extdom-extop: Fix warning Wformat * SPEC: Run cmocka based unit test in %check phase * BUILD: provide check target in custom Makefiles * cmocka_tests: Do not use deprecated cmocka interface * ipa_kdb_tests: Fix test with default krb5.conf === Martin Babinsky (50) === * ipa-ca-install: print more specific errors when CA is already installed * enable debugging of ntpd during client installation * fix broken search for users by their manager * ACI plugin: correctly parse bind rules enclosed in parentheses * test suite for user/host/service certificate management API commands * store certificates issued for user entries as userCertificate;binary * idranges: raise an error when local IPA ID range is being modified * fix typo in BasePathNamespace member pointing to ods exporter config * ipa-backup: archive DNSSEC zone file and kasp.db * ipa-restore: check whether DS is running before attempting connection * improve the handling of krb5-related errors in dnssec daemons * improve the usability of `ipa user-del --preserve` command * load RA backend plugins during standalone CA install on CA-less IPA master * destroy httpd ccache after stopping the service * ipa-server-install: mark master_password Knob as deprecated * re-kinit after ipa-restore in backup/restore CI tests * do not overwrite files with local users/groups when restoring authconfig * remove ID overrides when deleting a user * do not ask for segment direction when running topology commands * fix dsinstance.py:get_domain_level function * disable ipa-replica-prepare in non-zero IPA domain level * execute user-del pre-callback also during user preservation * fix class teardown in user plugin tests * always ask the resolver for the reverse zone when manipulating PTR records * silence pylint in Python 3-specific portion of ipalib/rpc.py * ipa-replica-prepare: domain level check improvements * fix error reporting when installer option is supplied with invalid choice * remove Kerberos authenticators when installing/uninstalling service instance * remove an unneccesary check from IPA server uninstaller * check for disconnected topology and deleted agreements for all suffices * suppress errors arising from adding existing LDAP entries during KRA install * update idrange tests to reflect disabled modification of local ID ranges * disconnect ldap2 backend after adding default CA ACL profiles * do not disconnect when using existing connection to check default CA ACLs * fix a typo in replica DS creation code * replica promotion: modify default.conf even if DS configuration fails * perform IPA client uninstallation as a last step of server uninstall * fix 'iparepltopomanagedsuffix' attribute consumers * extract domain level 1 topology-checking code from ipa-replica-manage * implement domain level 1 specific topology checks into IPA server uninstaller * replica install: improvements in the handling of CA-related IPA config entries * add auto-forwarders option to standalone DNS installer * add '--auto-forwarders' description to server/replica/DNS installer man pages * check whether replica exists before executing the domain level 1 deletion code * CI tests: ignore disconnected domain level 1 topology on IPA master teardown * add ACIs for custodia container to its parent during IPA upgrade * fix error message assertion in negative forced client reenrollment tests * prevent crashes of server uninstall check caused by failed LDAP connections * CI tests: remove '-p' option from ipa-dns-install calls * ipa-client-install: create a temporary directory for ccache files === Martin Ba?ti (92) === * Prevent to rename certprofile profile id * Stageusedr-activate: show username instead of DN * copy-schema-to-ca: allow to overwrite schema files * fix selinuxusermap search for non-admin users * Validate adding privilege to a permission * sysrestore: copy files instead of moving them to avoind SELinux issues * Allow value 'no' for replica-certify-all attr in abort-clean-ruv subcommand * Py3: replace tab with space * DNS: Consolidate DNS RR types in API and schema * DNS: check if DNS package is installed * Remove ico files from Makefile * Use 'mv -Z' in specfile to restore SELinux context * ULC: Fix stageused-add --from-delete command * Fix upgrade of sidgen and extdom plugins * Add dependency to SSSD 1.13.1 * Server Upgrade: Start DS before CA is started. * Add user-stage command * DNSSEC: fix forward zone forwarders checks * Fix: Remove leftover krbV reference * DNSSEC: remove "DNSSEC is experimental" warnings * Backup: back up the hosts file * Server Upgrade: fix traceback caused by cidict * Installer: do not modify /etc/hosts before user agreement * DNSSEC: backup and restore opendnssec zone list file * DNSSEC: remove ccache and keytab of ipa-ods-exporter * FIX vault tests * Server Upgrade: backup CS.cfg when dogtag is turned off * IPA Restore: allows to specify files that should be removed * Server Install: print message that client is being installed * DNSSEC: improve CI test * DNSSEC CI: test master migration * backup CI: test DNS/DNSSEC after backup and restore * Limit max age of replication changelog * Server Upgrade: addifnew should not create entry * CI: backup and restore with KRA * Replica inst. fix: do not require -r, -a, -p options in unattended mode * CI TEST: Vault * CI Test: add setup_kra options into install scripts * Replace tab with space in test_user_plugin.py * Make offline LDIF modify more robust * Add method to read changes from LDIF * Add option to specify LDIF file that contains DS configuration changes * CI: installation with customized DS config * Rename option --dirsrv-config-mods to --dirsrv-config-file * DNSSEC CI: wait until DS records is replicated * DNSSEC: store status of services only before first install * DNSSEC: Remove service containers from LDAP after uninstalling * DNSSEC: warn user if DNSSEC key master is not installed * ipa-replica-manage: fix undefined variable * Remove executable bit from ipa_kra_install.py * Domain levels: use constants rather than hardcoded values * KRA: fix check that CA is installed * ipa-csreplica-manage: disable connect/disconnect/del with domain level > 0 * Fix typo in ods-exporter uninstall to restore state * DNSSEC: remove sysrestore state after uninstall * Upgrade: enable custodia service during upgrade * Use domain level constants in topology plugin * Tests: DNS replace 192.0.2.0/24 with 198.18.0.0/15 range * Tests: DNS various exceptions can be raised in test * Drop configure.jar * Fix CI tests domain_level env config * CI test: Fix installation of KRA on a replica * fix caching in get_ipa_config * Move common code of user and stageuser to baseuser postcallback * Allow multiple managers per user - CLI part * upgrade: fix migration of old dns forward zones * remove forgotten print in DNS plugin * Install: Force service add during replica promotion * Fix upgrade of forwardzones when zone is in realmdomains * Remove invalid error messages from topology upgrade * Make command dns-resolve deprecated. * Call directly function is_host_resolvable instead do call via framework * Use absolute domain in detection of A/AAAA records * ipa-getkeytab: do not return error when translations cannot be loaded * Compare objectclasses as case insensitive in baseuser.py * KRA: do not stop certmonger during standalone uninstall * ipa-ca-install: error when replica file is passed with domain level > 0 * KRA install: show installation message only if install really started * ipa-kra-install: error when replica file is passed with domain level > 0 * FIX: ipa_kdb_principals: add missing break statement * Upgrade: increase time limit for upgrades * ipa-kra-install: allow to install first KRA on replica * Modify error message to install first instance of KRA * CI: test various topologies with multiple replicas * Force creation of services during replica install * CI: installation tests * CI: fix function that prepare the hosts file before CI run * CI: fix ipa-kra-install on domain level 1 * Install RA cert during replica promotion * Tests: test_ipagetkeytab: fix assert that is always true * DNS: fix file permissions * Explicitly call chmod on newly created directories === Martin Ko?ek (2) === * Update Contributors.txt * Update Build instructions === Michael Simacek (4) === * Port from python-kerberos to python-gssapi * Bump python-gssapi version to 1.1.2 * Port from python-krbV to python-gssapi * Rewrap errors in get_principal to CCacheError === Milan Kub?k (16) === * ipalib: pass api instance into textui in doctest snippets * spec file: update the python package names for libipa_hbac and libsss_nss_idmap * tests: Allow Tracker.dn be an instance of Fuzzy * ipatests: Take otptoken import test out of execution * ipatests: Add Certprofile tracker class implementation * ipatests: Add basic tests for certificate profile plugin * ipatests: configure Network Manager not to manage resolv.conf * Include ipatests/test_xmlrpc/data directory into distribution. * ipatests: add fuzzy instances for CA ACL DN and RDN * ipatests: Add initial CAACLTracker implementation * tests: add test to check the default ACL * ipatests: CA ACL - added config templates * ipatests: added unlock_principal_password and change_principal * ipatests: CA ACL and cert profile functional test * Applied tier0 and tier1 marks on unit tests and xmlrpc tests * Separated Tracker implementations into standalone package === Nathaniel McCallum (1) === * Fix an integer underflow bug in libotp === Niranjan MR (1) === * enable pem=True in export_pem_cert function === Niranjan Mallapadi (1) === * Use Exception class instead of StandardError === Oleg Fayans (9) === * Added test - topology plugin is listed among DS plugins * Added a user-friendly output to an import error * Temporary fix for ticket 5240 * Integration tests for topology plugin * Added a proper workaround for dnssec test failures in Beaker environment * Fixed a timing issue with drill returning non-zero exitcode * Updated the tests according to the new replica installation workflow * The test was made to be skipped if domainlevel is 0 * Fixed A record creation bug === Petr Viktorin (60) === * Modernize number literals * Modernize 'except' clauses * Modernize function and method attribute names * Replace dict.has_key with the 'in' operator * Import 'reduce' from functools * Use absolute imports * Remove use of sys.exc_value * Don't use a tuple in function arguments * Add python-six to dependencies * Remove the unused pygettext script * Use six.string_types instead of "basestring" * Use Python3-compatible dict method names * Replace filter() calls with list comprehensions * Use six.moves.input instead of raw_input * Use six.integer_types instead of (long, int) * Replace uses of map() * Use next() function on iterators * Use the print function * Use new-style raise syntax * Use six.reraise * Modernize use of range() * Convert zip() result to list() * ipap11helper: Port to Python 3 * rpc: Don't use undocumented urllib functions * ipapython.dn: Use rich comparisons * test_dn: Split bytes and unicode * Use sys.maxsize instead of sys.maxint * Use six.moves.urllib instead of urllib/urllib2/urlparse * Use six.moves.xmlrpc.client instead of xmlrpclib * Use six.moves.configparser instead of ConfigParser * Use six.moves.http_client instead of httplib * Use six.Stringio instead of StringIO.StringIO * Remove uses of the `types` module * ipapython.ssh: Port to Python 3 * Appease pylint * Do not compare types that are not comparable in Python 3 * x509: Port to Python 3 * Rename caught exception for use outside the except: block. * test_ipalib.test_frontend: Port unbound method tests to Python 3 * ipalib.aci: Port to Python 3 * Add `message` property to IPA's errors and warnings under Python 3 * test_keyring: Use str(e) instead of e.message for exceptions * ipalib.parameters: Handle 0-prefixed octal format of ints * ipalib.parameters: Require bytes for Bytes.pattern * rpc: Name argument to KerberosError * Alias long to int under Python 3 * ipaldap: Remove extraneous `long` (included in six.int_types) * Handle binascii.Error from base64.b64decode() * ipatest.util: Port to Python 3 * ipalib.messages: Add "message" property to PublicMessage * Fix more bytes/unicode issues * Work around ipalib.text (i18n) str/unicode handling * Fix left-over Python 3 syntax errors * ipapython.nsslib, ipalib.rpc: Remove code for Python 2.6 and below * ipapython.nsslib: Remove NSSHTTPS * ipapython.secrets: Port to Python 3 * test_parameters: Alias long to int under Python 3 * ipalib.rpc: Update for Python 3 * Refactor ipautil.run * Package ipapython, ipalib, ipaplatform, ipatests for Python 3 === Petr Voborn?k (45) === * Become IPA 4.2.0 * Bump 4.3 development version to 4.2.90 * do not import memcache on client * webui: fix user reset password dialog * fix hbac rule search for non-admin users * webui: add Kerberos configuration instructions for Chrome * webui: fix regressions failed auth messages * webui: add LDAP vs Kerberos behavior description to user auth types * adjust search so that it works for non-admin users * validate mutually exclusive options in vault-add * add permission: System: Manage User Certificates * vault: normalize service principal in service vault operations * vault: validate vault type * vault: change default vault type to symmetric * fix missing information in object metadata * webui: add option to establish bidirectional trust * vault: fix vault tests after default type change * vault: add vault container commands * webui: use manual Firefox configuration for Firefox >= 40 * webui: improve performance of search in association dialog * topology: add realm suffix to master entry on update * topology: manage ca replication agreements * enable topology plugin on upgrade * topology plugin configuration workaround * change pki-core required version for replica promotion * Update .po files * fix broken translations after last po update * webui: add Deferred/Promise API to rpc.command * webui: split facet header into two classes * webui: extract header and action logic from facet to separate mixins * webui: allow to update action_state directly * webui: add d3 library - version 3.5.6 * webui: topology graph component * webui: topology graph facet * webui: add segments on topology graph page * webui: remove segments on topology graph page * webui: update topology graph after raising domain level * topology: treat server suffix as multivalued attribute in API * use starttls in CSReplicationManager connection again * change suffices to suffixes * topologysuffix: change iparepltopoconfroot API properties * rename topology suffixes to "domain" and "ca" * Update ipa-(cs)replica-manage man pages * Extend topology help * Become IPA 4.3.0 === Petr ?pa?ek (19) === * Create server-dns sub-package. * DNSSEC: prevent ipa-ods-exporter from looping after service auto-restart * DNSSEC: Fix deadlock in ipa-ods-exporter <-> ods-enforcerd interaction * DNSSEC: Fix HSM synchronization in ipa-dnskeysyncd when running on DNSSEC key master * DNSSEC: Fix key metadata export * DNSSEC: Wrap master key using RSA OAEP instead of old PKCS v1.5. * Avoid ipa-dnskeysync-replica & ipa-ods-exporter crashes caused by exceeding LDAP limits * ipa-adtrust-install: Print complete SRV records * DNSSEC: on uninstall, do not restore OpenDNSSEC kasp.db if backup failed * DNSSEC: improve log messages in uninstaller * DNS record-add warns when a suspicious DNS name is detected * Remove dead code in ipaserver/install/installutils: read_ip_address() * Remove unused constant NEW_MASTER_MARK from ipaserver.install.dns * ipa-client-install: add support for Ed25519 SSH keys (RFC 7479) * ipa-dns-install offer IP addresses from resolv.conf as default forwarders * Remove global variable dns_forwarders from ipaserver.install.dns * add missing /ipaplatform/constants.py to .gitignore * Makefile: disable parallel build * dns: Handle SERVFAIL in check if domain already exists. === Rob Crittenden (1) === * Use %license instead of %doc for packaging the license === Robert Kuska (1) === * Replace StandardError with Exception === Simo Sorce (23) === * Fix DNS records installation for replicas * Remove custom utility function from krbinstance * Move sasl mappings creation to dsinstance * Simplify adding options in ipachangeconf * Insure the admin_conn is disconnected on stop * Remove unused arguments * Simplify the install_replica_ca function * Add ipa-custodia service * Require a DS version that has working DNA plugin * Implement replica promotion functionality * Change DNS installer code to use passed in api * Allow ipa-replica-conncheck to use default creds * Add function to extract CA certs for install * Allow to setup the CA when promoting a replica * Make checks for existing credentials reusable * Add low level helper to get domain level * Remove unused kra option * Allow ipa-ca-install to use the new promotion code * Allow to install the KRA on a promoted server * Check early if a CA is already installed locally * Return default TL_DATA is krbExtraData is missing * Support sourcing the IPA server name from config * Sync kerberos LDAP schema with upstream. === Stanislav Laznicka (3) === * ipa-client-install: warn when IP used in --server * Fixes disappearing automember expressions * Removed duplicate domain name validating function === Sumit Bose (3) === * ipasam: fix wrong usage of talloc_new() * ipasam: use more restrictive search filter for group lookup * ipasam: fix a use-after-free issue === Timo Aaltonen (7) === * paths: Add GENERATE_RNDC_KEY. * httpinstance: Replace a hardcoded path to password.conf with HTTPD_PASSWORD_CONF * ipaplatform: Add HTTPD_USER to constants, and use it. * ipaplatform: Add NAMED_USER to constants * httpinstance: Use full path via HTTPD_IPA_REWRITE_CONF for Include. * ipaplatform: Add SECURE_NFS_VAR to constants * ipaplatform: Add NTPD_OPTS_VAR and NTPD_OPTS_QUOTE to constants === Tom?? Babej (59) === * ipalib: Fix missing format for InvalidDomainLevelError * Revert "Hide topology and domainlevel features" * trusts: Check for AD root domain among our trusted domains * domainlevel: Fix incorrect initializations of InvalidDomainLevelError exceptions * ipaplatform: Add constants submodule * tests: user_plugin: Add preserved flag when --all is used * dcerpc: Expand explanation for WERR_ACCESS_DENIED * idviews: Check for the Default Trust View only if applying the view * tests: service_plugin: Make sure the cert is decoded from base64 * tests: realmdomains_plugin: Add explanatory comment * tests: Version is currently generated during command call * tests: vault_plugin: Skip tests if KRA not available * tests: test_rpc: Create connection for the current thread * tests: test_cert: Services can have multiple certificates * dcerpc: Fix UnboundLocalError for ccache_name * dcerpc: Add get_trusted_domain_object_type method * idviews: Restrict anchor to name and name to anchor conversions * idviews: Enforce objectclass check in idoverride*-del * replication: Fix incorrect exception invocation * Fix incorrect type comparison in trust-fetch-domains * dcerpc: Simplify generation of LSA-RPC binding strings * adtrust-install: Correctly determine 4.2 FreeIPA servers * trusts: Detect domain clash with IPA domain when adding a AD trust * trusts: Detect missing Samba instance * winsync-migrate: Add warning about passsync * winsync-migrate: Expand the man page * winsync: Add inetUser objectclass to the passsync sysaccount * ipa-backup: Add mechanism to store empty directory structure * winsync-migrate: Convert entity names to posix friendly strings * winsync-migrate: Properly handle collisions in the names of external groups * util: Add detect_dns_zone_realm_type helper * realmdomains: Minor style and wording improvements * realmdomains: Add validation that realmdomain being added is indeed from our realm * realmdomains: Issue a warning when automated management of realmdomains failed * realmdomains: Do not fail due the ValidationError when adding _kerberos TXT record * tests: Amend result assertions in realmdomains tests * idoverride: Ignore ValidationErrors when converting the anchor * tests: Add tests for idoverride object integrity * trusts: Make trust_show.get_dn raise properly formatted NotFound * trustdomain: Perform validation of the trust domain first * adtrustinstance: Wait for sidgen task completion * adtrustinstance: Restart samba service at the end of adtrust-install * adtrustinstance: Do not use bare except clauses * ipachangeconf: Remove reference to an old-style interface * spec: Add Provides directives to alternative package names * private_ccache: Harden the removal of KRB5CCNAME env variable * ipachangeconf: Add ability to preserve section case * ipa-client-automount: Leverage IPAChangeConf to configure the domain for idmapd * custodia: Make sure container is created with first custodia replica * replicainstall: Add possiblity to install client in one command * translations: Update ipa.pot file * man: Update the ipa-replica-install manpage with promotion related info * tests: Fix incorrect uninstall method invocation * replicainstall: Admin password should not conflict with replica file * topology: Make sure the old 'realm' topology suffix is not used * topology: Fix: Make sure the old 'realm' topology suffix is not used * tests: Add hostmask detection for sudo rules validating on hostmask * replicainstall: Add check for domain if server is specified * replicainstall: Make sure the enrollment state is preserved === Yuri Chornoivan (2) === * Fix minor typos * Fix minor typos -- Petr Vobornik From mbasti at redhat.com Sat Dec 19 19:02:52 2015 From: mbasti at redhat.com (Martin Basti) Date: Sat, 19 Dec 2015 20:02:52 +0100 Subject: [Freeipa-devel] [PATCH-0019][Tests]Enabled --auto-reverse by default for master installation In-Reply-To: <5673C54D.50805@redhat.com> References: <5672B04E.9040602@redhat.com> <20151217130511.GA16380@mail.corp.redhat.com> <5672B83C.3030700@redhat.com> <56730AD7.4090106@redhat.com> <5673C54D.50805@redhat.com> Message-ID: <5675A9DC.5060009@redhat.com> On 18.12.2015 09:35, Oleg Fayans wrote: > Hi Petr, > > On 12/17/2015 08:19 PM, Petr Spacek wrote: >> On 17.12.2015 14:27, Oleg Fayans wrote: >>> Commit message was updated. Thanks! >>> >>> On 12/17/2015 02:05 PM, Lukas Slebodnik wrote: >>>> On (17/12/15 13:53), Oleg Fayans wrote: >>>>> -- >>>>> Oleg Fayans >>>>> Quality Engineer >>>>> FreeIPA team >>>>> RedHat. >>>> >From ed4630140386c1043e36733eb42ec402cc276bee Mon Sep 17 00:00:00 2001 >>>>> From: Oleg Fayans >>>>> Date: Thu, 17 Dec 2015 13:50:19 +0100 >>>>> Subject: [PATCH] Enabled automatic creation of reverse zone during master >>>>> installation >>>>> >>>> The commit message does not contain ticket. >>>> It is also not explained why this change was done. >>>> Could you update commit message with verbose explanation? >>>> >>>> Proper commit message might prevent removal of this change in future >>>> based on "git blame" >> Seems reasonable in general but this should not happen in BRQ or ABCDE labs. >> Where do you see the problem? >> >> Otherwise ACK. > The problem occurs in local libvirt-powered VMs. Also it may (or may not > - I did not try it yet) be met in Beaker. > > NACK ipa-server-install: error: You cannot specify a --auto-reverse option without the --setup-dns option From mbasti at redhat.com Sat Dec 19 22:56:12 2015 From: mbasti at redhat.com (Martin Basti) Date: Sat, 19 Dec 2015 23:56:12 +0100 Subject: [Freeipa-devel] [PATCH 016 - 017] First part of the replica promotion tests + testplan In-Reply-To: <56727AA7.2030809@redhat.com> References: <5640A90B.6030906@redhat.com> <564CA530.3060006@redhat.com> <5655D741.4010402@redhat.com> <56560124.4060904@redhat.com> <5656CB3B.8010000@redhat.com> <5656D5EE.7010208@redhat.com> <565862E6.4020100@redhat.com> <56586807.9080805@redhat.com> <565870D6.40908@redhat.com> <56587616.5030609@redhat.com> <565DB7D3.5040500@redhat.com> <565F0BDD.8080206@redhat.com> <565F17EE.3020701@redhat.com> <5666DC18.2040906@redhat.com> <56675E31.3050203@redhat.com> <5667F49B.5020709@redhat.com> <5667FEFB.6090904@redhat.com> <56680C2E.7060805@redhat.com> <56680DDD.2000106@redhat.com> <566FDD80.7030805@redhat.com> <56717785.4020304@redhat.com> <567195B9.9070104@redhat.com> <56727AA7.2030809@redhat.com> Message-ID: <5675E08C.7030109@redhat.com> On 17.12.2015 10:04, Oleg Fayans wrote: > Hi Martin, > > I am sorry, in my previous email I attached the old version of patch > 0016. The correct on is attached. > > On 12/16/2015 05:47 PM, Martin Basti wrote: >> >> On 16.12.2015 15:39, Martin Basti wrote: >>> >>> On 15.12.2015 10:29, Oleg Fayans wrote: >>>> Hi Martin, >>>> >>>> The updated patches are attached. Patch 0017 includes all changes from >>>> patch 0018, so, if you approve this one, there would be no need to >>>> continue with the review of 0018. This one contains all changes related >>>> to you remarks from 0018 review. Please see my explanation on the >>>> stdout+stderr part in the thread from patch 0018. >>>> With these two patches applied one of the tests fails due this bug: >>>> https://fedorahosted.org/freeipa/ticket/5550 >>>> >>>> On 12/09/2015 12:17 PM, Martin Basti wrote: >>>>> On 09.12.2015 12:10, Martin Basti wrote: >>>>>> On 09.12.2015 11:14, Oleg Fayans wrote: >>>>>>> Hi Martin >>>>>>> >>>>>>> On 12/09/2015 10:30 AM, Martin Basti wrote: >>>>>>>> On 08.12.2015 23:48, Oleg Fayans wrote: >>>>>>>>> Substituted a hardcoded suffix name with a constant >>>>>>>>> DOMAIN_SUFFIX_NAME >>>>>>>>> >>>>>>>>> On 12/08/2015 02:33 PM, Oleg Fayans wrote: >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The patches are rebased against the current master. >>>>>>>>>> >>>>>>>>>> On 12/02/2015 05:10 PM, Martin Basti wrote: >>>>>>>>>>> On 02.12.2015 16:18, Oleg Fayans wrote: >>>>>>>>>>>> Hi Martin, >>>>>>>>>>>> >>>>>>>>>>>> On 12/01/2015 04:08 PM, Martin Basti wrote: >>>>>>>>>>>>> On 27.11.2015 16:26, Oleg Fayans wrote: >>>>>>>>>>>>>> And patch N 16 passes lint too: >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 11/27/2015 04:03 PM, Oleg Fayans wrote: >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 11/27/2015 03:26 PM, Martin Basti wrote: >>>>>>>>>>>>>>>> On 27.11.2015 15:04, Oleg Fayans wrote: >>>>>>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> All your suggestions were taken into account. Both >>>>>>>>>>>>>>>>> patches are >>>>>>>>>>>>>>>>> updated. Thank you for your help! >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 11/26/2015 10:50 AM, Martin Basti wrote: >>>>>>>>>>>>>>>>>> On 26.11.2015 10:04, Oleg Fayans wrote: >>>>>>>>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I agree to all your points but one. please, see my >>>>>>>>>>>>>>>>>>> comment >>>>>>>>>>>>>>>>>>> below >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 11/25/2015 07:42 PM, Martin Basti wrote: >>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 0) Note >>>>>>>>>>>>>>>>>>>> Please be aware of >>>>>>>>>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5469 >>>>>>>>>>>>>>>>>>>> during >>>>>>>>>>>>>>>>>>>> KRA testing >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>>>>>>> Please do not use MIN and MAX_DOMAIN_LEVEL constants, >>>>>>>>>>>>>>>>>>>> this may >>>>>>>>>>>>>>>>>>>> change >>>>>>>>>>>>>>>>>>>> over time, use DOMAIN_LEVEL_0 and DOMAIN_LEVEL_1 for >>>>>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>>>>> level 0 >>>>>>>>>>>>>>>>>>>> and 1 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>>>>>>> Why uninstall KRA then server, is not enough just >>>>>>>>>>>>>>>>>>>> uninstall >>>>>>>>>>>>>>>>>>>> server >>>>>>>>>>>>>>>>>>>> which >>>>>>>>>>>>>>>>>>>> covers KRA uninstall? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> + def teardown_method(self, method): >>>>>>>>>>>>>>>>>>>> + for host in self.replicas: >>>>>>>>>>>>>>>>>>>> + host.run_command(self.kra_uninstall, >>>>>>>>>>>>>>>>>>>> raiseonerr=False) >>>>>>>>>>>>>>>>>>>> + tasks.uninstall_master(host) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>>>>>>> Can be this function more generic? It should allow >>>>>>>>>>>>>>>>>>>> specify >>>>>>>>>>>>>>>>>>>> host >>>>>>>>>>>>>>>>>>>> where >>>>>>>>>>>>>>>>>>>> KRA should be installed not just master >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> + def test_kra_install_master(self): >>>>>>>>>>>>>>>>>>>> + self.master.run_command(self.kra_install) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> TestLevel0(Dummy): >>>>>>>>>>>>>>>>>>>> Can be the test name more specific, something like >>>>>>>>>>>>>>>>>>>> TestReplicaPromotionLevel0 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 5) >>>>>>>>>>>>>>>>>>>> please remove this, the patch is on review and it >>>>>>>>>>>>>>>>>>>> will be >>>>>>>>>>>>>>>>>>>> pushed >>>>>>>>>>>>>>>>>>>> sooner >>>>>>>>>>>>>>>>>>>> than tests >>>>>>>>>>>>>>>>>>>> + @pytest.mark.xfail # Ticket N 5455 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> and as I mentioned in ticket #5455, I cannot reproduce >>>>>>>>>>>>>>>>>>>> it with >>>>>>>>>>>>>>>>>>>> ipa-kra-install, so please provide steps to reproduce if >>>>>>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>>>>> insist >>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>> this still does not work as expected with KRA. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 6) This is completely wrong, it removes everything >>>>>>>>>>>>>>>>>>>> that we >>>>>>>>>>>>>>>>>>>> tried to >>>>>>>>>>>>>>>>>>>> achieve with previous patches with domain level in CI >>>>>>>>>>>>>>>>>>> Actually, being able to configure domain level per class >>>>>>>>>>>>>>>>>>> is WAY >>>>>>>>>>>>>>>>>>> more >>>>>>>>>>>>>>>>>>> convenient, than to always have to think which domain >>>>>>>>>>>>>>>>>>> level is >>>>>>>>>>>>>>>>>>> appropriate for which particular test during jenkins job >>>>>>>>>>>>>>>>>>> configuration. In fact, I should have thought about it >>>>>>>>>>>>>>>>>>> from the >>>>>>>>>>>>>>>>>>> very >>>>>>>>>>>>>>>>>>> beginning. For example, in test_replica_promotion.py we >>>>>>>>>>>>>>>>>>> have on >>>>>>>>>>>>>>>>>>> class, >>>>>>>>>>>>>>>>>>> which intiates with domain level = 1, while others - with >>>>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>>>>> 0. With config-based approach, we would have to >>>>>>>>>>>>>>>>>>> implement a >>>>>>>>>>>>>>>>>>> separate >>>>>>>>>>>>>>>>>>> step that raises domain level. Overall, I am against the >>>>>>>>>>>>>>>>>>> approach, >>>>>>>>>>>>>>>>>>> when you have to remember to set certain domain level in >>>>>>>>>>>>>>>>>>> config >>>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>> any particular test. The tests themselves should be >>>>>>>>>>>>>>>>>>> aware of >>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> domain level they need. >>>>>>>>>>>>>>>>>> I do not say that we should not have something that >>>>>>>>>>>>>>>>>> overrides >>>>>>>>>>>>>>>>>> settings >>>>>>>>>>>>>>>>>> in from config in a particular test case, I say your >>>>>>>>>>>>>>>>>> patch is >>>>>>>>>>>>>>>>>> doing it >>>>>>>>>>>>>>>>>> wrong. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I agree it is useful to have param domain_level in >>>>>>>>>>>>>>>>>> install_master, >>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>> intall_topo methods, but is cannot be MAX_DOMAIN_LEVEL by >>>>>>>>>>>>>>>>>> default, >>>>>>>>>>>>>>>>>> because with your current patch the domain_level in >>>>>>>>>>>>>>>>>> config is >>>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>> used >>>>>>>>>>>>>>>>>> at all, it will be always MAX_DOMAIN_LEVEL >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> For example I want to achieve this goal: >>>>>>>>>>>>>>>>>> test_vault.py, this test suite can run on domain level1 >>>>>>>>>>>>>>>>>> and on >>>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>>> level0, so with one test we can test 2 domain levels just >>>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>> putting >>>>>>>>>>>>>>>>>> domain level into config file. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I agree that with extraordinary test like replica >>>>>>>>>>>>>>>>>> promotion test >>>>>>>>>>>>>>>>>> is, we >>>>>>>>>>>>>>>>>> need something that allows override the config file. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> As I said bellow, domain_level default value should be >>>>>>>>>>>>>>>>>> None in >>>>>>>>>>>>>>>>>> install_master and install_topo plugin. If domain level >>>>>>>>>>>>>>>>>> was >>>>>>>>>>>>>>>>>> specified >>>>>>>>>>>>>>>>>> use the specified one, if not (value is None) use the >>>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>>>> from >>>>>>>>>>>>>>>>>> config file. >>>>>>>>>>>>>>>>> Agreed :) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Martin >>>>>>>>>>>>>>>>>>>> [PATCH] Enabled setting domain_level per class >>>>>>>>>>>>>>>>>>>> derived from >>>>>>>>>>>>>>>>>>>> TestIntegration >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> When I configure domain level 0 in yaml config, how >>>>>>>>>>>>>>>>>>>> is this >>>>>>>>>>>>>>>>>>>> supposed to >>>>>>>>>>>>>>>>>>>> get into install methods when you removed that code? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> - "--domain-level=%i" % host.config.domain_level >>>>>>>>>>>>>>>>>>>> + "--domain-level=%i" % domain_level >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> You always use MAX_DOMAIN_LEVEL in this case or >>>>>>>>>>>>>>>>>>>> whatever is >>>>>>>>>>>>>>>>>>>> specified in >>>>>>>>>>>>>>>>>>>> domain_level option. >>>>>>>>>>>>>>>>>>>> I suggest to use domain_level=None, and when it is >>>>>>>>>>>>>>>>>>>> None use >>>>>>>>>>>>>>>>>>>> 'host.config.domain_level', if it is not None, use >>>>>>>>>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> With this we can specify domain level in config file for >>>>>>>>>>>>>>>>>>>> test >>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>>> be used for both domain levels and you can manually >>>>>>>>>>>>>>>>>>>> specify >>>>>>>>>>>>>>>>>>>> domain >>>>>>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>>>>>> for test that requires specific domain level. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Also this should go away >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> @classmethod >>>>>>>>>>>>>>>>>>>> def install(cls, mh): >>>>>>>>>>>>>>>>>>>> + if hasattr(cls, "domain_level") and cls.master: >>>>>>>>>>>>>>>>>>>> + cls.master.config.domain_level = cls.domain_level >>>>>>>>>>>>>>>>>>>> if cls.topology is None: >>>>>>>>>>>>>>>>>>>> return >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I do not see reason why test should override >>>>>>>>>>>>>>>>>>>> configuration in >>>>>>>>>>>>>>>>>>>> config in >>>>>>>>>>>>>>>>>>>> this case. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Martin >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 25.11.2015 16:44, Oleg Fayans wrote: >>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Here is the updated version of the patch (more tests + >>>>>>>>>>>>>>>>>>>>> fixed the >>>>>>>>>>>>>>>>>>>>> issues of the first one) + patch 0017, that >>>>>>>>>>>>>>>>>>>>> implements the >>>>>>>>>>>>>>>>>>>>> necessary >>>>>>>>>>>>>>>>>>>>> changes in the background code, i. e. patch 16 does not >>>>>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>>>>> without >>>>>>>>>>>>>>>>>>>>> patch 17 >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 11/18/2015 05:20 PM, Martin Basti wrote: >>>>>>>>>>>>>>>>>>>>>> On 09.11.2015 15:09, Oleg Fayans wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi guys, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Here are first two automated testcases from this >>>>>>>>>>>>>>>>>>>>>>> (so far >>>>>>>>>>>>>>>>>>>>>>> incomplete) >>>>>>>>>>>>>>>>>>>>>>> testplan: >>>>>>>>>>>>>>>>>>>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Testplan review is highly appreciated >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> PATCH 16: NACK >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>>>>>>>>> What is the reason to add an unused parameter to >>>>>>>>>>>>>>>>>>>>>> 'domain_level' to >>>>>>>>>>>>>>>>>>>>>> install_topo()? >>>>>>>>>>>>>>>>>>>>>> Also it is good practise to add new option as the last >>>>>>>>>>>>>>>>>>>>>> parameter. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>>>>>>>>> cab you in both tests specify a domain level with >>>>>>>>>>>>>>>>>>>>>> constant >>>>>>>>>>>>>>>>>>>>>> instead of >>>>>>>>>>>>>>>>>>>>>> number literal? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 3) >>>>>>>>>>>>>>>>>>>>>> both test call install_topo with custom domain level, >>>>>>>>>>>>>>>>>>>>>> but it >>>>>>>>>>>>>>>>>>>>>> cannot >>>>>>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>>>>>> because 1) (did you run the test?) >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 4) >>>>>>>>>>>>>>>>>>>>>> How the test "TestLevel1" is supposed to work? >>>>>>>>>>>>>>>>>>>>>> Respectively why there is call of install_topo() that >>>>>>>>>>>>>>>>>>>>>> installs >>>>>>>>>>>>>>>>>>>>>> replica. >>>>>>>>>>>>>>>>>>>>>> As this test just tests that ipa-replica-prepare is >>>>>>>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>>> working >>>>>>>>>>>>>>>>>>>>>> anymore, >>>>>>>>>>>>>>>>>>>>>> is it worth to spend 20 minutes with installing >>>>>>>>>>>>>>>>>>>>>> replica and >>>>>>>>>>>>>>>>>>>>>> then >>>>>>>>>>>>>>>>>>>>>> just no >>>>>>>>>>>>>>>>>>>>>> tot use it? IMO to install master in install step is >>>>>>>>>>>>>>>>>>>>>> enough. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Martin^2 >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ./make-lint >>>>>>>>>>>>>>>> ************* Module ipatests.test_integration.base >>>>>>>>>>>>>>>> ipatests/test_integration/base.py:66: [E1101(no-member), >>>>>>>>>>>>>>>> IntegrationTest.install] Class 'IntegrationTest' has no >>>>>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>>>>> member) >>>>>>>>>>>>>>>> ************* Module >>>>>>>>>>>>>>>> ipatests.test_integration.test_replica_promotion >>>>>>>>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:16: >>>>>>>>>>>>>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no >>>>>>>>>>>>>>>> 'domain_level' >>>>>>>>>>>>>>>> member) >>>>>>>>>>>>>>>> ipatests/test_integration/test_replica_promotion.py:117: >>>>>>>>>>>>>>>> [E1101(no-member), >>>>>>>>>>>>>>>> TestCAInstall.test_ca_install_without_replica_file] >>>>>>>>>>>>>>>> Module 'ipatests.test_integration.tasks' has no >>>>>>>>>>>>>>>> 'setup_replica' >>>>>>>>>>>>>>>> member) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Is it so hard to run pylint before patch is posted for >>>>>>>>>>>>>>>> review? >>>>>>>>>>>>>>> Sorry, my bad. Fixed. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> 1) >>>>>>>>>>>>> Why is this change in the patch? >>>>>>>>>>>>> - # Clean up the test directory >>>>>>>>>>>>> - host.run_command(['rm', '-rvf', host.config.test_dir]) >>>>>>>>>>>> Otherwise 2 out of 8 tests fail with IOError at line 78 of >>>>>>>>>>>> ipatests/test_integration/tasks.py >>>>>>>>>>>> >>>>>>>>>>>> I do not understand yet how does this happen, but if you remove >>>>>>>>>>>> ipatests folder once, it then fails to be created again. >>>>>>>>>>>> >>>>>>>>>>> So this should be in separated patch and investigated properly. >>>>>>>>>> Agree. Removed >>>>>>>>>>>>> - >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> 2) >>>>>>>>>>>>> is enough to have this check only in install_master, >>>>>>>>>>>>> install_topo can >>>>>>>>>>>>> pass None to install_master >>>>>>>>>>>>> + if domain_level is None: >>>>>>>>>>>>> + domain_level = master.config.domain_level >>>>>>>>>>>> Done >>>>>>>>>>>> >>>>>>>>>>>>> 3) >>>>>>>>>>>>> IMO replica-manage del should cleanup hosts entry, so following >>>>>>>>>>>>> code >>>>>>>>>>>>> should not be needed. >>>>>>>>>>>>> + if cleanhost: >>>>>>>>>>>>> + kinit_admin(master) >>>>>>>>>>>>> + master.run_command(["ipa", "host-del", "--updatedns", >>>>>>>>>>>>> replica.hostname], >>>>>>>>>>>>> + raiseonerr=False) >>>>>>>>>>>> Well, in fact it does not. At least the corresponding dns record >>>>>>>>>>>> stays >>>>>>>>>>>> and causes the subsequent ipa-client-install to fail. Probably >>>>>>>>>>>> it's a >>>>>>>>>>>> bug. On the other hand, if I promote an existing client to >>>>>>>>>>>> replica and >>>>>>>>>>>> then delete this replica, then, I probably want the host record >>>>>>>>>>>> (that >>>>>>>>>>>> was created during client-install) to stay in the system. So, >>>>>>>>>>>> does not >>>>>>>>>>>> look like a bug to me. >>>>>>>>>>> No you don't, because replica uninstallation also removes the >>>>>>>>>>> client. >>>>>>>>>>> >>>>>>>>>>> I tried it with ipa42, ipa-replica-manage del removes host entry, >>>>>>>>>>> and >>>>>>>>>>> DNS A records, only SSHFP records stay there (I'm not sure if it >>>>>>>>>>> is bug >>>>>>>>>>> or feature) >>>>>>>>>>> >>>>>>>>>>> Also I received this message >>>>>>>>>>> """ >>>>>>>>>>> Failed to cleanup replica1.ipa.test DNS entries: no matching >>>>>>>>>>> entry >>>>>>>>>>> found >>>>>>>>>>> You may need to manually remove them from the tree >>>>>>>>>>> """ >>>>>>>>>>> But, A record has been removed, so this is probably false >>>>>>>>>>> positive and >>>>>>>>>>> it needs to have a ticket. >>>>>>>>>> Agree, that was an issue with my setup. >>>>>>>>>> Removed >>>>>>>>>> >>>>>>>>>>>>> 4) >>>>>>>>>>>>> This variable is not used >>>>>>>>>>>>> + kra_uninstall = ["ipa-kra-install", "--uninstall", "-U"] >>>>>>>>>>>> Removed >>>>>>>>>>>> >>>>>>>>>>>>> 5) >>>>>>>>>>>>> Why do you need this >>>>>>>>>>>>> + kra_install = ["ipa-kra-install", "-U", "-p", >>>>>>>>>>>>> config.dirman_password] >>>>>>>>>>>>> when you implemented tasks.install_kra that returns the exactly >>>>>>>>>>>>> the same >>>>>>>>>>>>> result? >>>>>>>>>>>> Right. Removed >>>>>>>>>>>> >>>>>>>>>>>>> 6) >>>>>>>>>>>>> PEP8 >>>>>>>>>>>>> ./ipatests/test_integration/tasks.py:928:1: E302 expected 2 >>>>>>>>>>>>> blank >>>>>>>>>>>>> lines, >>>>>>>>>>>>> found 1 >>>>>>>>>>>>> ./ipatests/test_integration/tasks.py:934:1: E302 expected 2 >>>>>>>>>>>>> blank >>>>>>>>>>>>> lines, >>>>>>>>>>>>> found 1 >>>>>>>>>>>>> ./ipatests/test_integration/tasks.py:939:1: E302 expected 2 >>>>>>>>>>>>> blank >>>>>>>>>>>>> lines, >>>>>>>>>>>>> found 1 >>>>>>>>>>>>> ./ipatests/test_integration/tasks.py:943:1: E302 expected 2 >>>>>>>>>>>>> blank >>>>>>>>>>>>> lines, >>>>>>>>>>>>> found 1 >>>>>>>>>>>>> ./ipatests/test_integration/tasks.py:950:80: E501 line too long >>>>>>>>>>>>> (80 > 79 >>>>>>>>>>>>> characters) >>>>>>>>>>>>> >>>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:29:80: >>>>>>>>>>>>> E501 >>>>>>>>>>>>> line >>>>>>>>>>>>> too long (85 > 79 characters) >>>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:64:80: >>>>>>>>>>>>> E501 >>>>>>>>>>>>> line >>>>>>>>>>>>> too long (85 > 79 characters) >>>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:67:80: >>>>>>>>>>>>> E501 >>>>>>>>>>>>> line >>>>>>>>>>>>> too long (88 > 79 characters) >>>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:93:80: >>>>>>>>>>>>> E501 >>>>>>>>>>>>> line >>>>>>>>>>>>> too long (80 > 79 characters) >>>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:94:80: >>>>>>>>>>>>> E501 >>>>>>>>>>>>> line >>>>>>>>>>>>> too long (83 > 79 characters) >>>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:118:80: E501 >>>>>>>>>>>>> >>>>>>>>>>>>> line >>>>>>>>>>>>> too long (81 > 79 characters) >>>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:128:80: E501 >>>>>>>>>>>>> >>>>>>>>>>>>> line >>>>>>>>>>>>> too long (80 > 79 characters) >>>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:129:80: E501 >>>>>>>>>>>>> >>>>>>>>>>>>> line >>>>>>>>>>>>> too long (82 > 79 characters) >>>>>>>>>>>>> ./ipatests/test_integration/test_replica_promotion.py:181:80: E501 >>>>>>>>>>>>> >>>>>>>>>>>>> line >>>>>>>>>>>>> too long (80 > 79 characters) >>>>>>>>>>>> Most of these complaints are unrelated to the current patches. >>>>>>>>>>>> It's better to create a separate patch addressing PEP8 errors. >>>>>>>>>>>> >>>>>>>>>>> I beg for your pardon, those PEP8 errors have been introduced by >>>>>>>>>>> your >>>>>>>>>>> patches. >>>>>>>>>> Fixed >>>>>>>>>> >>>>>>>>>>>>> 7) >>>>>>>>>>>>> Why this must be stored in instance? IMO to have it stored as >>>>>>>>>>>>> local >>>>>>>>>>>>> variable is perfect >>>>>>>>>>>>> TestKRAInstall, TestCAInstall >>>>>>>>>>>>> self.replica1_filename = >>>>>>>>>>>>> tasks.get_replica_filename(replica1) >>>>>>>>>>>>> self.replica2_filename = >>>>>>>>>>>>> tasks.get_replica_filename(replica2) >>>>>>>>>>>> Agree. Fixed >>>>>>>>>>>> >>>>>>>> This patch is missing something. >>>>>>> I am sorry, I forgot to revert my previous change. The correct >>>>>>> patch is >>>>>>> attached >>>>>>> >>>>>> ************* Module ipatests.test_integration.test_replica_promotion >>>>>> ipatests/test_integration/test_replica_promotion.py:15: >>>>>> [E1123(unexpected-keyword-arg), Dummy.install] Unexpected keyword >>>>>> argument 'domain_level' in function call) >>>>>> ipatests/test_integration/test_replica_promotion.py:15: >>>>>> [E1101(no-member), Dummy.install] Class 'Dummy' has no 'domain_level' >>>>>> member) >>>>>> ipatests/test_integration/test_replica_promotion.py:19: >>>>>> [E1101(no-member), Dummy.teardown_method] Module >>>>>> 'ipatests.test_integration.tasks' has no 'uninstall_replica' member) >>>>>> ipatests/test_integration/test_replica_promotion.py:67: >>>>>> [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] >>>>>> Module 'ipatests.test_integration.tasks' has no 'ipa_backup' member) >>>>>> ipatests/test_integration/test_replica_promotion.py:72: >>>>>> [E1101(no-member), TestReplicaPromotionLevel0.test_backup_restore] >>>>>> Module 'ipatests.test_integration.tasks' has no 'ipa_restore' member) >>>>>> ipatests/test_integration/test_replica_promotion.py:120: >>>>>> [E1123(unexpected-keyword-arg), TestCAInstall.install] Unexpected >>>>>> keyword argument 'domain_level' in function call) >>>>>> >>>>> Sorry I forgot to apply patch 17, my bad, I'm continuing with review >>> LGTM, I haven't had time to test it, but if you are sure that test is >>> working, we may push this. >>> >> Is this expected due the bug you mentioned? >> _____ >> __________________________________________________________________________ >> TestReplicaPromotionLevel0.test_kra_install_master >> ________________________________________________________________________________ >> >> >> self = >> > object at 0x7f5071a59e50> >> >> def test_kra_install_master(self): >> result1 = tasks.install_kra(self.master, raiseonerr=False) >>> assert result1.returncode == 0, result1.stderr_text >> E AssertionError: Usage: ipa-kra-install [options] [replica_file] >> E >> E ipa-kra-install: error: Replica file >> /root/ipatests/replica-info.gpg does not exist >> E The ipa-kra-install command failed. See >> /var/log/ipaserver-kra-install.log for more information >> E >> E assert 2 == 0 >> E + where 2 = > 0x7f5071adbd50>.returncode >> IMO the test needs fix, KRA on replica file needs KRA related certificates in replica file [ipa.ipatests.test_integration.host.Host.replica2.ParamikoTransport] RUN ['ipa-kra-install', '-U', '-p', 'Secret123', '/root/ipatests/replica-info.gpg'] [ipa.ipatests.test_integration.host.Host.replica2.cmd27] RUN ['ipa-kra-install', '-U', '-p', 'Secret123', '/root/ipatests/replica-info.gpg'] [ipa.ipatests.test_integration.host.Host.replica2.cmd27] Missing KRA certificates, please create a new replica file. [ipa.ipatests.test_integration.host.Host.replica2.cmd27] The ipa-kra-install command failed. See /var/log/ipaserver-kra-install.log for more information [ipa.ipatests.test_integration.host.Host.replica2.cmd27] Exit code: 1 FAILED >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> self = From akasurde at redhat.com Mon Dec 21 06:47:08 2015 From: akasurde at redhat.com (Abhijeet Kasurde) Date: Mon, 21 Dec 2015 12:17:08 +0530 Subject: [Freeipa-devel] Added kpasswd_server directive in client krb5.conf Message-ID: <5677A06C.3080708@redhat.com> Hi All, Please review patches attached. Thanks, Abhijeet Kasurde -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-akasurde-0001-Added-kpasswd_server-directive-in-client-krb5.conf-ipa-4-2.patch Type: text/x-patch Size: 1723 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-akasurde-0001-Added-kpasswd_server-directive-in-client-krb5.conf.patch Type: text/x-patch Size: 1723 bytes Desc: not available URL: From ofayans at redhat.com Mon Dec 21 10:25:52 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Mon, 21 Dec 2015 11:25:52 +0100 Subject: [Freeipa-devel] [PATCH-0019][Tests]Enabled --auto-reverse by default for master installation In-Reply-To: <5675A9DC.5060009@redhat.com> References: <5672B04E.9040602@redhat.com> <20151217130511.GA16380@mail.corp.redhat.com> <5672B83C.3030700@redhat.com> <56730AD7.4090106@redhat.com> <5673C54D.50805@redhat.com> <5675A9DC.5060009@redhat.com> Message-ID: <5677D3B0.9050106@redhat.com> Hi Martin, On 12/19/2015 08:02 PM, Martin Basti wrote: > > > On 18.12.2015 09:35, Oleg Fayans wrote: >> Hi Petr, >> >> On 12/17/2015 08:19 PM, Petr Spacek wrote: >>> On 17.12.2015 14:27, Oleg Fayans wrote: >>>> Commit message was updated. Thanks! >>>> >>>> On 12/17/2015 02:05 PM, Lukas Slebodnik wrote: >>>>> On (17/12/15 13:53), Oleg Fayans wrote: >>>>>> -- >>>>>> Oleg Fayans >>>>>> Quality Engineer >>>>>> FreeIPA team >>>>>> RedHat. >>>>> >From ed4630140386c1043e36733eb42ec402cc276bee Mon Sep 17 00:00:00 >>>>> 2001 >>>>>> From: Oleg Fayans >>>>>> Date: Thu, 17 Dec 2015 13:50:19 +0100 >>>>>> Subject: [PATCH] Enabled automatic creation of reverse zone during >>>>>> master >>>>>> installation >>>>>> >>>>> The commit message does not contain ticket. >>>>> It is also not explained why this change was done. >>>>> Could you update commit message with verbose explanation? >>>>> >>>>> Proper commit message might prevent removal of this change in future >>>>> based on "git blame" >>> Seems reasonable in general but this should not happen in BRQ or >>> ABCDE labs. >>> Where do you see the problem? >>> >>> Otherwise ACK. >> The problem occurs in local libvirt-powered VMs. Also it may (or may not >> - I did not try it yet) be met in Beaker. >> >> > NACK > > ipa-server-install: error: You cannot specify a --auto-reverse option > without the --setup-dns option Agreed. Updated patch is attached. > -- Oleg Fayans Quality Engineer FreeIPA team RedHat. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ofayans-0019.2-Enabled-automatic-creation-of-reverse-zone.patch Type: text/x-patch Size: 938 bytes Desc: not available URL: From mbasti at redhat.com Mon Dec 21 11:08:04 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 21 Dec 2015 12:08:04 +0100 Subject: [Freeipa-devel] [PATCH 0101] raise more descriptive Backend connection-related exceptions In-Reply-To: <564F255D.8020008@redhat.com> References: <564F255D.8020008@redhat.com> Message-ID: <5677DD94.7040503@redhat.com> On 20.11.2015 14:51, Martin Babinsky wrote: > Fixes https://fedorahosted.org/freeipa/ticket/5473 > > Patch is for master only, I will rebase it for 4-2 if we decide that > it can land there as well. > > > ACK Pushed to master: cd5fa389450d15d5bc696131f73d062000d3558f Pushed to ipa-4-3: f1244a1058aba53b02f513403eb9cf009ffc42b1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbasti at redhat.com Mon Dec 21 11:13:26 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 21 Dec 2015 12:13:26 +0100 Subject: [Freeipa-devel] [PATCH 0114] harden domain level 1 topology connectivity checks In-Reply-To: <566E84FA.2070008@redhat.com> References: <566706D9.2080006@redhat.com> <566E84FA.2070008@redhat.com> Message-ID: <5677DED6.1090509@redhat.com> On 14.12.2015 09:59, Martin Babinsky wrote: > On 12/08/2015 05:35 PM, Martin Babinsky wrote: >> A sort of auxilliary patch which makes topology checks more resistant to >> https://fedorahosted.org/freeipa/ticket/5526 >> >> If required I will open a separate ticket for it though. >> >> >> > Bump for review. > ACK I removed ticket from commit, because ticket has been already closed. If this patch should be backported to IPA 4.3.x, please open a new ticket. Pushed to master: 66de9a6c9b148043e7ee244f2fe6932d0b87aebb From pspacek at redhat.com Mon Dec 21 13:01:09 2015 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 21 Dec 2015 14:01:09 +0100 Subject: [Freeipa-devel] [PATCH 0368] FIX DNS tests In-Reply-To: <565F1D7C.8070407@redhat.com> References: <565F1D7C.8070407@redhat.com> Message-ID: <5677F815.10806@redhat.com> On 2.12.2015 17:34, Martin Basti wrote: > Test needs to be updated due deprecation of dns-resolve command. > > Patch attached ACK -- Petr^2 Spacek From mbasti at redhat.com Mon Dec 21 13:07:43 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 21 Dec 2015 14:07:43 +0100 Subject: [Freeipa-devel] [PATCH 0368] FIX DNS tests In-Reply-To: <5677F815.10806@redhat.com> References: <565F1D7C.8070407@redhat.com> <5677F815.10806@redhat.com> Message-ID: <5677F99F.2020905@redhat.com> On 21.12.2015 14:01, Petr Spacek wrote: > On 2.12.2015 17:34, Martin Basti wrote: >> Test needs to be updated due deprecation of dns-resolve command. >> >> Patch attached > ACK > Pushed to: master: ee51ad0f25d4de2c20c3c666156781e4ff678863 ipa-4-3: 81e40af95de3b4975c006d0936dbd6e65200c5c0 From mbasti at redhat.com Mon Dec 21 13:08:50 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 21 Dec 2015 14:08:50 +0100 Subject: [Freeipa-devel] [patch 0027] ipatests: Roll back the forwarder config after a test case In-Reply-To: <567309F7.3060507@redhat.com> References: <5672BA12.4070608@redhat.com> <567309F7.3060507@redhat.com> Message-ID: <5677F9E2.3020407@redhat.com> On 17.12.2015 20:16, Petr Spacek wrote: > On 17.12.2015 14:35, Milan Kub?k wrote: >> Patch attached. >> >> -- >> Milan Kubik >> >> >> freeipa-mkubik-0027-ipatests-Roll-back-the-forwarder-config-after-a-test.patch >> >> >> From 66db35cf2a315f0a58e002b06f0fcc5bd915a5c5 Mon Sep 17 00:00:00 2001 >> From: Milan Kubik >> Date: Thu, 17 Dec 2015 14:17:22 +0100 >> Subject: [PATCH] ipatests: Roll back the forwarder config after a test case >> >> --- >> ipatests/test_xmlrpc/test_dns_plugin.py | 9 +++++++++ >> 1 file changed, 9 insertions(+) >> >> diff --git a/ipatests/test_xmlrpc/test_dns_plugin.py b/ipatests/test_xmlrpc/test_dns_plugin.py >> index 5f692528eb9a5ae0dc488c663ab43cc47ffd29b3..cba6b001053e6a5c708e9d4359119ad071d70280 100644 >> --- a/ipatests/test_xmlrpc/test_dns_plugin.py >> +++ b/ipatests/test_xmlrpc/test_dns_plugin.py >> @@ -1768,6 +1768,15 @@ class test_dns(Declarative): >> }, >> ), >> >> + dict( >> + desc='Update global DNS settings - rollback', >> + command=('dnsconfig_mod', [], {'idnsforwarders' : None,}), >> + expected={ >> + 'value': None, >> + 'summary': u'Global DNS configuration is empty', >> + 'result': {}, >> + }, >> + ), >> >> dict( >> desc='Try to add invalid allow-query to zone %r' % zone1, > I have seen this code to work on David's VM. ACK. > Pushed to: master: c8a03593628b0ad77f490a05074bfe0c8459722e ipa-4-3: ddd5d408862b411b978c6994a69e3df7b1a43ba4 From mbasti at redhat.com Mon Dec 21 14:02:48 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 21 Dec 2015 15:02:48 +0100 Subject: [Freeipa-devel] [PATCH-0019][Tests]Enabled --auto-reverse by default for master installation In-Reply-To: <5677D3B0.9050106@redhat.com> References: <5672B04E.9040602@redhat.com> <20151217130511.GA16380@mail.corp.redhat.com> <5672B83C.3030700@redhat.com> <56730AD7.4090106@redhat.com> <5673C54D.50805@redhat.com> <5675A9DC.5060009@redhat.com> <5677D3B0.9050106@redhat.com> Message-ID: <56780688.80509@redhat.com> On 21.12.2015 11:25, Oleg Fayans wrote: > Hi Martin, > > On 12/19/2015 08:02 PM, Martin Basti wrote: >> >> On 18.12.2015 09:35, Oleg Fayans wrote: >>> Hi Petr, >>> >>> On 12/17/2015 08:19 PM, Petr Spacek wrote: >>>> On 17.12.2015 14:27, Oleg Fayans wrote: >>>>> Commit message was updated. Thanks! >>>>> >>>>> On 12/17/2015 02:05 PM, Lukas Slebodnik wrote: >>>>>> On (17/12/15 13:53), Oleg Fayans wrote: >>>>>>> -- >>>>>>> Oleg Fayans >>>>>>> Quality Engineer >>>>>>> FreeIPA team >>>>>>> RedHat. >>>>>> >From ed4630140386c1043e36733eb42ec402cc276bee Mon Sep 17 00:00:00 >>>>>> 2001 >>>>>>> From: Oleg Fayans >>>>>>> Date: Thu, 17 Dec 2015 13:50:19 +0100 >>>>>>> Subject: [PATCH] Enabled automatic creation of reverse zone during >>>>>>> master >>>>>>> installation >>>>>>> >>>>>> The commit message does not contain ticket. >>>>>> It is also not explained why this change was done. >>>>>> Could you update commit message with verbose explanation? >>>>>> >>>>>> Proper commit message might prevent removal of this change in future >>>>>> based on "git blame" >>>> Seems reasonable in general but this should not happen in BRQ or >>>> ABCDE labs. >>>> Where do you see the problem? >>>> >>>> Otherwise ACK. >>> The problem occurs in local libvirt-powered VMs. Also it may (or may not >>> - I did not try it yet) be met in Beaker. >>> >>> >> NACK >> >> ipa-server-install: error: You cannot specify a --auto-reverse option >> without the --setup-dns option > Agreed. Updated patch is attached. > ACK Pushed to: master: 36e85b10db7a8671c9116233ab4497ac6410a4a2 ipa-4-3: 9e3e51d354ff84098952b04ed98af0d93ae129e2 From mbasti at redhat.com Mon Dec 21 14:11:50 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 21 Dec 2015 15:11:50 +0100 Subject: [Freeipa-devel] [PATCH 0020][Tests] Added --realm parameter to master installation procedure in integration tests In-Reply-To: <5673C5BD.7090203@redhat.com> References: <5672D495.1010302@redhat.com> <5672D873.3030706@redhat.com> <5673C5BD.7090203@redhat.com> Message-ID: <567808A6.6060904@redhat.com> On 18.12.2015 09:37, Oleg Fayans wrote: > Sorry, my bad :) > > > On 12/17/2015 04:44 PM, Martin Babinsky wrote: >> On 12/17/2015 04:28 PM, Oleg Fayans wrote: >>> >>> >> Oleg, >> >> you sent a vim swap file instead of the patch. >> > > ACK Pushed to: master: b12ba14e3d07b5f2d8dd50245407da029ae7dc54 ipa-4-3: c47a5df724779183b9ab9918d49051743656f3b2 -------------- next part -------------- An HTML attachment was scrubbed... URL: From pspacek at redhat.com Mon Dec 21 14:33:30 2015 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 21 Dec 2015 15:33:30 +0100 Subject: [Freeipa-devel] [PATCH 0072-0081] DNSSEC: ipa-dnskeysyncd: call ods-signer ldap-cleanup on zone removaDNSSEC: ipa-ods-exporter: add ldap-cleanup commanDNSSEC: ipa-dnskeysyncd: Skip zones with old DNSSEC metadata in LDADNSSEC: remove keys purged by OpenDNSSEC from master HSM from LDADNSSEC: logging improvements in ldapkeydb.pDNSSEC: add debug mode to ldapkeydb.pDNSSEC: remove obsolete TODO notDNSSEC: Make sure that current key state in LDAP matches key state in BINDNSSEC: Make sure that current state in OpenDNSSEC matches key state in LDADNSSEC: Improve error reporting from ipa-ods-exporte Message-ID: <56780DBA.60703@redhat.com> Hello, this patch set fixes key rotation in DNSSEC. You can use attached template files for OpenDNSSEC config to shorten time intervals between key rotations. Please let me know if you have any questions, I'm all ears! -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0072-DNSSEC-Improve-error-reporting-from-ipa-ods-exporter.patch Type: text/x-patch Size: 1060 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0073-DNSSEC-Make-sure-that-current-state-in-OpenDNSSEC-ma.patch Type: text/x-patch Size: 8091 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0074-DNSSEC-Make-sure-that-current-key-state-in-LDAP-matc.patch Type: text/x-patch Size: 1800 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0075-DNSSEC-remove-obsolete-TODO-note.patch Type: text/x-patch Size: 1005 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0076-DNSSEC-add-debug-mode-to-ldapkeydb.py.patch Type: text/x-patch Size: 3061 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0077-DNSSEC-logging-improvements-in-ldapkeydb.py.patch Type: text/x-patch Size: 2756 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0078-DNSSEC-remove-keys-purged-by-OpenDNSSEC-from-master-.patch Type: text/x-patch Size: 7533 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0079-DNSSEC-ipa-dnskeysyncd-Skip-zones-with-old-DNSSEC-me.patch Type: text/x-patch Size: 4522 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0080-DNSSEC-ipa-ods-exporter-add-ldap-cleanup-command.patch Type: text/x-patch Size: 4806 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0081-DNSSEC-ipa-dnskeysyncd-call-ods-signer-ldap-cleanup-.patch Type: text/x-patch Size: 1444 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: opendnssec_conf.template Type: text/xml Size: 1090 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: opendnssec_kasp.template Type: text/xml Size: 1527 bytes Desc: not available URL: From mbasti at redhat.com Mon Dec 21 14:45:25 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 21 Dec 2015 15:45:25 +0100 Subject: [Freeipa-devel] [PATCH 0072-0081] DNSSEC: ipa-dnskeysyncd: call ods-signer ldap-cleanup on zone removaDNSSEC: ipa-ods-exporter: add ldap-cleanup commanDNSSEC: ipa-dnskeysyncd: Skip zones with old DNSSEC metadata in LDADNSSEC: remove keys purged by OpenDNSSEC from master HSM from LDADNSSEC: logging improvements in ldapkeydb.pDNSSEC: add debug mode to ldapkeydb.pDNSSEC: remove obsolete TODO notDNSSEC: Make sure that current key state in LDAP matches key state in BINDNSSEC: Make sure that current state in OpenDNSSEC matches key state in LDADNSSEC: Improve error reporting from ipa-ods-exporte In-Reply-To: <56780DBA.60703@redhat.com> References: <56780DBA.60703@redhat.com> Message-ID: <56781085.4070106@redhat.com> On 21.12.2015 15:33, Petr Spacek wrote: > Hello, > > this patch set fixes key rotation in DNSSEC. > > You can use attached template files for OpenDNSSEC config to shorten time > intervals between key rotations. > > Please let me know if you have any questions, I'm all ears! > Please fix whitespace error: Applying: DNSSEC: logging improvements in ldapkeydb.py /home/mbasti/work/freeipa-devel/.git/rebase-apply/patch:14: trailing whitespace. warning: 1 line adds whitespace errors. From mbasti at redhat.com Mon Dec 21 15:39:47 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 21 Dec 2015 16:39:47 +0100 Subject: [Freeipa-devel] [PATCH-0019][Tests]Enabled --auto-reverse by default for master installation In-Reply-To: <56780688.80509@redhat.com> References: <5672B04E.9040602@redhat.com> <20151217130511.GA16380@mail.corp.redhat.com> <5672B83C.3030700@redhat.com> <56730AD7.4090106@redhat.com> <5673C54D.50805@redhat.com> <5675A9DC.5060009@redhat.com> <5677D3B0.9050106@redhat.com> <56780688.80509@redhat.com> Message-ID: <56781D43.8090504@redhat.com> On 21.12.2015 15:02, Martin Basti wrote: > > > On 21.12.2015 11:25, Oleg Fayans wrote: >> Hi Martin, >> >> On 12/19/2015 08:02 PM, Martin Basti wrote: >>> >>> On 18.12.2015 09:35, Oleg Fayans wrote: >>>> Hi Petr, >>>> >>>> On 12/17/2015 08:19 PM, Petr Spacek wrote: >>>>> On 17.12.2015 14:27, Oleg Fayans wrote: >>>>>> Commit message was updated. Thanks! >>>>>> >>>>>> On 12/17/2015 02:05 PM, Lukas Slebodnik wrote: >>>>>>> On (17/12/15 13:53), Oleg Fayans wrote: >>>>>>>> -- >>>>>>>> Oleg Fayans >>>>>>>> Quality Engineer >>>>>>>> FreeIPA team >>>>>>>> RedHat. >>>>>>> >From ed4630140386c1043e36733eb42ec402cc276bee Mon Sep 17 00:00:00 >>>>>>> 2001 >>>>>>>> From: Oleg Fayans >>>>>>>> Date: Thu, 17 Dec 2015 13:50:19 +0100 >>>>>>>> Subject: [PATCH] Enabled automatic creation of reverse zone during >>>>>>>> master >>>>>>>> installation >>>>>>>> >>>>>>> The commit message does not contain ticket. >>>>>>> It is also not explained why this change was done. >>>>>>> Could you update commit message with verbose explanation? >>>>>>> >>>>>>> Proper commit message might prevent removal of this change in >>>>>>> future >>>>>>> based on "git blame" >>>>> Seems reasonable in general but this should not happen in BRQ or >>>>> ABCDE labs. >>>>> Where do you see the problem? >>>>> >>>>> Otherwise ACK. >>>> The problem occurs in local libvirt-powered VMs. Also it may (or >>>> may not >>>> - I did not try it yet) be met in Beaker. >>>> >>>> >>> NACK >>> >>> ipa-server-install: error: You cannot specify a --auto-reverse option >>> without the --setup-dns option >> Agreed. Updated patch is attached. >> > ACK > > Pushed to: > master: 36e85b10db7a8671c9116233ab4497ac6410a4a2 > ipa-4-3: 9e3e51d354ff84098952b04ed98af0d93ae129e2 > Question: shouldn't tasks.dns_install be called with --auto-reverse option too? Martin From mbasti at redhat.com Mon Dec 21 17:38:58 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 21 Dec 2015 18:38:58 +0100 Subject: [Freeipa-devel] [PATCH 0075-0076] Fix installer regression In-Reply-To: <56741470.9040609@redhat.com> References: <567167C7.8060605@redhat.com> <56729920.3040500@redhat.com> <5672A9F4.40205@redhat.com> <5672AE2E.2020701@redhat.com> <567402D4.8010605@redhat.com> <567405E9.2070909@redhat.com> <56741470.9040609@redhat.com> Message-ID: <56783932.3090505@redhat.com> On 18.12.2015 15:13, Jan Cholasta wrote: > On 18.12.2015 14:11, David Kupka wrote: >> On 18/12/15 13:57, David Kupka wrote: >>> On 17/12/15 13:44, Jan Cholasta wrote: >>>> On 17.12.2015 13:26, David Kupka wrote: >>>>> On 17/12/15 12:14, Petr Vobornik wrote: >>>>>> On 12/16/2015 02:31 PM, David Kupka wrote: >>>>>>> https://www.redhat.com/archives/freeipa-users/2015-December/msg00203.html >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> please link the patch to >>>>>> https://fedorahosted.org/freeipa/ticket/5556 >>>>> >>>>> Updated patches attached. >>>> >>>> NACK, this is the correct procedure for the __getattr__(), sorry for >>>> confusing you earlier: >>>> >>>> def __getattr__(self, name): >>>> for owner_cls, knob_name in self.knobs(): >>>> if knob_name == name: >>>> break >>>> else: >>>> raise AttributeError(name) >>>> >>>> for component in self.__components: >>>> if isinstance(component, owner_cls): >>>> break >>>> else: >>>> raise AttributeError(name) >>>> >>>> return getattr(component, name) >>>> >>>> Honza >>>> >>> Updated patches attached. >>> >> Completing the update. > > Thanks, ACK. > > Not pushed yet because ipa-4-3 is cursed. > > Honza > Pushed to master: f0703d3c2a399012fa4cae0b856c08ff18b42463 ipa-4-3: 2b3a0a4519b409c4c363afcb61da77ad9b9a1436 ipa-4-2: d655b949d3bd8f5d192605acd579314e82492b86 From mbasti at redhat.com Mon Dec 21 17:56:33 2015 From: mbasti at redhat.com (Martin Basti) Date: Mon, 21 Dec 2015 18:56:33 +0100 Subject: [Freeipa-devel] [PATCH 0072-0081] DNSSEC: ipa-dnskeysyncd: call ods-signer ldap-cleanup on zone removaDNSSEC: ipa-ods-exporter: add ldap-cleanup commanDNSSEC: ipa-dnskeysyncd: Skip zones with old DNSSEC metadata in LDADNSSEC: remove keys purged by OpenDNSSEC from master HSM from LDADNSSEC: logging improvements in ldapkeydb.pDNSSEC: add debug mode to ldapkeydb.pDNSSEC: remove obsolete TODO notDNSSEC: Make sure that current key state in LDAP matches key state in BINDNSSEC: Make sure that current state in OpenDNSSEC matches key state in LDADNSSEC: Improve error reporting from ipa-ods-exporte In-Reply-To: <56781085.4070106@redhat.com> References: <56780DBA.60703@redhat.com> <56781085.4070106@redhat.com> Message-ID: <56783D51.3070005@redhat.com> On 21.12.2015 15:45, Martin Basti wrote: > > > On 21.12.2015 15:33, Petr Spacek wrote: >> Hello, >> >> this patch set fixes key rotation in DNSSEC. >> >> You can use attached template files for OpenDNSSEC config to shorten >> time >> intervals between key rotations. >> >> Please let me know if you have any questions, I'm all ears! >> > Please fix whitespace error: > > Applying: DNSSEC: logging improvements in ldapkeydb.py > /home/mbasti/work/freeipa-devel/.git/rebase-apply/patch:14: trailing > whitespace. > > warning: 1 line adds whitespace errors. > *) DNSSEC: ipa-dnskeysyncd: call ods-signer ldap-cleanup on zone removal Is is safe to do not use try - except with ipatuil.run()? What if ods-signer command failed? *) DNSSEC: Improve error reporting from ipa-ods-exporter IMO log.exception(ex) is enough, do we need to add traceback to msg? *) DNSSEC: Make sure that current state in OpenDNSSEC matches key state in LDAP I think this is okay because we want to use KSK instantly, but just to be sure, is Publish->Activate okay? + bind_times['idnsSecKeyActivate'] = ods_times['idnsSecKeyPublish'] Just to be sure how this will be handle during KSK key rotation? *) DNSSEC: Make sure that current key state in LDAP matches key state in BIND LGTM *) DNSSEC: remove obsolete TODO note ACK *) DNSSEC: add debug mode to ldapkeydb.py A) You can remove __str__ method, python will use __repr__ as defaul B) for attr in ['ipaPrivateKey', 'ipaPublicKey', 'ipk11publickeyinfo']: Do we need to sanitize *public*Key and publicKeyinfo? C) in odsmgr.py is used ipa_log_manager, can we use the same for consistency? D) Do we need logging there, everything is printed via print except debug info about connecting, can you just redirect it to stderr, and usable data leave in stdout? *) DNSSEC: logging improvements in ldapkeydb.py IMO commit message should be: ".... in ipa-ods-exporter" Otherwise LGTM *) DNSSEC: remove keys purged by OpenDNSSEC from master HSM from LDAP A) coding style: please use (), instead of "\" assert set(pubkeys_local) == set(privkeys_local), ( "IDs of private and public keys for DNS zones in local HSM does " "not match to key pairs: %s vs. %s" % (hex_set(pubkeys_local), hex_set(privkeys_local)) ) B) coding style assert not matched_already, ( "key %s is in more than one keyset" % hexlify(keyid) ) C) schedule_key_deletion() how about case when keyid is not in any keyset, then keyid will not be replaced by object and it blow up somewhere else D) +class KeyDeleter(object): I would like to have a check there which blows up nicely if _update_key() is called twice on the same object. With current implementation you will get NoneType has no delete_entry method. E) I somehow does not like the placeholder object. Could we just extend Key object with attribute "to_be_deleted" or something similar, and if this attribute is set to True, Key._update_key() can remove, instead of creation a new object. Key.prepare_deletion() can set the value "to_be_deleted" to True. *) DNSSEC: ipa-dnskeysyncd: Skip zones with old DNSSEC metadata in LDAP How often is keysyncer initialized? Might happen the case where one of dnssec_zones has been disabled and keysyncer is not aware of this change? You may want to use DNSName from ipapython.dnsutil instead of pure dns.name *) DNSSEC: ipa-ods-exporter: add ldap-cleanup command LGTM Martin^2 From jcholast at redhat.com Tue Dec 22 07:06:25 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 22 Dec 2015 08:06:25 +0100 Subject: [Freeipa-devel] [PATCH 0379 - 0393] Winter code cleanup In-Reply-To: <5672B631.7030709@redhat.com> References: <56720BCC.1050504@redhat.com> <56725C34.5060400@redhat.com> <5672B631.7030709@redhat.com> Message-ID: <5678F671.8080707@redhat.com> On 17.12.2015 14:18, Martin Basti wrote: > > > On 17.12.2015 07:54, Jan Cholasta wrote: >> Hi, >> >> On 17.12.2015 02:11, Martin Basti wrote: >>> Hello, >>> following patches cleanup the code and add checks to pylint to avoid the >>> mess again >>> >>> Patches: 379-381: >>> the most important patches, they cleanup imports >>> >>> Patch 382: >>> enables various pylint checks, we may reduce the list of check if >>> needed. >> >> Would it be possible to turn this into a blacklist of disabled >> warnings rather than a whitelist of enabled warnings? >> >>> >>> Patches 383 - 393: >>> remove minor issues from code, and enable particular checks >>> Feel free to nack patches/checks that should not be there. >>> >>> Please apply patches in order. >>> >>> Martin^2 >>> >>> >> >> Honza >> > Updated patches attached. Patch 379: ACK Patch 380: 1) This patch needs to be rebased. 2) make-lint is reporting this to me: ************* Module ipatests.test_cmdline.test_ipagetkeytab ipatests/test_cmdline/test_ipagetkeytab.py:30: [W0611(unused-import), ] Unused import nose) ipatests/test_cmdline/test_ipagetkeytab.py:34: [W0611(unused-import), ] Unused DN imported from ipapython.dn) ************* Module ipatests.test_cmdline.test_help ipatests/test_cmdline/test_help.py:21: [W0611(unused-import), ] Unused import contextlib) ipatests/test_cmdline/test_help.py:23: [W0611(unused-import), ] Unused assert_raises imported from nose.tools) ************* Module ipatests.test_cmdline.test_cli ipatests/test_cmdline/test_cli.py:11: [W0611(unused-import), ] Unused API_VERSION imported from ipapython.version) ************* Module ipaplatform.services ipaplatform/services.py:59: [W0611(unused-import), ] Unused timedate_services imported from ipaplatform.redhat.services) ************* Module ipaplatform.rhel.services ipaplatform/rhel/services.py:59: [W0611(unused-import), ] Unused timedate_services imported from ipaplatform.redhat.services) ************* Module ipaplatform.redhat.services ipaplatform/redhat/services.py:306: [W0611(unused-import), ] Unused timedate_services imported from ipaplatform.base.services) ************* Module ipaplatform.fedora.services ipaplatform/fedora/services.py:59: [W0611(unused-import), ] Unused timedate_services imported from ipaplatform.redhat.services) ************* Module ipalib.setup ipalib/setup.py:28: [W0611(unused-import), ] Unused import distutils.sysconfig) Note that the "from ipaplatform.${PLATFORM}.services import timedate_services" in services.py should be rewritten to "timedate_services = ${PLATFORM}_services.timedate_services" to fix this. Patch 381: ACK Patch 382: Nitpick: according to , the order of message categories is F, E, W, C, R from the most severe to the least severe (it does not mention I, but I think it should be last, after R), IMO we should keep this order here as well for clarity. (Don't worry, doing this should not require rebasing the following patches.) Patch 383 - 387: ACK Patch 388: IMO it would be better to rewrite this: [(t.id, t.options) for t in doc.getKeyPackages()] # pylint: disable=expression-not-assigned into this: for t in doc.getKeyPackages(): t._PSKCKeyPackage__process() rather than disabling the check. Patch 389: All this patch does is that it enables a check globally and then disables it everywhere locally. IMO the patch should be retired for now, or make-lint should be taught to automatically skip the check inside TestCase.assertRaises() context. Patch 390: ACK Patch 391: default_from uses argument names of the specified function as param names from which to create the default. These changes break default_from: - default_from=lambda name_from_ip: _reverse_zone_name(name_from_ip), + default_from=_reverse_zone_name, - default_from=lambda idnsname: default_zone_update_policy(idnsname), + default_from=default_zone_update_policy, This change adds dependency on non-existent param: - default_from=lambda: krb_utils.get_principal(), + default_from=krb_utils.get_principal, The result of this check is misleading for default_from, so IMO the patch should be retired for now, or make-lint should be taught to automatically disable the check for the default_from argument in param definitions. Patch 392: I think the existing occurences of exec()/eval() should be removed before this check can be enabled. Patch 393: ACK I would like to see a patch which enables the useless-suppression info message, so that we can catch dangling "# pylint: disable=" comments (there are some in the code). (Also, it would be nice to finally rewrite make-lint to pylint config file + plugins.) -- Jan Cholasta From pspacek at redhat.com Tue Dec 22 08:45:08 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 22 Dec 2015 09:45:08 +0100 Subject: [Freeipa-devel] [PATCH-0019][Tests]Enabled --auto-reverse by default for master installation In-Reply-To: <56781D43.8090504@redhat.com> References: <5672B04E.9040602@redhat.com> <20151217130511.GA16380@mail.corp.redhat.com> <5672B83C.3030700@redhat.com> <56730AD7.4090106@redhat.com> <5673C54D.50805@redhat.com> <5675A9DC.5060009@redhat.com> <5677D3B0.9050106@redhat.com> <56780688.80509@redhat.com> <56781D43.8090504@redhat.com> Message-ID: <56790D94.9040006@redhat.com> On 21.12.2015 16:39, Martin Basti wrote: > > > On 21.12.2015 15:02, Martin Basti wrote: >> >> >> On 21.12.2015 11:25, Oleg Fayans wrote: >>> Hi Martin, >>> >>> On 12/19/2015 08:02 PM, Martin Basti wrote: >>>> >>>> On 18.12.2015 09:35, Oleg Fayans wrote: >>>>> Hi Petr, >>>>> >>>>> On 12/17/2015 08:19 PM, Petr Spacek wrote: >>>>>> On 17.12.2015 14:27, Oleg Fayans wrote: >>>>>>> Commit message was updated. Thanks! >>>>>>> >>>>>>> On 12/17/2015 02:05 PM, Lukas Slebodnik wrote: >>>>>>>> On (17/12/15 13:53), Oleg Fayans wrote: >>>>>>>>> -- >>>>>>>>> Oleg Fayans >>>>>>>>> Quality Engineer >>>>>>>>> FreeIPA team >>>>>>>>> RedHat. >>>>>>>> >From ed4630140386c1043e36733eb42ec402cc276bee Mon Sep 17 00:00:00 >>>>>>>> 2001 >>>>>>>>> From: Oleg Fayans >>>>>>>>> Date: Thu, 17 Dec 2015 13:50:19 +0100 >>>>>>>>> Subject: [PATCH] Enabled automatic creation of reverse zone during >>>>>>>>> master >>>>>>>>> installation >>>>>>>>> >>>>>>>> The commit message does not contain ticket. >>>>>>>> It is also not explained why this change was done. >>>>>>>> Could you update commit message with verbose explanation? >>>>>>>> >>>>>>>> Proper commit message might prevent removal of this change in future >>>>>>>> based on "git blame" >>>>>> Seems reasonable in general but this should not happen in BRQ or >>>>>> ABCDE labs. >>>>>> Where do you see the problem? >>>>>> >>>>>> Otherwise ACK. >>>>> The problem occurs in local libvirt-powered VMs. Also it may (or may not >>>>> - I did not try it yet) be met in Beaker. >>>>> >>>>> >>>> NACK >>>> >>>> ipa-server-install: error: You cannot specify a --auto-reverse option >>>> without the --setup-dns option >>> Agreed. Updated patch is attached. >>> >> ACK >> >> Pushed to: >> master: 36e85b10db7a8671c9116233ab4497ac6410a4a2 >> ipa-4-3: 9e3e51d354ff84098952b04ed98af0d93ae129e2 >> > Question: shouldn't tasks.dns_install be called with --auto-reverse option too? > Martin Yeah, it probably should. -- Petr^2 Spacek From ofayans at redhat.com Tue Dec 22 10:33:36 2015 From: ofayans at redhat.com (Oleg Fayans) Date: Tue, 22 Dec 2015 11:33:36 +0100 Subject: [Freeipa-devel] [PATCH-0019][Tests]Enabled --auto-reverse by default for master installation In-Reply-To: <56781D43.8090504@redhat.com> References: <5672B04E.9040602@redhat.com> <20151217130511.GA16380@mail.corp.redhat.com> <5672B83C.3030700@redhat.com> <56730AD7.4090106@redhat.com> <5673C54D.50805@redhat.com> <5675A9DC.5060009@redhat.com> <5677D3B0.9050106@redhat.com> <56780688.80509@redhat.com> <56781D43.8090504@redhat.com> Message-ID: <56792700.3020603@redhat.com> On 12/21/2015 04:39 PM, Martin Basti wrote: > > > On 21.12.2015 15:02, Martin Basti wrote: >> >> >> On 21.12.2015 11:25, Oleg Fayans wrote: >>> Hi Martin, >>> >>> On 12/19/2015 08:02 PM, Martin Basti wrote: >>>> >>>> On 18.12.2015 09:35, Oleg Fayans wrote: >>>>> Hi Petr, >>>>> >>>>> On 12/17/2015 08:19 PM, Petr Spacek wrote: >>>>>> On 17.12.2015 14:27, Oleg Fayans wrote: >>>>>>> Commit message was updated. Thanks! >>>>>>> >>>>>>> On 12/17/2015 02:05 PM, Lukas Slebodnik wrote: >>>>>>>> On (17/12/15 13:53), Oleg Fayans wrote: >>>>>>>>> -- >>>>>>>>> Oleg Fayans >>>>>>>>> Quality Engineer >>>>>>>>> FreeIPA team >>>>>>>>> RedHat. >>>>>>>> >From ed4630140386c1043e36733eb42ec402cc276bee Mon Sep 17 00:00:00 >>>>>>>> 2001 >>>>>>>>> From: Oleg Fayans >>>>>>>>> Date: Thu, 17 Dec 2015 13:50:19 +0100 >>>>>>>>> Subject: [PATCH] Enabled automatic creation of reverse zone during >>>>>>>>> master >>>>>>>>> installation >>>>>>>>> >>>>>>>> The commit message does not contain ticket. >>>>>>>> It is also not explained why this change was done. >>>>>>>> Could you update commit message with verbose explanation? >>>>>>>> >>>>>>>> Proper commit message might prevent removal of this change in >>>>>>>> future >>>>>>>> based on "git blame" >>>>>> Seems reasonable in general but this should not happen in BRQ or >>>>>> ABCDE labs. >>>>>> Where do you see the problem? >>>>>> >>>>>> Otherwise ACK. >>>>> The problem occurs in local libvirt-powered VMs. Also it may (or >>>>> may not >>>>> - I did not try it yet) be met in Beaker. >>>>> >>>>> >>>> NACK >>>> >>>> ipa-server-install: error: You cannot specify a --auto-reverse option >>>> without the --setup-dns option >>> Agreed. Updated patch is attached. >>> >> ACK >> >> Pushed to: >> master: 36e85b10db7a8671c9116233ab4497ac6410a4a2 >> ipa-4-3: 9e3e51d354ff84098952b04ed98af0d93ae129e2 >> > Question: shouldn't tasks.dns_install be called with --auto-reverse > option too? > Martin [11:31:22]ofayans at ofayans:~/tmp/freeipa]$ grep "def dns_install" ipatests/test_integration/tasks.py [11:32:01]ofayans at ofayans:~/tmp/freeipa]$ [11:32:01]ofayans at ofayans:~/tmp/freeipa]$ echo $? 1 [11:32:32]ofayans at ofayans:~/tmp/freeipa]$ Maybe you meant something else? -- Oleg Fayans Quality Engineer FreeIPA team RedHat. From fskola at redhat.com Tue Dec 22 10:56:15 2015 From: fskola at redhat.com (Filip Skola) Date: Tue, 22 Dec 2015 05:56:15 -0500 (EST) Subject: [Freeipa-devel] [PATCH 0005] Refactor test_nesting, create HostGroupTracker In-Reply-To: <9409745.1326095.1450781617976.JavaMail.zimbra@redhat.com> Message-ID: <2134068455.1326509.1450781775764.JavaMail.zimbra@redhat.com> Hi, another patch from refactoring-test_xmlrpc series. Filip -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-fskola-0005-Refactor-test_nesting-create-HostGroupTracker.patch Type: text/x-patch Size: 44377 bytes Desc: not available URL: From fskola at redhat.com Tue Dec 22 10:57:50 2015 From: fskola at redhat.com (Filip Skola) Date: Tue, 22 Dec 2015 05:57:50 -0500 (EST) Subject: [Freeipa-devel] [PATCH 0006] Refactor test_hostgroup_plugin In-Reply-To: <1693360345.1327020.1450781862487.JavaMail.zimbra@redhat.com> Message-ID: <991418926.1327028.1450781870579.JavaMail.zimbra@redhat.com> And also sending refactored hostgroup plugin test. F -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-fskola-0006-Refactor-test_hostgroup_plugin.patch Type: text/x-patch Size: 14196 bytes Desc: not available URL: From mbasti at redhat.com Tue Dec 22 11:15:42 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 22 Dec 2015 12:15:42 +0100 Subject: [Freeipa-devel] [PATCH-0019][Tests]Enabled --auto-reverse by default for master installation In-Reply-To: <56792700.3020603@redhat.com> References: <5672B04E.9040602@redhat.com> <20151217130511.GA16380@mail.corp.redhat.com> <5672B83C.3030700@redhat.com> <56730AD7.4090106@redhat.com> <5673C54D.50805@redhat.com> <5675A9DC.5060009@redhat.com> <5677D3B0.9050106@redhat.com> <56780688.80509@redhat.com> <56781D43.8090504@redhat.com> <56792700.3020603@redhat.com> Message-ID: <567930DE.5020809@redhat.com> On 22.12.2015 11:33, Oleg Fayans wrote: > > On 12/21/2015 04:39 PM, Martin Basti wrote: >> >> On 21.12.2015 15:02, Martin Basti wrote: >>> >>> On 21.12.2015 11:25, Oleg Fayans wrote: >>>> Hi Martin, >>>> >>>> On 12/19/2015 08:02 PM, Martin Basti wrote: >>>>> On 18.12.2015 09:35, Oleg Fayans wrote: >>>>>> Hi Petr, >>>>>> >>>>>> On 12/17/2015 08:19 PM, Petr Spacek wrote: >>>>>>> On 17.12.2015 14:27, Oleg Fayans wrote: >>>>>>>> Commit message was updated. Thanks! >>>>>>>> >>>>>>>> On 12/17/2015 02:05 PM, Lukas Slebodnik wrote: >>>>>>>>> On (17/12/15 13:53), Oleg Fayans wrote: >>>>>>>>>> -- >>>>>>>>>> Oleg Fayans >>>>>>>>>> Quality Engineer >>>>>>>>>> FreeIPA team >>>>>>>>>> RedHat. >>>>>>>>> >From ed4630140386c1043e36733eb42ec402cc276bee Mon Sep 17 00:00:00 >>>>>>>>> 2001 >>>>>>>>>> From: Oleg Fayans >>>>>>>>>> Date: Thu, 17 Dec 2015 13:50:19 +0100 >>>>>>>>>> Subject: [PATCH] Enabled automatic creation of reverse zone during >>>>>>>>>> master >>>>>>>>>> installation >>>>>>>>>> >>>>>>>>> The commit message does not contain ticket. >>>>>>>>> It is also not explained why this change was done. >>>>>>>>> Could you update commit message with verbose explanation? >>>>>>>>> >>>>>>>>> Proper commit message might prevent removal of this change in >>>>>>>>> future >>>>>>>>> based on "git blame" >>>>>>> Seems reasonable in general but this should not happen in BRQ or >>>>>>> ABCDE labs. >>>>>>> Where do you see the problem? >>>>>>> >>>>>>> Otherwise ACK. >>>>>> The problem occurs in local libvirt-powered VMs. Also it may (or >>>>>> may not >>>>>> - I did not try it yet) be met in Beaker. >>>>>> >>>>>> >>>>> NACK >>>>> >>>>> ipa-server-install: error: You cannot specify a --auto-reverse option >>>>> without the --setup-dns option >>>> Agreed. Updated patch is attached. >>>> >>> ACK >>> >>> Pushed to: >>> master: 36e85b10db7a8671c9116233ab4497ac6410a4a2 >>> ipa-4-3: 9e3e51d354ff84098952b04ed98af0d93ae129e2 >>> >> Question: shouldn't tasks.dns_install be called with --auto-reverse >> option too? >> Martin > [11:31:22]ofayans at ofayans:~/tmp/freeipa]$ grep "def dns_install" > ipatests/test_integration/tasks.py > [11:32:01]ofayans at ofayans:~/tmp/freeipa]$ > [11:32:01]ofayans at ofayans:~/tmp/freeipa]$ echo $? > 1 > [11:32:32]ofayans at ofayans:~/tmp/freeipa]$ > > Maybe you meant something else? > Yes, I meant function dns_install in ipatests/test_integration/tasks.py module From dkupka at redhat.com Tue Dec 22 12:14:56 2015 From: dkupka at redhat.com (David Kupka) Date: Tue, 22 Dec 2015 13:14:56 +0100 Subject: [Freeipa-devel] [PATCH 0077] ipa-dns-install: Do not check for zone overlap when DNS, installed. In-Reply-To: <56741AA8.5060807@redhat.com> References: <5673DF56.8060906@redhat.com> <5673E832.3020609@redhat.com> <56741AA8.5060807@redhat.com> Message-ID: <56793EC0.3010107@redhat.com> On 18/12/15 15:39, Petr Vobornik wrote: > On 12/18/2015 12:04 PM, Petr Vobornik wrote: >> On 12/18/2015 11:26 AM, David Kupka wrote: >>> Standalone DNS installer always performed overlap check effectively >>> preventing installation on replica when other DNS instance was already >>> installed in topology. >>> >>> >> >> I don't like the position of api argument in the install_check. It is >> not consistent with install_check in KRA plus it's between two related >> options: standalone and replica. >> >> Is there a reason behind it which I don't see? > > NACK, the API call is incorrect. > > It should be api.Command['dns_is_enabled']()['result'] or > api.Command.dns_is_enabled()['result'] > > > ipa-dns-install --forwarder=$DNS_FORWARDER > > The log file for this installation can be found in > /var/log/ipaserver-install.log > Unexpected error - see /var/log/ipaserver-install.log for details: > TypeError: __call__() takes exactly 1 argument (2 given) > > > log file: > > 2015-12-18T14:33:30Z DEBUG File > "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", > line 736, in run_script > return_value = main_function() > > File "/sbin/ipa-dns-install", line 137, in main > dns_installer.install_check(True, api, False, options, > hostname=api.env.host) > > File "/usr/lib/python2.7/site-packages/ipaserver/install/dns.py", > line 113, in install_check > already_enabled = api.Command('dns_is_enabled')['result'] > > 2015-12-18T14:33:30Z DEBUG The ipa-dns-install command failed, > exception: TypeError: __call__() takes exactly 1 argument (2 given) > Thanks for catch. Sometimes I mess up with patch files. Fixed patch attached. -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0077.1-ipa-dns-install-Do-not-check-for-zone-overlap-when-D.patch Type: text/x-patch Size: 4706 bytes Desc: not available URL: From mbasti at redhat.com Tue Dec 22 12:43:57 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 22 Dec 2015 13:43:57 +0100 Subject: [Freeipa-devel] [PATCH 0394, 0379 - 0393] Winter code cleanup In-Reply-To: <5678F671.8080707@redhat.com> References: <56720BCC.1050504@redhat.com> <56725C34.5060400@redhat.com> <5672B631.7030709@redhat.com> <5678F671.8080707@redhat.com> Message-ID: <5679458D.9090207@redhat.com> On 22.12.2015 08:06, Jan Cholasta wrote: > On 17.12.2015 14:18, Martin Basti wrote: >> >> >> On 17.12.2015 07:54, Jan Cholasta wrote: >>> Hi, >>> >>> On 17.12.2015 02:11, Martin Basti wrote: >>>> Hello, >>>> following patches cleanup the code and add checks to pylint to >>>> avoid the >>>> mess again >>>> >>>> Patches: 379-381: >>>> the most important patches, they cleanup imports >>>> >>>> Patch 382: >>>> enables various pylint checks, we may reduce the list of check if >>>> needed. >>> >>> Would it be possible to turn this into a blacklist of disabled >>> warnings rather than a whitelist of enabled warnings? >>> >>>> >>>> Patches 383 - 393: >>>> remove minor issues from code, and enable particular checks >>>> Feel free to nack patches/checks that should not be there. >>>> >>>> Please apply patches in order. >>>> >>>> Martin^2 >>>> >>>> >>> >>> Honza >>> >> Updated patches attached. > > Patch 379: ACK > > > Patch 380: > > 1) This patch needs to be rebased. > > > 2) make-lint is reporting this to me: > > ************* Module ipatests.test_cmdline.test_ipagetkeytab > ipatests/test_cmdline/test_ipagetkeytab.py:30: [W0611(unused-import), > ] Unused import nose) > ipatests/test_cmdline/test_ipagetkeytab.py:34: [W0611(unused-import), > ] Unused DN imported from ipapython.dn) > ************* Module ipatests.test_cmdline.test_help > ipatests/test_cmdline/test_help.py:21: [W0611(unused-import), ] Unused > import contextlib) > ipatests/test_cmdline/test_help.py:23: [W0611(unused-import), ] Unused > assert_raises imported from nose.tools) > ************* Module ipatests.test_cmdline.test_cli > ipatests/test_cmdline/test_cli.py:11: [W0611(unused-import), ] Unused > API_VERSION imported from ipapython.version) > ************* Module ipaplatform.services > ipaplatform/services.py:59: [W0611(unused-import), ] Unused > timedate_services imported from ipaplatform.redhat.services) > ************* Module ipaplatform.rhel.services > ipaplatform/rhel/services.py:59: [W0611(unused-import), ] Unused > timedate_services imported from ipaplatform.redhat.services) > ************* Module ipaplatform.redhat.services > ipaplatform/redhat/services.py:306: [W0611(unused-import), ] Unused > timedate_services imported from ipaplatform.base.services) > ************* Module ipaplatform.fedora.services > ipaplatform/fedora/services.py:59: [W0611(unused-import), ] Unused > timedate_services imported from ipaplatform.redhat.services) > ************* Module ipalib.setup > ipalib/setup.py:28: [W0611(unused-import), ] Unused import > distutils.sysconfig) > > Note that the "from ipaplatform.${PLATFORM}.services import > timedate_services" in services.py should be rewritten to > "timedate_services = ${PLATFORM}_services.timedate_services" to fix this. > > > Patch 381: ACK > > > Patch 382: > > Nitpick: according to , the order > of message categories is F, E, W, C, R from the most severe to the > least severe (it does not mention I, but I think it should be last, > after R), IMO we should keep this order here as well for clarity. > (Don't worry, doing this should not require rebasing the following > patches.) > > > Patch 383 - 387: ACK > > > Patch 388: > > IMO it would be better to rewrite this: > > [(t.id, t.options) for t in doc.getKeyPackages()] # > pylint: disable=expression-not-assigned > > into this: > > for t in doc.getKeyPackages(): > t._PSKCKeyPackage__process() > > rather than disabling the check. > > > Patch 389: > > All this patch does is that it enables a check globally and then > disables it everywhere locally. > > IMO the patch should be retired for now, or make-lint should be taught > to automatically skip the check inside TestCase.assertRaises() context. > > > Patch 390: ACK > > > Patch 391: > > default_from uses argument names of the specified function as param > names from which to create the default. > > These changes break default_from: > > - default_from=lambda name_from_ip: > _reverse_zone_name(name_from_ip), > + default_from=_reverse_zone_name, > > - default_from=lambda idnsname: > default_zone_update_policy(idnsname), > + default_from=default_zone_update_policy, > > This change adds dependency on non-existent param: > > - default_from=lambda: krb_utils.get_principal(), > + default_from=krb_utils.get_principal, > > The result of this check is misleading for default_from, so IMO the > patch should be retired for now, or make-lint should be taught to > automatically disable the check for the default_from argument in param > definitions. > > > Patch 392: > > I think the existing occurences of exec()/eval() should be removed > before this check can be enabled. > > > Patch 393: ACK > > > I would like to see a patch which enables the useless-suppression info > message, so that we can catch dangling "# pylint: disable=" comments > (there are some in the code). > > (Also, it would be nice to finally rewrite make-lint to pylint config > file + plugins.) > Updated patches attached, please apply patch 394 first. Patches 389, 391, 392 have been removed, issues will be addressed later. useless-suppression, unbalanced-tuple-unpacking, unpacking-non-sequence, and python3 checks are on my TODO list I can try to rewrite pylint to plugins and config file. Martin^2 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0380.3-Remove-unused-imports.patch Type: text/x-patch Size: 106936 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0381.3-Remove-wildcard-imports.patch Type: text/x-patch Size: 45573 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0382.3-Enable-multiple-warnings-checks-in-Pylint.patch Type: text/x-patch Size: 4726 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0383.3-Enable-pylint-lost-exception-check.patch Type: text/x-patch Size: 1667 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0384.3-Enable-pylint-duplicated-key-check.patch Type: text/x-patch Size: 1918 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0385.3-Enable-pylint-trailing-whitespace-check.patch Type: text/x-patch Size: 3827 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0386.3-Enable-pylint-missing-final-newline-check.patch Type: text/x-patch Size: 1637 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0387.3-Enable-pylint-unused-format-string-key-check.patch Type: text/x-patch Size: 2774 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0388.3-Enable-pylint-expression-not-assigned-check.patch Type: text/x-patch Size: 8068 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0390.3-Enable-pylint-empty-docstring-check.patch Type: text/x-patch Size: 2590 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0393.3-Enable-pylint-unnecessary-pass-check.patch Type: text/x-patch Size: 11020 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0394-Use-module-variables-for-timedate_services.patch Type: text/x-patch Size: 2192 bytes Desc: not available URL: From pspacek at redhat.com Tue Dec 22 13:04:04 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 22 Dec 2015 14:04:04 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <566EC44C.8010905@redhat.com> References: <56687174.8000305@redhat.com> <20151210081357.GB21550@mail.corp.redhat.com> <56695291.5090705@redhat.com> <566990E3.2080903@redhat.com> <5669A3D3.7050608@redhat.com> <566A8B22.5020101@redhat.com> <566B0F94.5000007@redhat.com> <566E5FFA.5050705@redhat.com> <566E7CC6.1010201@redhat.com> <566E8A1F.70702@redhat.com> <566EC44C.8010905@redhat.com> Message-ID: <56794A44.2040609@redhat.com> On 14.12.2015 14:29, Tomas Babej wrote: > > > On 12/14/2015 10:21 AM, Martin Basti wrote: >> >> >> On 14.12.2015 09:24, Martin Kosek wrote: >>> On 12/14/2015 07:21 AM, Jan Cholasta wrote: >>>> On 11.12.2015 19:01, Tomas Babej wrote: >>>>> >>>>> On 12/11/2015 09:36 AM, Martin Kosek wrote: >>>>>> On 12/10/2015 05:09 PM, Martin Basti wrote: >>>>>>> >>>>>>> On 10.12.2015 15:49, Tomas Babej wrote: >>>>>>>> On 12/10/2015 11:23 AM, Martin Basti wrote: >>>>>>>>> On 10.12.2015 09:13, Lukas Slebodnik wrote: >>>>>>>>>> On (09/12/15 19:22), Martin Basti wrote: >>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>>>>>> >>>>>>>>>>> Patch attached. >>>>>>>>>> >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 00:00:00 >>>>>>>>>> 2001 >>>>>>>>>>> From: Martin Basti >>>>>>>>>>> Date: Wed, 9 Dec 2015 18:53:35 +0100 >>>>>>>>>>> Subject: [PATCH] Fix version comparison >>>>>>>>>>> >>>>>>>>>>> Use RPM library to compare vendor versions of IPA for redhat >>>>>>>>>>> platform >>>>>>>>>>> >>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>>>>>> --- >>>>>>>>>>> freeipa.spec.in | 2 ++ >>>>>>>>>>> ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ >>>>>>>>>>> 2 files changed, 21 insertions(+) >>>>>>>>>>> >>>>>>>>>>> diff --git a/freeipa.spec.in b/freeipa.spec.in >>>>>>>>>>> index >>>>>>>>>>> 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 100644 >>>>>>>>>>> --- a/freeipa.spec.in >>>>>>>>>>> +++ b/freeipa.spec.in >>>>>>>>>>> @@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} >>>>>>>>>>> Requires: gzip >>>>>>>>>>> Requires: python-gssapi >= 1.1.0 >>>>>>>>>>> Requires: custodia >>>>>>>>>>> +Requires: rpm-python >>>>>>>>>>> +Requires: rpmdevtools >>>>>>>>>> Could you explain why do you need the 2nd package? >>>>>>>>>> It does not contains any python modules >>>>>>>>>> and I cannot see usage of any binary in this patch >>>>>>>>>> >>>>>>>>>> LS >>>>>>>>> Thanks for this catch, it is actually located in yum package, I >>>>>>>>> rather >>>>>>>>> copy stringToVersion function from there to IPA, to avoid >>>>>>>>> dependency >>>>>>>>> hell >>>>>>>>> >>>>>>>>> Updated patch attached. >>>>>>>>> >>>>>>>>> >>>>>>>> Looking good. The __cmp__ function, however, is not available in >>>>>>>> Python >>>>>>>> 3. As we will eventually support python3 in RHEL as well, maybe we >>>>>>>> should make sure even platform-dependent parts are python3 >>>>>>>> compatible? >>>>>>>> For the future's sake. >>>>>>>> >>>>>>>> Tomas >>>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> python 3 compatible patch attached. >>>>>> I wonder why we have to add so much code here and reimplement RPM >>>>>> version checking if it is already provided by rpmdevtools: >>>>>> >>>>>> ~~~ >>>>>> $ /usr/bin/rpmdev-vercmp ipa-4.2.0-15.el7 4.2.0-15.el7_2.3; echo $? >>>>>> WARNING: hyphen in release1: 4.2.0-15.el7 >>>>>> >>>>>> rpmdev-vercmp >>>>>> rpmdev-vercmp >>>>>> rpmdev-vercmp # with no arguments, prompt >>>>>> >>>>>> Exit status is 0 if the EVR's are equal, 11 if EVR1 is newer, and >>>>>> 12 if >>>>>> EVR2 >>>>>> is newer. Other exit statuses indicate problems. >>>>>> >>>>>> ipa-4.2.0-15.el7 < 4.2.0-15.el7_2.3 >>>>>> 12 >>>>>> ~~~ >>>>>> >>>>>> which could be directly called from __eq__ or __lt__, since we are in >>>>>> platform specific code anyway already. >>>>>> >>>>>> Martin >>>>> Imho we should generally prefer reaching out to a Python library rather >>>>> launching a subprocess to compare the versions, it's unnecessary >>>>> overhead. >>> I would not be too worried about miliseconds longer execution on a >>> function >>> called during RPM upgrade. >>> >>>>> That said, the main argument for the usage of rpm-python over >>>>> rpmdevtools I see is that rpm-python is very likely to be present on >>>>> the >>>>> system (i.e. it is yum's own dependency), while rpmdevtools will not be >>>>> present by default. >>>>> >>>>> From the standpoint of trying to minimize the size of freeipa >>>>> installation (i.e recent wget -> curl migration), we should keep the >>>>> spirit here and do not introduce a dependency for a collection of >>>>> developer tools. >>>>> >>>>> /2 cents >>>> +1, also a single function is hardly "much code". >>> Ok then. If you all want to add yet another N-V-R parsing function in the >>> FreeIPA code, I can live with it (with raised eyebrows though). >> >> Rebased patch attached. > > I tested the patch, and it works fine - so conditional ACK from me for > the current iteration of the patch, given developer consensus which was > not reached yet. > > There's a split of opinions (external binary camp vs. copy&paste camp), > so we need to decide if we both camps are OK with proceeding. Further inspection shows that rpmdevtools depends on Perl stack which seems to be too much for such a simple thing. So, I'm hesitantly changing my NACK to ACK for copy&paste camp. -- Petr^2 Spacek From pvoborni at redhat.com Tue Dec 22 13:06:08 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 22 Dec 2015 14:06:08 +0100 Subject: [Freeipa-devel] [PATCH] 942 webui: add examples to network address validator error message Message-ID: <56794AC0.4030605@redhat.com> https://fedorahosted.org/freeipa/ticket/5532 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0942-webui-add-examples-to-network-address-validator-erro.patch Type: text/x-patch Size: 2098 bytes Desc: not available URL: From mbasti at redhat.com Tue Dec 22 13:06:29 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 22 Dec 2015 14:06:29 +0100 Subject: [Freeipa-devel] [PATCH 0365] Remove unused KRA code from ipa-server-install In-Reply-To: <5670353F.6070801@redhat.com> References: <565C6BBC.4050902@redhat.com> <566EC95C.2010208@redhat.com> <20151214140540.GT4620@redhat.com> <566ED2A6.8000202@redhat.com> <20151214155442.GV4620@redhat.com> <566FB63E.8080205@redhat.com> <56702FD9.7050008@redhat.com> <5670353F.6070801@redhat.com> Message-ID: <56794AD5.4000800@redhat.com> On 15.12.2015 16:43, David Kupka wrote: > On 15/12/15 16:20, Martin Kosek wrote: >> On 12/15/2015 07:42 AM, Jan Cholasta wrote: >>> On 14.12.2015 16:54, Alexander Bokovoy wrote: >>>> On Mon, 14 Dec 2015, David Kupka wrote: >> ... >>> We can always have both. With the new installer framework it is >>> trivial to fold >>> installers like this without code duplication. It is still work in >>> progress >>> though, so I would prefer not to add --setup-kra to >>> ipa-server-install now (if >>> ever). >> >> +1, I would also rather not make ipa-server-install too monolithic. >> Even from >> testing POV, I think doing KRA fixes in one installer is much more >> easier than >> doing it also in replica or server installers (although Jan's patches >> help >> here, that's true). >> > > If you look on the patch Martin send, you will see that there is > nothing monolithic in installing KRA in ipa-server-install. > In fact, it calls kra.check_install() and kra.install(), the very same > functions that are called from ipa-kra-install. > > But it seems that majority of us is decided to remove the ability to > install KRA from ipa-server-install and since I've tested the patch > and didn't find any issues we can push it, ACK. > Pushed to master: e622da3e1a25c05b77fed538634c284e68e2397f From mbasti at redhat.com Tue Dec 22 13:10:51 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 22 Dec 2015 14:10:51 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <56794A44.2040609@redhat.com> References: <56687174.8000305@redhat.com> <20151210081357.GB21550@mail.corp.redhat.com> <56695291.5090705@redhat.com> <566990E3.2080903@redhat.com> <5669A3D3.7050608@redhat.com> <566A8B22.5020101@redhat.com> <566B0F94.5000007@redhat.com> <566E5FFA.5050705@redhat.com> <566E7CC6.1010201@redhat.com> <566E8A1F.70702@redhat.com> <566EC44C.8010905@redhat.com> <56794A44.2040609@redhat.com> Message-ID: <56794BDB.9050602@redhat.com> On 22.12.2015 14:04, Petr Spacek wrote: > On 14.12.2015 14:29, Tomas Babej wrote: >> >> On 12/14/2015 10:21 AM, Martin Basti wrote: >>> >>> On 14.12.2015 09:24, Martin Kosek wrote: >>>> On 12/14/2015 07:21 AM, Jan Cholasta wrote: >>>>> On 11.12.2015 19:01, Tomas Babej wrote: >>>>>> On 12/11/2015 09:36 AM, Martin Kosek wrote: >>>>>>> On 12/10/2015 05:09 PM, Martin Basti wrote: >>>>>>>> On 10.12.2015 15:49, Tomas Babej wrote: >>>>>>>>> On 12/10/2015 11:23 AM, Martin Basti wrote: >>>>>>>>>> On 10.12.2015 09:13, Lukas Slebodnik wrote: >>>>>>>>>>> On (09/12/15 19:22), Martin Basti wrote: >>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>>>>>>> >>>>>>>>>>>> Patch attached. >>>>>>>>>>> >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 00:00:00 >>>>>>>>>>> 2001 >>>>>>>>>>>> From: Martin Basti >>>>>>>>>>>> Date: Wed, 9 Dec 2015 18:53:35 +0100 >>>>>>>>>>>> Subject: [PATCH] Fix version comparison >>>>>>>>>>>> >>>>>>>>>>>> Use RPM library to compare vendor versions of IPA for redhat >>>>>>>>>>>> platform >>>>>>>>>>>> >>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>>>>>>> --- >>>>>>>>>>>> freeipa.spec.in | 2 ++ >>>>>>>>>>>> ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ >>>>>>>>>>>> 2 files changed, 21 insertions(+) >>>>>>>>>>>> >>>>>>>>>>>> diff --git a/freeipa.spec.in b/freeipa.spec.in >>>>>>>>>>>> index >>>>>>>>>>>> 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 100644 >>>>>>>>>>>> --- a/freeipa.spec.in >>>>>>>>>>>> +++ b/freeipa.spec.in >>>>>>>>>>>> @@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} >>>>>>>>>>>> Requires: gzip >>>>>>>>>>>> Requires: python-gssapi >= 1.1.0 >>>>>>>>>>>> Requires: custodia >>>>>>>>>>>> +Requires: rpm-python >>>>>>>>>>>> +Requires: rpmdevtools >>>>>>>>>>> Could you explain why do you need the 2nd package? >>>>>>>>>>> It does not contains any python modules >>>>>>>>>>> and I cannot see usage of any binary in this patch >>>>>>>>>>> >>>>>>>>>>> LS >>>>>>>>>> Thanks for this catch, it is actually located in yum package, I >>>>>>>>>> rather >>>>>>>>>> copy stringToVersion function from there to IPA, to avoid >>>>>>>>>> dependency >>>>>>>>>> hell >>>>>>>>>> >>>>>>>>>> Updated patch attached. >>>>>>>>>> >>>>>>>>>> >>>>>>>>> Looking good. The __cmp__ function, however, is not available in >>>>>>>>> Python >>>>>>>>> 3. As we will eventually support python3 in RHEL as well, maybe we >>>>>>>>> should make sure even platform-dependent parts are python3 >>>>>>>>> compatible? >>>>>>>>> For the future's sake. >>>>>>>>> >>>>>>>>> Tomas >>>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> python 3 compatible patch attached. >>>>>>> I wonder why we have to add so much code here and reimplement RPM >>>>>>> version checking if it is already provided by rpmdevtools: >>>>>>> >>>>>>> ~~~ >>>>>>> $ /usr/bin/rpmdev-vercmp ipa-4.2.0-15.el7 4.2.0-15.el7_2.3; echo $? >>>>>>> WARNING: hyphen in release1: 4.2.0-15.el7 >>>>>>> >>>>>>> rpmdev-vercmp >>>>>>> rpmdev-vercmp >>>>>>> rpmdev-vercmp # with no arguments, prompt >>>>>>> >>>>>>> Exit status is 0 if the EVR's are equal, 11 if EVR1 is newer, and >>>>>>> 12 if >>>>>>> EVR2 >>>>>>> is newer. Other exit statuses indicate problems. >>>>>>> >>>>>>> ipa-4.2.0-15.el7 < 4.2.0-15.el7_2.3 >>>>>>> 12 >>>>>>> ~~~ >>>>>>> >>>>>>> which could be directly called from __eq__ or __lt__, since we are in >>>>>>> platform specific code anyway already. >>>>>>> >>>>>>> Martin >>>>>> Imho we should generally prefer reaching out to a Python library rather >>>>>> launching a subprocess to compare the versions, it's unnecessary >>>>>> overhead. >>>> I would not be too worried about miliseconds longer execution on a >>>> function >>>> called during RPM upgrade. >>>> >>>>>> That said, the main argument for the usage of rpm-python over >>>>>> rpmdevtools I see is that rpm-python is very likely to be present on >>>>>> the >>>>>> system (i.e. it is yum's own dependency), while rpmdevtools will not be >>>>>> present by default. >>>>>> >>>>>> From the standpoint of trying to minimize the size of freeipa >>>>>> installation (i.e recent wget -> curl migration), we should keep the >>>>>> spirit here and do not introduce a dependency for a collection of >>>>>> developer tools. >>>>>> >>>>>> /2 cents >>>>> +1, also a single function is hardly "much code". >>>> Ok then. If you all want to add yet another N-V-R parsing function in the >>>> FreeIPA code, I can live with it (with raised eyebrows though). >>> Rebased patch attached. >> I tested the patch, and it works fine - so conditional ACK from me for >> the current iteration of the patch, given developer consensus which was >> not reached yet. >> >> There's a split of opinions (external binary camp vs. copy&paste camp), >> so we need to decide if we both camps are OK with proceeding. > Further inspection shows that rpmdevtools depends on Perl stack which seems to > be too much for such a simple thing. So, I'm hesitantly changing my NACK to > ACK for copy&paste camp. > It seems that copy&paste camp won. Pushed to: master: 91913c5ba7c380fe6456e1c3e35fcbfbecef5ff1 ipa-4-3: a249de3b00f20956214a6ee0c1d614b972826637 Patch for ipa-4-2 needs rebase ... wait for it please Martin^2 From mbasti at redhat.com Tue Dec 22 13:18:46 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 22 Dec 2015 14:18:46 +0100 Subject: [Freeipa-devel] [PATCH 0373] Upgrade: Fix IPA version comparison In-Reply-To: <56794BDB.9050602@redhat.com> References: <56687174.8000305@redhat.com> <20151210081357.GB21550@mail.corp.redhat.com> <56695291.5090705@redhat.com> <566990E3.2080903@redhat.com> <5669A3D3.7050608@redhat.com> <566A8B22.5020101@redhat.com> <566B0F94.5000007@redhat.com> <566E5FFA.5050705@redhat.com> <566E7CC6.1010201@redhat.com> <566E8A1F.70702@redhat.com> <566EC44C.8010905@redhat.com> <56794A44.2040609@redhat.com> <56794BDB.9050602@redhat.com> Message-ID: <56794DB6.6000403@redhat.com> On 22.12.2015 14:10, Martin Basti wrote: > > > On 22.12.2015 14:04, Petr Spacek wrote: >> On 14.12.2015 14:29, Tomas Babej wrote: >>> >>> On 12/14/2015 10:21 AM, Martin Basti wrote: >>>> >>>> On 14.12.2015 09:24, Martin Kosek wrote: >>>>> On 12/14/2015 07:21 AM, Jan Cholasta wrote: >>>>>> On 11.12.2015 19:01, Tomas Babej wrote: >>>>>>> On 12/11/2015 09:36 AM, Martin Kosek wrote: >>>>>>>> On 12/10/2015 05:09 PM, Martin Basti wrote: >>>>>>>>> On 10.12.2015 15:49, Tomas Babej wrote: >>>>>>>>>> On 12/10/2015 11:23 AM, Martin Basti wrote: >>>>>>>>>>> On 10.12.2015 09:13, Lukas Slebodnik wrote: >>>>>>>>>>>> On (09/12/15 19:22), Martin Basti wrote: >>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>>>>>>>> >>>>>>>>>>>>> Patch attached. >>>>>>>>>>>> >From 8ef93485d61e8732166fb0c5b6c4559209740f3e Mon Sep 17 >>>>>>>>>>>> 00:00:00 >>>>>>>>>>>> 2001 >>>>>>>>>>>>> From: Martin Basti >>>>>>>>>>>>> Date: Wed, 9 Dec 2015 18:53:35 +0100 >>>>>>>>>>>>> Subject: [PATCH] Fix version comparison >>>>>>>>>>>>> >>>>>>>>>>>>> Use RPM library to compare vendor versions of IPA for redhat >>>>>>>>>>>>> platform >>>>>>>>>>>>> >>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5535 >>>>>>>>>>>>> --- >>>>>>>>>>>>> freeipa.spec.in | 2 ++ >>>>>>>>>>>>> ipaplatform/redhat/tasks.py | 19 +++++++++++++++++++ >>>>>>>>>>>>> 2 files changed, 21 insertions(+) >>>>>>>>>>>>> >>>>>>>>>>>>> diff --git a/freeipa.spec.in b/freeipa.spec.in >>>>>>>>>>>>> index >>>>>>>>>>>>> 9f82b3695fb10c4db65cc31278364b3b34e26098..09feba7b8324f5e645da3e8010de86b6c3ee5ab9 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> 100644 >>>>>>>>>>>>> --- a/freeipa.spec.in >>>>>>>>>>>>> +++ b/freeipa.spec.in >>>>>>>>>>>>> @@ -166,6 +166,8 @@ Requires: %{etc_systemd_dir} >>>>>>>>>>>>> Requires: gzip >>>>>>>>>>>>> Requires: python-gssapi >= 1.1.0 >>>>>>>>>>>>> Requires: custodia >>>>>>>>>>>>> +Requires: rpm-python >>>>>>>>>>>>> +Requires: rpmdevtools >>>>>>>>>>>> Could you explain why do you need the 2nd package? >>>>>>>>>>>> It does not contains any python modules >>>>>>>>>>>> and I cannot see usage of any binary in this patch >>>>>>>>>>>> >>>>>>>>>>>> LS >>>>>>>>>>> Thanks for this catch, it is actually located in yum package, I >>>>>>>>>>> rather >>>>>>>>>>> copy stringToVersion function from there to IPA, to avoid >>>>>>>>>>> dependency >>>>>>>>>>> hell >>>>>>>>>>> >>>>>>>>>>> Updated patch attached. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> Looking good. The __cmp__ function, however, is not available in >>>>>>>>>> Python >>>>>>>>>> 3. As we will eventually support python3 in RHEL as well, >>>>>>>>>> maybe we >>>>>>>>>> should make sure even platform-dependent parts are python3 >>>>>>>>>> compatible? >>>>>>>>>> For the future's sake. >>>>>>>>>> >>>>>>>>>> Tomas >>>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> python 3 compatible patch attached. >>>>>>>> I wonder why we have to add so much code here and reimplement RPM >>>>>>>> version checking if it is already provided by rpmdevtools: >>>>>>>> >>>>>>>> ~~~ >>>>>>>> $ /usr/bin/rpmdev-vercmp ipa-4.2.0-15.el7 4.2.0-15.el7_2.3; >>>>>>>> echo $? >>>>>>>> WARNING: hyphen in release1: 4.2.0-15.el7 >>>>>>>> >>>>>>>> rpmdev-vercmp >>>>>>>> >>>>>>>> rpmdev-vercmp >>>>>>>> rpmdev-vercmp # with no arguments, prompt >>>>>>>> >>>>>>>> Exit status is 0 if the EVR's are equal, 11 if EVR1 is newer, and >>>>>>>> 12 if >>>>>>>> EVR2 >>>>>>>> is newer. Other exit statuses indicate problems. >>>>>>>> >>>>>>>> ipa-4.2.0-15.el7 < 4.2.0-15.el7_2.3 >>>>>>>> 12 >>>>>>>> ~~~ >>>>>>>> >>>>>>>> which could be directly called from __eq__ or __lt__, since we >>>>>>>> are in >>>>>>>> platform specific code anyway already. >>>>>>>> >>>>>>>> Martin >>>>>>> Imho we should generally prefer reaching out to a Python library >>>>>>> rather >>>>>>> launching a subprocess to compare the versions, it's unnecessary >>>>>>> overhead. >>>>> I would not be too worried about miliseconds longer execution on a >>>>> function >>>>> called during RPM upgrade. >>>>> >>>>>>> That said, the main argument for the usage of rpm-python over >>>>>>> rpmdevtools I see is that rpm-python is very likely to be >>>>>>> present on >>>>>>> the >>>>>>> system (i.e. it is yum's own dependency), while rpmdevtools will >>>>>>> not be >>>>>>> present by default. >>>>>>> >>>>>>> From the standpoint of trying to minimize the size of freeipa >>>>>>> installation (i.e recent wget -> curl migration), we should keep >>>>>>> the >>>>>>> spirit here and do not introduce a dependency for a collection of >>>>>>> developer tools. >>>>>>> >>>>>>> /2 cents >>>>>> +1, also a single function is hardly "much code". >>>>> Ok then. If you all want to add yet another N-V-R parsing function >>>>> in the >>>>> FreeIPA code, I can live with it (with raised eyebrows though). >>>> Rebased patch attached. >>> I tested the patch, and it works fine - so conditional ACK from me for >>> the current iteration of the patch, given developer consensus which was >>> not reached yet. >>> >>> There's a split of opinions (external binary camp vs. copy&paste camp), >>> so we need to decide if we both camps are OK with proceeding. >> Further inspection shows that rpmdevtools depends on Perl stack which >> seems to >> be too much for such a simple thing. So, I'm hesitantly changing my >> NACK to >> ACK for copy&paste camp. >> > It seems that copy&paste camp won. > > Pushed to: > master: 91913c5ba7c380fe6456e1c3e35fcbfbecef5ff1 > ipa-4-3: a249de3b00f20956214a6ee0c1d614b972826637 > > > Patch for ipa-4-2 needs rebase ... wait for it please > > Martin^2 > Pushed to ipa-4-2: dccdade484698d5ef553f03ce6aab5d9db7a7cf0 From pvoborni at redhat.com Tue Dec 22 13:23:18 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 22 Dec 2015 14:23:18 +0100 Subject: [Freeipa-devel] [PATCH] 943 webui: pwpolicy cospriority field was marked as required Message-ID: <56794EC6.8040302@redhat.com> https://fedorahosted.org/freeipa/ticket/5553 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0943-webui-pwpolicy-cospriority-field-was-marked-as-requi.patch Type: text/x-patch Size: 913 bytes Desc: not available URL: From pspacek at redhat.com Tue Dec 22 13:32:11 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 22 Dec 2015 14:32:11 +0100 Subject: [Freeipa-devel] [PATCH 0072-0081] DNSSEC: fixes In-Reply-To: <56783D51.3070005@redhat.com> References: <56780DBA.60703@redhat.com> <56781085.4070106@redhat.com> <56783D51.3070005@redhat.com> Message-ID: <567950DB.7050708@redhat.com> On 21.12.2015 18:56, Martin Basti wrote: > > > On 21.12.2015 15:45, Martin Basti wrote: >> >> >> On 21.12.2015 15:33, Petr Spacek wrote: >>> Hello, >>> >>> this patch set fixes key rotation in DNSSEC. >>> >>> You can use attached template files for OpenDNSSEC config to shorten time >>> intervals between key rotations. >>> >>> Please let me know if you have any questions, I'm all ears! >>> >> Please fix whitespace error: >> >> Applying: DNSSEC: logging improvements in ldapkeydb.py >> /home/mbasti/work/freeipa-devel/.git/rebase-apply/patch:14: trailing >> whitespace. >> >> warning: 1 line adds whitespace errors. >> > *) DNSSEC: ipa-dnskeysyncd: call ods-signer ldap-cleanup on zone removal > > Is is safe to do not use try - except with ipatuil.run()? What if ods-signer > command failed? That is intentional. The call should never fail, so if it fails there is no way how to recover cleanly except restarting the daemon. The unhandled exception will kill the daemon and systemd will restart it later on. > *) DNSSEC: Improve error reporting from ipa-ods-exporter > IMO log.exception(ex) is enough, do we need to add traceback to msg? msg is sent over socket to another process (see send_systemd_reply(conn, msg) call in finally: block). Without this the remote party would not receive the error information. > *) DNSSEC: Make sure that current state in OpenDNSSEC matches key state in LDAP > I think this is okay because we want to use KSK instantly, but just to be > sure, is Publish->Activate okay? > + bind_times['idnsSecKeyActivate'] = ods_times['idnsSecKeyPublish'] > > Just to be sure how this will be handle during KSK key rotation? We have to copy semantics from OpenDNSSEC. Please see design page https://fedorahosted.org/bind-dyndb-ldap/wiki/BIND9/Design/DNSSEC/OpenDNSSEC2BINDKeyStates , it describes in detail why I done it this way. > *) DNSSEC: Make sure that current key state in LDAP matches key state in BIND > LGTM > > *) DNSSEC: remove obsolete TODO note > ACK > > *) DNSSEC: add debug mode to ldapkeydb.py > A) > You can remove __str__ method, python will use __repr__ as default Done. > B) > for attr in ['ipaPrivateKey', 'ipaPublicKey', 'ipk11publickeyinfo']: > Do we need to sanitize *public*Key and publicKeyinfo? Yes, we need it. The output with any of ['ipaPrivateKey', 'ipaPublicKey', 'ipk11publickeyinfo'] is huge blob and printing it does not help readability. Purpose of the patch is to make it easy to read and debug so printing useless blobs would go directly against the purpose :-) > C) > in odsmgr.py is used ipa_log_manager, can we use the same for consistency? Fixed, thanks. > D) > Do we need logging there, everything is printed via print except debug info > about connecting, can you just redirect it to stderr, and usable data leave in > stdout? Yes, we need it because it eases debugging. print() prints useful information to stdout. 'Garbage' about connecting to LDAP, IPA framework initialization and so on does via logger to stderr, so it can be easily separated from useful information using redirection in BASH. I've added a comment right below if __name__ == '__main__': to make it clear why we do not use logger in there. > *) DNSSEC: logging improvements in ldapkeydb.py > IMO commit message should be: ".... in ipa-ods-exporter" > > Otherwise LGTM Fixed, thanks. > *) DNSSEC: remove keys purged by OpenDNSSEC from master HSM from LDAP > > A) coding style: please use (), instead of "\" > assert set(pubkeys_local) == set(privkeys_local), ( > "IDs of private and public keys for DNS zones in local HSM does " > "not match to key pairs: %s vs. %s" % > (hex_set(pubkeys_local), hex_set(privkeys_local)) > ) Fixed. > B) coding style > assert not matched_already, ( > "key %s is in more than one keyset" % hexlify(keyid) > ) Not relevant anymore, see below. > C) schedule_key_deletion() > how about case when keyid is not in any keyset, then keyid will not be > replaced by object and it blow up somewhere else Not relevant anymore, see below. > D) +class KeyDeleter(object): > I would like to have a check there which blows up nicely if _update_key() is > called twice on the same object. With current implementation you will get > NoneType has no delete_entry method. Not relevant anymore, see below. > E) > I somehow does not like the placeholder object. Could we just extend Key > object with attribute "to_be_deleted" or something similar, and if this > attribute is set to True, Key._update_key() can remove, instead of creation a > new object. > Key.prepare_deletion() can set the value "to_be_deleted" to True. Main purpose of the KeyDeleter object was to be incompatible the Key object. I want to be 100 % that is not possible to call schedule_delete() and subsequenty modify the Key object. I've reworked the Key object so it has schedule_deletion() method and that all other methods call __assert_not_deleted() to make sure that the object was not deleted. Is it better? > *) DNSSEC: ipa-dnskeysyncd: Skip zones with old DNSSEC metadata in LDAP > How often is keysyncer initialized? Might happen the case where one of > dnssec_zones has been disabled and keysyncer is not aware of this change? KeySyncer is SyncReplConsumer, so it gets constant stream of updates from LDAP. IMHO the answer is no, it cannot miss the update. > You may want to use DNSName from ipapython.dnsutil instead of pure dns.name Other places in daemons are using dns.name too, so I will keep it that way. For this particular case there would be no advantage anyway. > *) DNSSEC: ipa-ods-exporter: add ldap-cleanup command > LGTM Old and new version of patches can be found on Github: * old: https://github.com/pspacek/freeipa/tree/dnssec_fixes * new: https://github.com/pspacek/freeipa/tree/dnssec_fixes2 Fixed patched (+1 new) are attached. -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0072-2-DNSSEC-Improve-error-reporting-from-ipa-ods-exporter.patch Type: text/x-patch Size: 1060 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0073-2-DNSSEC-Make-sure-that-current-state-in-OpenDNSSEC-ma.patch Type: text/x-patch Size: 8091 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0074-2-DNSSEC-Make-sure-that-current-key-state-in-LDAP-matc.patch Type: text/x-patch Size: 1800 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0075-2-DNSSEC-remove-obsolete-TODO-note.patch Type: text/x-patch Size: 1005 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0076-2-DNSSEC-add-debug-mode-to-ldapkeydb.py.patch Type: text/x-patch Size: 3514 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0077-2-DNSSEC-logging-improvements-in-ipa-ods-exporter.patch Type: text/x-patch Size: 2756 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0078-2-DNSSEC-remove-keys-purged-by-OpenDNSSEC-from-master-.patch Type: text/x-patch Size: 9666 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0079-2-DNSSEC-ipa-dnskeysyncd-Skip-zones-with-old-DNSSEC-me.patch Type: text/x-patch Size: 4522 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0080-2-DNSSEC-ipa-ods-exporter-add-ldap-cleanup-command.patch Type: text/x-patch Size: 4806 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0081-2-DNSSEC-ipa-dnskeysyncd-call-ods-signer-ldap-cleanup-.patch Type: text/x-patch Size: 1444 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pspacek-0082-DNSSEC-Log-debug-messages-at-log-level-DEBUG.patch Type: text/x-patch Size: 1112 bytes Desc: not available URL: From pspacek at redhat.com Tue Dec 22 13:33:21 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 22 Dec 2015 14:33:21 +0100 Subject: [Freeipa-devel] [PATCH 0077] ipa-dns-install: Do not check for zone overlap when DNS, installed. In-Reply-To: <56793EC0.3010107@redhat.com> References: <5673DF56.8060906@redhat.com> <5673E832.3020609@redhat.com> <56741AA8.5060807@redhat.com> <56793EC0.3010107@redhat.com> Message-ID: <56795121.7060804@redhat.com> On 22.12.2015 13:14, David Kupka wrote: > On 18/12/15 15:39, Petr Vobornik wrote: >> On 12/18/2015 12:04 PM, Petr Vobornik wrote: >>> On 12/18/2015 11:26 AM, David Kupka wrote: >>>> Standalone DNS installer always performed overlap check effectively >>>> preventing installation on replica when other DNS instance was already >>>> installed in topology. >>>> >>>> >>> >>> I don't like the position of api argument in the install_check. It is >>> not consistent with install_check in KRA plus it's between two related >>> options: standalone and replica. >>> >>> Is there a reason behind it which I don't see? >> >> NACK, the API call is incorrect. >> >> It should be api.Command['dns_is_enabled']()['result'] or >> api.Command.dns_is_enabled()['result'] >> >> >> ipa-dns-install --forwarder=$DNS_FORWARDER >> >> The log file for this installation can be found in >> /var/log/ipaserver-install.log >> Unexpected error - see /var/log/ipaserver-install.log for details: >> TypeError: __call__() takes exactly 1 argument (2 given) >> >> >> log file: >> >> 2015-12-18T14:33:30Z DEBUG File >> "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", >> line 736, in run_script >> return_value = main_function() >> >> File "/sbin/ipa-dns-install", line 137, in main >> dns_installer.install_check(True, api, False, options, >> hostname=api.env.host) >> >> File "/usr/lib/python2.7/site-packages/ipaserver/install/dns.py", >> line 113, in install_check >> already_enabled = api.Command('dns_is_enabled')['result'] >> >> 2015-12-18T14:33:30Z DEBUG The ipa-dns-install command failed, >> exception: TypeError: __call__() takes exactly 1 argument (2 given) >> > Thanks for catch. Sometimes I mess up with patch files. Fixed patch attached. ACK, works for me. -- Petr^2 Spacek From redhatrises at gmail.com Tue Dec 22 13:33:32 2015 From: redhatrises at gmail.com (Gabe Alford) Date: Tue, 22 Dec 2015 06:33:32 -0700 Subject: [Freeipa-devel] [PATCH] 942 webui: add examples to network address validator error message In-Reply-To: <56794AC0.4030605@redhat.com> References: <56794AC0.4030605@redhat.com> Message-ID: LGTM. Gabe On Tue, Dec 22, 2015 at 6:06 AM, Petr Vobornik wrote: > https://fedorahosted.org/freeipa/ticket/5532 > -- > Petr Vobornik > > -- > Manage your subscription for the Freeipa-devel mailing list: > https://www.redhat.com/mailman/listinfo/freeipa-devel > Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dkupka at redhat.com Tue Dec 22 14:14:43 2015 From: dkupka at redhat.com (David Kupka) Date: Tue, 22 Dec 2015 15:14:43 +0100 Subject: [Freeipa-devel] [PATCH 0077-8] ipa-replica-prepare: Add '--auto-reverse' and, '--allow-zone-overlap' options Message-ID: <56795AD3.7010102@redhat.com> https://fedorahosted.org/freeipa/ticket/5563 -- David Kupka -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0078.0-ipa-replica-prepare-Add-auto-reverse-and-allow-zone-.patch Type: text/x-patch Size: 1459 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-dkupka-0079.0-installer-Change-reverse-zones-question-to-better-re.patch Type: text/x-patch Size: 918 bytes Desc: not available URL: From mbasti at redhat.com Tue Dec 22 14:18:47 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 22 Dec 2015 15:18:47 +0100 Subject: [Freeipa-devel] [PATCH 0394, 0379 - 0393] Winter code cleanup In-Reply-To: <5679458D.9090207@redhat.com> References: <56720BCC.1050504@redhat.com> <56725C34.5060400@redhat.com> <5672B631.7030709@redhat.com> <5678F671.8080707@redhat.com> <5679458D.9090207@redhat.com> Message-ID: <56795BC7.6070001@redhat.com> On 22.12.2015 13:43, Martin Basti wrote: > > > On 22.12.2015 08:06, Jan Cholasta wrote: >> On 17.12.2015 14:18, Martin Basti wrote: >>> >>> >>> On 17.12.2015 07:54, Jan Cholasta wrote: >>>> Hi, >>>> >>>> On 17.12.2015 02:11, Martin Basti wrote: >>>>> Hello, >>>>> following patches cleanup the code and add checks to pylint to >>>>> avoid the >>>>> mess again >>>>> >>>>> Patches: 379-381: >>>>> the most important patches, they cleanup imports >>>>> >>>>> Patch 382: >>>>> enables various pylint checks, we may reduce the list of check if >>>>> needed. >>>> >>>> Would it be possible to turn this into a blacklist of disabled >>>> warnings rather than a whitelist of enabled warnings? >>>> >>>>> >>>>> Patches 383 - 393: >>>>> remove minor issues from code, and enable particular checks >>>>> Feel free to nack patches/checks that should not be there. >>>>> >>>>> Please apply patches in order. >>>>> >>>>> Martin^2 >>>>> >>>>> >>>> >>>> Honza >>>> >>> Updated patches attached. >> >> Patch 379: ACK >> >> >> Patch 380: >> >> 1) This patch needs to be rebased. >> >> >> 2) make-lint is reporting this to me: >> >> ************* Module ipatests.test_cmdline.test_ipagetkeytab >> ipatests/test_cmdline/test_ipagetkeytab.py:30: [W0611(unused-import), >> ] Unused import nose) >> ipatests/test_cmdline/test_ipagetkeytab.py:34: [W0611(unused-import), >> ] Unused DN imported from ipapython.dn) >> ************* Module ipatests.test_cmdline.test_help >> ipatests/test_cmdline/test_help.py:21: [W0611(unused-import), ] >> Unused import contextlib) >> ipatests/test_cmdline/test_help.py:23: [W0611(unused-import), ] >> Unused assert_raises imported from nose.tools) >> ************* Module ipatests.test_cmdline.test_cli >> ipatests/test_cmdline/test_cli.py:11: [W0611(unused-import), ] Unused >> API_VERSION imported from ipapython.version) >> ************* Module ipaplatform.services >> ipaplatform/services.py:59: [W0611(unused-import), ] Unused >> timedate_services imported from ipaplatform.redhat.services) >> ************* Module ipaplatform.rhel.services >> ipaplatform/rhel/services.py:59: [W0611(unused-import), ] Unused >> timedate_services imported from ipaplatform.redhat.services) >> ************* Module ipaplatform.redhat.services >> ipaplatform/redhat/services.py:306: [W0611(unused-import), ] Unused >> timedate_services imported from ipaplatform.base.services) >> ************* Module ipaplatform.fedora.services >> ipaplatform/fedora/services.py:59: [W0611(unused-import), ] Unused >> timedate_services imported from ipaplatform.redhat.services) >> ************* Module ipalib.setup >> ipalib/setup.py:28: [W0611(unused-import), ] Unused import >> distutils.sysconfig) >> >> Note that the "from ipaplatform.${PLATFORM}.services import >> timedate_services" in services.py should be rewritten to >> "timedate_services = ${PLATFORM}_services.timedate_services" to fix >> this. >> >> >> Patch 381: ACK >> >> >> Patch 382: >> >> Nitpick: according to , the order >> of message categories is F, E, W, C, R from the most severe to the >> least severe (it does not mention I, but I think it should be last, >> after R), IMO we should keep this order here as well for clarity. >> (Don't worry, doing this should not require rebasing the following >> patches.) >> >> >> Patch 383 - 387: ACK >> >> >> Patch 388: >> >> IMO it would be better to rewrite this: >> >> [(t.id, t.options) for t in doc.getKeyPackages()] # >> pylint: disable=expression-not-assigned >> >> into this: >> >> for t in doc.getKeyPackages(): >> t._PSKCKeyPackage__process() >> >> rather than disabling the check. >> >> >> Patch 389: >> >> All this patch does is that it enables a check globally and then >> disables it everywhere locally. >> >> IMO the patch should be retired for now, or make-lint should be >> taught to automatically skip the check inside TestCase.assertRaises() >> context. >> >> >> Patch 390: ACK >> >> >> Patch 391: >> >> default_from uses argument names of the specified function as param >> names from which to create the default. >> >> These changes break default_from: >> >> - default_from=lambda name_from_ip: >> _reverse_zone_name(name_from_ip), >> + default_from=_reverse_zone_name, >> >> - default_from=lambda idnsname: >> default_zone_update_policy(idnsname), >> + default_from=default_zone_update_policy, >> >> This change adds dependency on non-existent param: >> >> - default_from=lambda: krb_utils.get_principal(), >> + default_from=krb_utils.get_principal, >> >> The result of this check is misleading for default_from, so IMO the >> patch should be retired for now, or make-lint should be taught to >> automatically disable the check for the default_from argument in >> param definitions. >> >> >> Patch 392: >> >> I think the existing occurences of exec()/eval() should be removed >> before this check can be enabled. >> >> >> Patch 393: ACK >> >> >> I would like to see a patch which enables the useless-suppression >> info message, so that we can catch dangling "# pylint: disable=" >> comments (there are some in the code). >> >> (Also, it would be nice to finally rewrite make-lint to pylint config >> file + plugins.) >> > Updated patches attached, please apply patch 394 first. > > Patches 389, 391, 392 have been removed, issues will be addressed later. > > useless-suppression, unbalanced-tuple-unpacking, > unpacking-non-sequence, and python3 checks are on my TODO list > > I can try to rewrite pylint to plugins and config file. > > Martin^2 > > I forgot to attach patch 379, all patches again (394 should go first) -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0379.3-Remove-empty-test-file.patch Type: text/x-patch Size: 1537 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0380.3-Remove-unused-imports.patch Type: text/x-patch Size: 106936 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0381.3-Remove-wildcard-imports.patch Type: text/x-patch Size: 45573 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0382.3-Enable-multiple-warnings-checks-in-Pylint.patch Type: text/x-patch Size: 4726 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0383.3-Enable-pylint-lost-exception-check.patch Type: text/x-patch Size: 1667 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0384.3-Enable-pylint-duplicated-key-check.patch Type: text/x-patch Size: 1918 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0385.3-Enable-pylint-trailing-whitespace-check.patch Type: text/x-patch Size: 3827 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0386.3-Enable-pylint-missing-final-newline-check.patch Type: text/x-patch Size: 1637 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0387.3-Enable-pylint-unused-format-string-key-check.patch Type: text/x-patch Size: 2774 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0388.3-Enable-pylint-expression-not-assigned-check.patch Type: text/x-patch Size: 8068 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0390.3-Enable-pylint-empty-docstring-check.patch Type: text/x-patch Size: 2590 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0393.3-Enable-pylint-unnecessary-pass-check.patch Type: text/x-patch Size: 11020 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0394-Use-module-variables-for-timedate_services.patch Type: text/x-patch Size: 2192 bytes Desc: not available URL: From pspacek at redhat.com Tue Dec 22 14:50:11 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 22 Dec 2015 15:50:11 +0100 Subject: [Freeipa-devel] ipa-devel repos on jdennis.fedorapeople.org In-Reply-To: <20150827083452.GN22106@redhat.com> References: <55A532DF.7070302@redhat.com> <55A53D93.4050201@redhat.com> <20150715074450.GL4218@redhat.com> <55DEC9F5.4070302@redhat.com> <20150827083452.GN22106@redhat.com> Message-ID: <56796323.1000303@redhat.com> On 27.8.2015 10:34, Alexander Bokovoy wrote: > On Thu, 27 Aug 2015, Petr Spacek wrote: >> On 15.7.2015 09:44, Jan Pazdziora wrote: >>> On Tue, Jul 14, 2015 at 12:49:23PM -0400, John Dennis wrote: >>>> On 07/14/2015 12:03 PM, Petr Spacek wrote: >>>>> Hello, >>>>> >>>>> Is anyone using repos >>>>> https://jdennis.fedorapeople.org/ipa-devel/ >>>>> ? >>>>> >>>>> AFAIK nobody in Brno is seriously using it but I'm not sure about people >>>>> outside the Brno. >>>>> >>>>> Could we use COPR instead and get out of builder business? Upcoming lab >>>>> maintenance window could be a good time to do that. >>>> >>>> I would love to get out of the builder business and I suspect Nalin would as >>>> well [1]. The question came up in our Monday meeting as well. Nobody seem to >>>> know if anyone was using these builds and why we weren't using COPR. The >>> >>> The Fedora infra admins should be able to provide HTTP logs for the >>> repo, if you needs some numbers about potential usage. >> >> That is a good idea! I got logs from Fedora admins and as far as I can tell, >> in the last month there were only 7 RPM downloads and nothing else. >> >> The 7 hits I found was for >> /ipa-devel/rhel/6/x86_64/os/sssd-1.13.1-0.20150813T1121Zgit137d5dd.el6.i686.rpm >> and >> other packages from the same version. >> >> I did not find any hits for IPA packages at all. All the remaining traffic >> (except the 7 RPM hits) was from repo data refreshes: >> - 83 % is RHEL 5 repodata >> - 13 % is RHEL 6 repodata >> - remaining ~ 4 % of noise is Fedora repodata >> >> It seems to me that we can get out the builder business completely and >> decommission ipa-devel and replace it with COPR. >> >> Do you agree? John? Nathaniel? Stephen? Anyone? :-) > Yes, I think we can decommission this repository thanks to COPR > infrastructure. John, the machines which used to generate the repos are basically dead now. Could you remove the directories and replace them with a README with sentence that the repos were discontinued, please? -- Petr^2 Spacek From pspacek at redhat.com Tue Dec 22 14:54:03 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 22 Dec 2015 15:54:03 +0100 Subject: [Freeipa-devel] [PATCH] Add option to disable setkeytab extended operations In-Reply-To: <20151201110057.GP9605@redhat.com> References: <1448394152.29102.10.camel@redhat.com> <565D5735.7080405@redhat.com> <20151201082128.GL9605@redhat.com> <565D5BA2.7080608@redhat.com> <20151201084720.GM9605@redhat.com> <565D685E.9080704@redhat.com> <20151201095348.GO9605@redhat.com> <20151201110057.GP9605@redhat.com> Message-ID: <5679640B.8040206@redhat.com> On 1.12.2015 12:00, Alexander Bokovoy wrote: > On Tue, 01 Dec 2015, Alexander Bokovoy wrote: >> On Tue, 01 Dec 2015, Petr Spacek wrote: >>> On 1.12.2015 09:47, Alexander Bokovoy wrote: >>>> On Tue, 01 Dec 2015, Petr Spacek wrote: >>>>> On 1.12.2015 09:21, Alexander Bokovoy wrote: >>>>>> On Tue, 01 Dec 2015, Petr Spacek wrote: >>>>>>> On 24.11.2015 20:42, Simo Sorce wrote: >>>>>>>> Since some time we use the getkeytab operation to fetch keytabs on newer >>>>>>>> clients. According to bug #232 setkeytab can be used to circumvent >>>>>>>> password quality controls so it needs to be slowly retired. >>>>>>>> >>>>>>>> The attached patches implement #5485 in 2 parts. >>>>>>>> >>>>>>>> The first introduces the option DisableSetKeytab which globally disables >>>>>>>> the setkeytab extended operation. This is set to false by default for >>>>>>>> backwards compatibility. >>>>>>>> >>>>>>>> The second introduces an option called DisableUserSetKeytab, which is >>>>>>>> active by default in new installs (but not in upgraded ones), and only >>>>>>>> disables the use of setkeytab for ipa suers, but not for hosts/services. >>>>>>>> This is because user's are the ones that may abuse the interface to >>>>>>>> escape password policies and users also normally do not acquire keytabs, >>>>>>>> so it is a safe bet to disable just them by default in new installs. >>>>>>> >>>>>>> On a related note, how this works with plain kadmin & kpasswd protocols? >>>>>> It is unrelated. We don't support principal manipulation via kadmin >>>>>> protocol. >>>>> >>>>> Sure, I know that, but I'm trying to find out if we re-invented the wheel or >>>>> if our wheel has some additional features which cannot be incorporated to >>>>> the >>>>> original wheel :-) >>>> There is no need to incorporate something specific into kadmin protocol, >>>> the problem is with the fact that we don't have access controls within >>>> our KDC driver. It always connects to LDAP server over ldapi as root and >>>> thus maps to cn=Directory Manager which bypasses LDAP ACIs. As such, >>>> this means we'll need to have some access control added to KDC DAL >>>> driver before we can allow kadmin operations on principals. >>>> >>>> Adding those access controls is not an easy feat. >>> >>> Hmm, why is that? We could use LDAP Proxy control (RFC 4370) and let DS to >>> evaluate ACIs as usual. The change in kadmin would be adding LDAP Proxy >>> control with proper DN (i.e. mapping from principal name to DN) to the LDAP >>> requests. >> Not sure how does it help: >> $ ldapsearch -Dcn=Directory\ Manager -W -e >> \!authzid=dn:uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li -b >> uid=admin,cn=users,cn=accounts,dc=vda,dc=li userPassword >> Enter LDAP Password: # extended LDIF >> # >> # LDAPv3 >> # base with scope subtree >> # filter: (objectclass=*) >> # requesting: userPassword # >> >> # admin, users, accounts, vda.li >> dn: uid=admin,cn=users,cn=accounts,dc=vda,dc=li >> userPassword:: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX= >> = >> >> # search result >> search: 2 >> result: 0 Success >> >> # numResponses: 2 >> # numEntries: 1 >> $ ldapsearch -Y GSSAPI -e >> \!authzid=dn:uid=admin,cn=users,cn=accounts,dc=vda,dc=li -b >> uid=admin,cn=users,cn=accounts,dc=vda,dc=li userPassword >> SASL/GSSAPI authentication started >> SASL username: abokovoy at VDA.LI >> SASL SSF: 56 >> SASL data security layer installed. >> # extended LDIF >> # >> # LDAPv3 >> # base with scope subtree >> # filter: (objectclass=*) >> # requesting: userPassword # >> >> # search result >> search: 4 >> result: 0 Success >> >> # numResponses: 1 >> >> As you can see, posing different authzid does not allow to get through >> ACLs unless you were already a directory manager. Which means the >> proxyauth control is not really useful here: >> >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=0 BIND dn="cn=Directory Manager" >> method=128 version=3 >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=0 RESULT err=0 tag=97 nentries=0 >> etime=0 dn="cn=directory manager" >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=1 SRCH >> base="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" scope=2 >> filter="(objectClass=*)" attrs="userPassword" >> authzid="uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li" >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=1 RESULT err=0 tag=101 nentries=1 >> etime=0 notes=U >> [01/Dec/2015:10:51:22 +0100] conn=12466 op=2 UNBIND >> >> and >> >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=0 BIND dn="" method=sasl >> version=3 mech=GSSAPI >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=0 RESULT err=14 tag=97 nentries=0 >> etime=0, SASL bind in progress >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=1 BIND dn="" method=sasl >> version=3 mech=GSSAPI >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=1 RESULT err=14 tag=97 nentries=0 >> etime=0, SASL bind in progress >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=2 BIND dn="" method=sasl >> version=3 mech=GSSAPI >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=2 RESULT err=0 tag=97 nentries=0 >> etime=0 dn="uid=abokovoy,cn=users,cn=accounts,dc=vda,dc=li" >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=3 SRCH >> base="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" scope=2 >> filter="(objectClass=*)" attrs="userPassword" >> authzid="uid=admin,cn=users,cn=accounts,dc=vda,dc=li" >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=3 RESULT err=0 tag=101 nentries=0 >> etime=0 notes=U >> [01/Dec/2015:10:48:44 +0100] conn=12465 op=4 UNBIND >> > Ludwig confirmed that ACL plugin short-circuits DM access before > proxy auth checked. I've filed a bug about it: > https://fedorahosted.org/389/ticket/48366 I just tested a new DS version from https://copr.fedoraproject.org/coprs/lkrispen/389-ds-current/build/146702/ and it works! Nice! -- Petr^2 Spacek From mbasti at redhat.com Tue Dec 22 15:06:32 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 22 Dec 2015 16:06:32 +0100 Subject: [Freeipa-devel] [PATCH 0077] ipa-dns-install: Do not check for zone overlap when DNS, installed. In-Reply-To: <56795121.7060804@redhat.com> References: <5673DF56.8060906@redhat.com> <5673E832.3020609@redhat.com> <56741AA8.5060807@redhat.com> <56793EC0.3010107@redhat.com> <56795121.7060804@redhat.com> Message-ID: <567966F8.9090900@redhat.com> On 22.12.2015 14:33, Petr Spacek wrote: > On 22.12.2015 13:14, David Kupka wrote: >> On 18/12/15 15:39, Petr Vobornik wrote: >>> On 12/18/2015 12:04 PM, Petr Vobornik wrote: >>>> On 12/18/2015 11:26 AM, David Kupka wrote: >>>>> Standalone DNS installer always performed overlap check effectively >>>>> preventing installation on replica when other DNS instance was already >>>>> installed in topology. >>>>> >>>>> >>>> I don't like the position of api argument in the install_check. It is >>>> not consistent with install_check in KRA plus it's between two related >>>> options: standalone and replica. >>>> >>>> Is there a reason behind it which I don't see? >>> NACK, the API call is incorrect. >>> >>> It should be api.Command['dns_is_enabled']()['result'] or >>> api.Command.dns_is_enabled()['result'] >>> >>> >>> ipa-dns-install --forwarder=$DNS_FORWARDER >>> >>> The log file for this installation can be found in >>> /var/log/ipaserver-install.log >>> Unexpected error - see /var/log/ipaserver-install.log for details: >>> TypeError: __call__() takes exactly 1 argument (2 given) >>> >>> >>> log file: >>> >>> 2015-12-18T14:33:30Z DEBUG File >>> "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", >>> line 736, in run_script >>> return_value = main_function() >>> >>> File "/sbin/ipa-dns-install", line 137, in main >>> dns_installer.install_check(True, api, False, options, >>> hostname=api.env.host) >>> >>> File "/usr/lib/python2.7/site-packages/ipaserver/install/dns.py", >>> line 113, in install_check >>> already_enabled = api.Command('dns_is_enabled')['result'] >>> >>> 2015-12-18T14:33:30Z DEBUG The ipa-dns-install command failed, >>> exception: TypeError: __call__() takes exactly 1 argument (2 given) >>> >> Thanks for catch. Sometimes I mess up with patch files. Fixed patch attached. > ACK, works for me. > Pushed to master: 8ad39a974fd4e1cc6fd2a09b1d9d0c8724983b15 Pushed to ipa-4-3: bfad829586f40e20ada21ea932377089901aaf61 From pspacek at redhat.com Tue Dec 22 15:19:10 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 22 Dec 2015 16:19:10 +0100 Subject: [Freeipa-devel] [PATCH 0077-8] ipa-replica-prepare: Add '--auto-reverse' and, '--allow-zone-overlap' options In-Reply-To: <56795AD3.7010102@redhat.com> References: <56795AD3.7010102@redhat.com> Message-ID: <567969EE.3000001@redhat.com> On 22.12.2015 15:14, David Kupka wrote: > https://fedorahosted.org/freeipa/ticket/5563 NACK: $ ipa-replica-prepare vm-058-106.abc.idm.lab.eng.brq.redhat.com --ip-address=10.34.58.106 --reverse-zone=58.34.10.in-addr.arpa. Directory Manager (existing master) password: Checking DNS domain 58.34.10.in-addr.arpa., please wait ... Values instance has no attribute 'unattended' The ipa-replica-prepare command failed. -- Petr^2 Spacek From pvoborni at redhat.com Tue Dec 22 15:31:28 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 22 Dec 2015 16:31:28 +0100 Subject: [Freeipa-devel] 4.3 on rawhide build task fail Message-ID: <56796CD0.5010905@redhat.com> Build of 4.3 on Fedora rawhide failed at the end on rpmdiff check. Builds for all arches were successful and also works in COPR. 0 free 1 open 4 done 0 failed 12284450 build (rawhide, /freeipa:b2442d51ba3f2a5f907f72e9bd90c5889bd89c0e): open (buildppcle-07.phx2.fedoraproject.org) -> FAILED: BuildError: mismatch when analyzing python3-ipatests-4.3.0-1.fc24.noarch.rpm, rpmdiff output was: error: cannot open Packages index using db5 - Permission denied (13) error: cannot open Packages database in /var/lib/rpm error: cannot open Packages database in /var/lib/rpm removed REQUIRES python3-ipalib(armv7hl-32) = 4.3.0-1.fc24 added REQUIRES python3-ipalib(x86-64) = 4.3.0-1.fc24 0 free 0 open 4 done 1 failed Rebuild ended up with the same issue. Do we have something wrong in spec file? Or the "Permission denied" is the cause and the issue might be in Fedora infra? Any ideas? Full output: $ fedpkg build Building freeipa-4.3.0-1.fc24 for rawhide Created task: 12284450 Task info: http://koji.fedoraproject.org/koji/taskinfo?taskID=12284450 Watching tasks (this may be safely interrupted)... 12284450 build (rawhide, /freeipa:b2442d51ba3f2a5f907f72e9bd90c5889bd89c0e): free 12284450 build (rawhide, /freeipa:b2442d51ba3f2a5f907f72e9bd90c5889bd89c0e): free -> open (buildppcle-07.phx2.fedoraproject.org) 12284454 buildSRPMFromSCM (/freeipa:b2442d51ba3f2a5f907f72e9bd90c5889bd89c0e): free 12284454 buildSRPMFromSCM (/freeipa:b2442d51ba3f2a5f907f72e9bd90c5889bd89c0e): free -> open (buildhw-08.phx2.fedoraproject.org) 12284454 buildSRPMFromSCM (/freeipa:b2442d51ba3f2a5f907f72e9bd90c5889bd89c0e): open (buildhw-08.phx2.fedoraproject.org) -> closed 0 free 1 open 1 done 0 failed 12284512 buildArch (freeipa-4.3.0-1.fc24.src.rpm, armv7hl): free 12284514 buildArch (freeipa-4.3.0-1.fc24.src.rpm, i686): free 12284513 buildArch (freeipa-4.3.0-1.fc24.src.rpm, x86_64): open (buildhw-05.phx2.fedoraproject.org) 12284514 buildArch (freeipa-4.3.0-1.fc24.src.rpm, i686): free -> open (buildvm-26.phx2.fedoraproject.org) 12284512 buildArch (freeipa-4.3.0-1.fc24.src.rpm, armv7hl): free -> open (arm04-builder19.arm.fedoraproject.org) 12284514 buildArch (freeipa-4.3.0-1.fc24.src.rpm, i686): open (buildvm-26.phx2.fedoraproject.org) -> closed 0 free 3 open 2 done 0 failed 12284513 buildArch (freeipa-4.3.0-1.fc24.src.rpm, x86_64): open (buildhw-05.phx2.fedoraproject.org) -> closed 0 free 2 open 3 done 0 failed 12284512 buildArch (freeipa-4.3.0-1.fc24.src.rpm, armv7hl): open (arm04-builder19.arm.fedoraproject.org) -> closed 0 free 1 open 4 done 0 failed 12284450 build (rawhide, /freeipa:b2442d51ba3f2a5f907f72e9bd90c5889bd89c0e): open (buildppcle-07.phx2.fedoraproject.org) -> FAILED: BuildError: mismatch when analyzing python3-ipatests-4.3.0-1.fc24.noarch.rpm, rpmdiff output was: error: cannot open Packages index using db5 - Permission denied (13) error: cannot open Packages database in /var/lib/rpm error: cannot open Packages database in /var/lib/rpm removed REQUIRES python3-ipalib(armv7hl-32) = 4.3.0-1.fc24 added REQUIRES python3-ipalib(x86-64) = 4.3.0-1.fc24 0 free 0 open 4 done 1 failed 12284450 build (rawhide, /freeipa:b2442d51ba3f2a5f907f72e9bd90c5889bd89c0e) failed -- Petr Vobornik From mbasti at redhat.com Tue Dec 22 15:45:13 2015 From: mbasti at redhat.com (Martin Basti) Date: Tue, 22 Dec 2015 16:45:13 +0100 Subject: [Freeipa-devel] [PATCH 0374-0375, 395] Fix permissions on newly created directories In-Reply-To: <566ECAFD.8010108@redhat.com> References: <566989B2.2080502@redhat.com> <566B13A7.4090107@redhat.com> <566ECAFD.8010108@redhat.com> Message-ID: <56797009.9010304@redhat.com> On 14.12.2015 14:58, Tomas Babej wrote: > > On 12/11/2015 07:19 PM, Martin Basti wrote: >> >> On 10.12.2015 15:18, Martin Basti wrote: >>> Hello, >>> >>> patch 0374 fixes the ticket, but I found more issues with directory >>> permission, I fixed them in 0375 >>> >>> https://fedorahosted.org/freeipa/ticket/5520 >>> >>> Patches attached. >> Patches attached. > > ACK, works as expected. > > Pushed to master: 4272ba40ea909b1f783a6fada5b1eebb6efbdf93 I found issue, there was an extra mkdir instead of chmod, patch pushed under one-liner rule Pushed to: master: 403652b5b9cc5e80869019067cbe1424b5c02bd1 ipa-4-3: f3a8ef0601b21fe4ddace08d8fd2b7a821efd658 ipa-4-2: 066ecf466403e4e321253238d28a235c7bcc0cd6 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mbasti-0395-Fix-replace-mkdir-with-chmod.patch Type: text/x-patch Size: 1032 bytes Desc: not available URL: From pspacek at redhat.com Tue Dec 22 15:50:19 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 22 Dec 2015 16:50:19 +0100 Subject: [Freeipa-devel] [PATCH 0069] Add 'review' target for make In-Reply-To: <567149E9.1040506@redhat.com> References: <56683B03.40703@redhat.com> <5669B09B.9000001@redhat.com> <566AE2B3.3010201@redhat.com> <56712496.5060907@redhat.com> <5671268F.8020302@redhat.com> <567128B2.80009@redhat.com> <567139D0.7050807@redhat.com> <5671448D.1050102@redhat.com> <567149E9.1040506@redhat.com> Message-ID: <5679713B.6060203@redhat.com> On 16.12.2015 12:24, Martin Kosek wrote: > On 12/16/2015 12:01 PM, Petr Spacek wrote: >> On 16.12.2015 11:15, Martin Kosek wrote: >>> On 12/16/2015 10:02 AM, Petr Spacek wrote: >>>> On 16.12.2015 09:53, Jan Cholasta wrote: >>>>> On 16.12.2015 09:45, Petr Spacek wrote: >>>>>> On 11.12.2015 15:50, Jan Cholasta wrote: >>>>>>> Hi, >>>>>>> >>>>>>> On 10.12.2015 18:04, Petr Spacek wrote: >>>>>>>> On 9.12.2015 15:30, Petr Spacek wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> this patch automates some of sanity checks proposed by Petr Vobornik. >>>>>>>>> >>>>>>>>> 'make review' should be used in root of clean Git tree which has patches >>>>>>>>> under >>>>>>>>> review applied. >>>>>>>>> >>>>>>>>> Magic in review.sh attempts to detect nearest remote branch which can be >>>>>>>>> used >>>>>>>>> as diff base for review. Please see review.sh for further details. >>>>>>>> >>>>>>>> And here is the patch! :-) >>>>>>> >>>>>>> Nice, but I would rather see this in ipatool >>>>>>> (). Or is there any obvious benefit >>>>>>> in having this in freeipa itself that I'm missing? >>>>>> >>>>>> For me the obvious benefit is: >>>>>> git clone >>>>>> git am >>>>>> make review >>>>>> >>>>>> Done. >>>>>> >>>>>> No need to find & learn other tool, no risk of using wrong version of the tool >>>>>> to wrong version of source tree etc. >>>>> >>>>> AFAIK all IPA developers are supposed to use ipatool, and it already has a >>>> >>>> Good to know. How does a newcomer learn about it? Honestly I never used >>>> ipatool (or not even cloned it). >>> >>> ipatool targets rather upstream members with write access, so they are hardly >>> newcomers. >> >> I though that we want to make it easy to contribute, so why are you talking >> about core developers? > > I was mostly refering to ipatool's "push" command that I mostly used, but it's > true there is also "start-review" or "am" commands that could be useful to > others too. > >> Shouldn't we make it easy to self-review own patches for everyone? Including >> random people who want to submit few patches and go away? (Think how we can >> apply usability principles to development.) > > We should, I hope my reply did not suggest otherwise. Hmm, so I finally tried ipatool and it blew up: $ ../tool/ipatool --help Configuration is specified in the file given by --config. Run ipatool sample-config to see what needs to go in it. $ ../tool/ipatool sample-config Traceback (most recent call last): File "../tool/ipatool", line 701, in Context(docopt.docopt(__doc__)).run() File "../tool/ipatool", line 233, in __init__ with open(os.path.expanduser(options['--config'])) as conf_file: FileNotFoundError: [Errno 2] No such file or directory: '/home/pspacek/.ipa/toolconf.yaml' ... after a while I found out that it is too heavy Python magic inside (for me as non-Pythonist) so I'm giving up. Moreover it AFAIK does not work on tree with applied patches (which is normal situation before you send the patched to list for review), so I'm not even sure how I could integrate the review scripts into it. For now I'm going to live with my review.sh scripts in following Git repo: https://github.com/pspacek/ipareview.git Pull requests or rewrites for ipatool are welcome. You can take this as a Christmas present :-) -- Petr^2 Spacek From pspacek at redhat.com Tue Dec 22 15:56:23 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 22 Dec 2015 16:56:23 +0100 Subject: [Freeipa-devel] [PATCH 0394, 0379 - 0393] Winter code cleanup In-Reply-To: <56795BC7.6070001@redhat.com> References: <56720BCC.1050504@redhat.com> <56725C34.5060400@redhat.com> <5672B631.7030709@redhat.com> <5678F671.8080707@redhat.com> <5679458D.9090207@redhat.com> <56795BC7.6070001@redhat.com> Message-ID: <567972A7.8090308@redhat.com> On 22.12.2015 15:18, Martin Basti wrote: > > > On 22.12.2015 13:43, Martin Basti wrote: >> >> >> On 22.12.2015 08:06, Jan Cholasta wrote: >>> On 17.12.2015 14:18, Martin Basti wrote: >>>> >>>> >>>> On 17.12.2015 07:54, Jan Cholasta wrote: >>>>> Hi, >>>>> >>>>> On 17.12.2015 02:11, Martin Basti wrote: >>>>>> Hello, >>>>>> following patches cleanup the code and add checks to pylint to avoid the >>>>>> mess again >>>>>> >>>>>> Patches: 379-381: >>>>>> the most important patches, they cleanup imports >>>>>> >>>>>> Patch 382: >>>>>> enables various pylint checks, we may reduce the list of check if >>>>>> needed. >>>>> >>>>> Would it be possible to turn this into a blacklist of disabled >>>>> warnings rather than a whitelist of enabled warnings? >>>>> >>>>>> >>>>>> Patches 383 - 393: >>>>>> remove minor issues from code, and enable particular checks >>>>>> Feel free to nack patches/checks that should not be there. >>>>>> >>>>>> Please apply patches in order. >>>>>> >>>>>> Martin^2 >>>>>> >>>>>> >>>>> >>>>> Honza >>>>> >>>> Updated patches attached. >>> >>> Patch 379: ACK >>> >>> >>> Patch 380: >>> >>> 1) This patch needs to be rebased. >>> >>> >>> 2) make-lint is reporting this to me: >>> >>> ************* Module ipatests.test_cmdline.test_ipagetkeytab >>> ipatests/test_cmdline/test_ipagetkeytab.py:30: [W0611(unused-import), ] >>> Unused import nose) >>> ipatests/test_cmdline/test_ipagetkeytab.py:34: [W0611(unused-import), ] >>> Unused DN imported from ipapython.dn) >>> ************* Module ipatests.test_cmdline.test_help >>> ipatests/test_cmdline/test_help.py:21: [W0611(unused-import), ] Unused >>> import contextlib) >>> ipatests/test_cmdline/test_help.py:23: [W0611(unused-import), ] Unused >>> assert_raises imported from nose.tools) >>> ************* Module ipatests.test_cmdline.test_cli >>> ipatests/test_cmdline/test_cli.py:11: [W0611(unused-import), ] Unused >>> API_VERSION imported from ipapython.version) >>> ************* Module ipaplatform.services >>> ipaplatform/services.py:59: [W0611(unused-import), ] Unused >>> timedate_services imported from ipaplatform.redhat.services) >>> ************* Module ipaplatform.rhel.services >>> ipaplatform/rhel/services.py:59: [W0611(unused-import), ] Unused >>> timedate_services imported from ipaplatform.redhat.services) >>> ************* Module ipaplatform.redhat.services >>> ipaplatform/redhat/services.py:306: [W0611(unused-import), ] Unused >>> timedate_services imported from ipaplatform.base.services) >>> ************* Module ipaplatform.fedora.services >>> ipaplatform/fedora/services.py:59: [W0611(unused-import), ] Unused >>> timedate_services imported from ipaplatform.redhat.services) >>> ************* Module ipalib.setup >>> ipalib/setup.py:28: [W0611(unused-import), ] Unused import >>> distutils.sysconfig) >>> >>> Note that the "from ipaplatform.${PLATFORM}.services import >>> timedate_services" in services.py should be rewritten to "timedate_services >>> = ${PLATFORM}_services.timedate_services" to fix this. >>> >>> >>> Patch 381: ACK >>> >>> >>> Patch 382: >>> >>> Nitpick: according to , the order of >>> message categories is F, E, W, C, R from the most severe to the least >>> severe (it does not mention I, but I think it should be last, after R), IMO >>> we should keep this order here as well for clarity. (Don't worry, doing >>> this should not require rebasing the following patches.) >>> >>> >>> Patch 383 - 387: ACK >>> >>> >>> Patch 388: >>> >>> IMO it would be better to rewrite this: >>> >>> [(t.id, t.options) for t in doc.getKeyPackages()] # pylint: >>> disable=expression-not-assigned >>> >>> into this: >>> >>> for t in doc.getKeyPackages(): >>> t._PSKCKeyPackage__process() >>> >>> rather than disabling the check. >>> >>> >>> Patch 389: >>> >>> All this patch does is that it enables a check globally and then disables >>> it everywhere locally. >>> >>> IMO the patch should be retired for now, or make-lint should be taught to >>> automatically skip the check inside TestCase.assertRaises() context. >>> >>> >>> Patch 390: ACK >>> >>> >>> Patch 391: >>> >>> default_from uses argument names of the specified function as param names >>> from which to create the default. >>> >>> These changes break default_from: >>> >>> - default_from=lambda name_from_ip: >>> _reverse_zone_name(name_from_ip), >>> + default_from=_reverse_zone_name, >>> >>> - default_from=lambda idnsname: >>> default_zone_update_policy(idnsname), >>> + default_from=default_zone_update_policy, >>> >>> This change adds dependency on non-existent param: >>> >>> - default_from=lambda: krb_utils.get_principal(), >>> + default_from=krb_utils.get_principal, >>> >>> The result of this check is misleading for default_from, so IMO the patch >>> should be retired for now, or make-lint should be taught to automatically >>> disable the check for the default_from argument in param definitions. >>> >>> >>> Patch 392: >>> >>> I think the existing occurences of exec()/eval() should be removed before >>> this check can be enabled. >>> >>> >>> Patch 393: ACK >>> >>> >>> I would like to see a patch which enables the useless-suppression info >>> message, so that we can catch dangling "# pylint: disable=" comments (there >>> are some in the code). >>> >>> (Also, it would be nice to finally rewrite make-lint to pylint config file >>> + plugins.) >>> >> Updated patches attached, please apply patch 394 first. >> >> Patches 389, 391, 392 have been removed, issues will be addressed later. >> >> useless-suppression, unbalanced-tuple-unpacking, unpacking-non-sequence, and >> python3 checks are on my TODO list >> >> I can try to rewrite pylint to plugins and config file. >> >> Martin^2 >> >> > I forgot to attach patch 379, all patches again (394 should go first) I just tested the patches and functionaly-wise it seems okay. -- Petr^2 Spacek From pspacek at redhat.com Tue Dec 22 15:59:00 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 22 Dec 2015 16:59:00 +0100 Subject: [Freeipa-devel] [PATCH] 811 performance: faster DN implementation In-Reply-To: <552D4F3A.90705@redhat.com> References: <551A72DB.3020105@redhat.com> <551D11C2.4070902@redhat.com> <552668E0.4050601@redhat.com> <5527AFEF.3060704@redhat.com> <552CC294.6030908@redhat.com> <552D4F3A.90705@redhat.com> Message-ID: <56797344.3000108@redhat.com> On 14.4.2015 19:32, Petr Vobornik wrote: > > Pushed to master: 11bd9d96f191066f7ba760549f00179c128a9787 > >> >> Please be so kind and fix naming. AFAIK the patch refers to 'openldap' DN >> format but in fact it is python-ldap-ishm. It will surely confuse next >> generation of FreeIPA developers :-) >> > > Will be in separate patch. Petr, Santa found out that you did not send the patch! You are risking Christmas without any presents ... :-) -- Petr^2 Spacek From jdennis at redhat.com Tue Dec 22 16:18:01 2015 From: jdennis at redhat.com (John Dennis) Date: Tue, 22 Dec 2015 11:18:01 -0500 Subject: [Freeipa-devel] ipa-devel repos on jdennis.fedorapeople.org In-Reply-To: <56796323.1000303@redhat.com> References: <55A532DF.7070302@redhat.com> <55A53D93.4050201@redhat.com> <20150715074450.GL4218@redhat.com> <55DEC9F5.4070302@redhat.com> <20150827083452.GN22106@redhat.com> <56796323.1000303@redhat.com> Message-ID: <567977B9.9020907@redhat.com> On 12/22/2015 09:50 AM, Petr Spacek wrote: > John, the machines which used to generate the repos are basically dead now. > > Could you remove the directories and replace them with a README with sentence > that the repos were discontinued, please? I'd be happy to, but shouldn't the README contain a pointer to the preferred repo/copr whatever it is now? What is it? -- John From pspacek at redhat.com Tue Dec 22 16:19:55 2015 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 22 Dec 2015 17:19:55 +0100 Subject: [Freeipa-devel] ipa-devel repos on jdennis.fedorapeople.org In-Reply-To: <567977B9.9020907@redhat.com> References: <55A532DF.7070302@redhat.com> <55A53D93.4050201@redhat.com> <20150715074450.GL4218@redhat.com> <55DEC9F5.4070302@redhat.com> <20150827083452.GN22106@redhat.com> <56796323.1000303@redhat.com> <567977B9.9020907@redhat.com> Message-ID: <5679782B.4040601@redhat.com> On 22.12.2015 17:18, John Dennis wrote: > On 12/22/2015 09:50 AM, Petr Spacek wrote: >> John, the machines which used to generate the repos are basically dead now. >> >> Could you remove the directories and replace them with a README with sentence >> that the repos were discontinued, please? > > I'd be happy to, but shouldn't the README contain a pointer to the preferred > repo/copr whatever it is now? What is it? AFAIK we do not have nightly builds yet :-( Petr, could we have some nightly or post-commit builds in Fedora Copr? -- Petr^2 Spacek From pvoborni at redhat.com Tue Dec 22 16:37:51 2015 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 22 Dec 2015 17:37:51 +0100 Subject: [Freeipa-devel] ipa-devel repos on jdennis.fedorapeople.org In-Reply-To: <5679782B.4040601@redhat.com> References: <55A532DF.7070302@redhat.com> <55A53D93.4050201@redhat.com> <20150715074450.GL4218@redhat.com> <55DEC9F5.4070302@redhat.com> <20150827083452.GN22106@redhat.com> <56796323.1000303@redhat.com> <567977B9.9020907@redhat.com> <5679782B.4040601@redhat.com> Message-ID: <56797C5F.7040800@redhat.com> On 12/22/2015 05:19 PM, Petr Spacek wrote: > On 22.12.2015 17:18, John Dennis wrote: >> On 12/22/2015 09:50 AM, Petr Spacek wrote: >>> John, the machines which used to generate the repos are basically dead now. >>> >>> Could you remove the directories and replace them with a README with sentence >>> that the repos were discontinued, please? >> >> I'd be happy to, but shouldn't the README contain a pointer to the preferred >> repo/copr whatever it is now? What is it? > > AFAIK we do not have nightly builds yet :-( > > Petr, could we have some nightly or post-commit builds in Fedora Copr? > It is easy to setup a vm for nightly copr builds. We should investigate a possibility of getting a project copr account. It is not good to build everything in mkosek's namespace. -- Petr Vobornik From jcholast at redhat.com Wed Dec 23 07:00:09 2015 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 23 Dec 2015 08:00:09 +0100 Subject: [Freeipa-devel] [PATCH 0394, 0379 - 0393] Winter code cleanup In-Reply-To: <56795BC7.6070001@redhat.com> References: <56720BCC.1050504@redhat.com> <56725C34.5060400@redhat.com> <5672B631.7030709@redhat.com> <5678F671.8080707@redhat.com> <5679458D.9090207@redhat.com> <56795BC7.6070001@redhat.com> Message-ID: <567A4679.9080208@redhat.com> On 22.12.2015 15:18, Martin Basti wrote: > > > On 22.12.2015 13:43, Martin Basti wrote: >> >> >> On 22.12.2015 08:06, Jan Cholasta wrote: >>> On 17.12.2015 14:18, Martin Basti wrote: >>>> >>>> >>>> On 17.12.2015 07:54, Jan Cholasta wrote: >>>>> Hi, >>>>> >>>>> On 17.12.2015 02:11, Martin Basti wrote: >>>>>> Hello, >>>>>> following patches cleanup the code and add checks to pylint to >>>>>> avoid the >>>>>> mess again >>>>>> >>>>>> Patches: 379-381: >>>>>> the most important patches, they cleanup imports >>>>>> >>>>>> Patch 382: >>>>>> enables various pylint checks, we may reduce the list of check if >>>>>> needed. >>>>> >>>>> Would it be possible to turn this into a blacklist of disabled >>>>> warnings rather than a whitelist of enabled warnings? >>>>> >>>>>> >>>>>> Patches 383 - 393: >>>>>> remove minor issues from code, and enable particular checks >>>>>> Feel free to nack patches/checks that should not be there. >>>>>> >>>>>> Please apply patches in order. >>>>>> >>>>>> Martin^2 >>>>>> >>>>>> >>>>> >>>>> Honza >>>>> >>>> Updated patches attached. >>> >>> Patch 379: ACK >>> >>> >>> Patch 380: >>> >>> 1) This patch needs to be rebased. >>> >>> >>> 2) make-lint is reporting this to me: >>> >>> ************* Module ipatests.test_cmdline.test_ipagetkeytab >>> ipatests/test_cmdline/test_ipagetkeytab.py:30: [W0611(unused-import), >>> ] Unused import nose) >>> ipatests/test_cmdline/test_ipagetkeytab.py:34: [W0611(unused-import), >>> ] Unused DN imported from ipapython.dn) >>> ************* Module ipatests.test_cmdline.test_help >>> ipatests/test_cmdline/test_help.py:21: [W0611(unused-import), ] >>> Unused import contextlib) >>> ipatests/test_cmdline/test_help.py:23: [W0611(unused-import), ] >>> Unused assert_raises imported from nose.tools) >>> ************* Module ipatests.test_cmdline.test_cli >>> ipatests/test_cmdline/test_cli.py:11: [W0611(unused-import), ] Unused >>> API_VERSION imported from ipapython.version) >>> ************* Module ipaplatform.services >>> ipaplatform/services.py:59: [W0611(unused-import), ] Unused >>> timedate_services imported from ipaplatform.redhat.services) >>> ************* Module ipaplatform.rhel.services >>> ipaplatform/rhel/services.py:59: [W0611(unused-import), ] Unused >>> timedate_services imported from ipaplatform.redhat.services) >>> ************* Module ipaplatform.redhat.services >>> ipaplatform/redhat/services.py:306: [W0611(unused-import), ] Unused >>> timedate_services imported from ipaplatform.base.services) >>> ************* Module ipaplatform.fedora.services >>> ipaplatform/fedora/services.py:59: [W0611(unused-import), ] Unused >>> timedate_services imported from ipaplatform.redhat.services) >>> ************* Module ipalib.setup >>> ipalib/setup.py:28: [W0611(unused-import), ] Unused import >>> distutils.sysconfig) >>> >>> Note that the "from ipaplatform.${PLATFORM}.services import >>> timedate_services" in services.py should be rewritten to >>> "timedate_services = ${PLATFORM}_services.timedate_services" to fix >>> this. >>> >>> >>> Patch 381: ACK >>> >>> >>> Patch 382: >>> >>> Nitpick: according to , the order >>> of message categories is F, E, W, C, R from the most severe to the >>> least severe (it does not mention I, but I think it should be last, >>> after R), IMO we should keep this order here as well for clarity. >>> (Don't worry, doing this should not require rebasing the following >>> patches.) >>> >>> >>> Patch 383 - 387: ACK >>> >>> >>> Patch 388: >>> >>> IMO it would be better to rewrite this: >>> >>> [(t.id, t.options) for t in doc.getKeyPackages()] # >>> pylint: disable=expression-not-assigned >>> >>> into this: >>> >>> for t in doc.getKeyPackages(): >>> t._PSKCKeyPackage__process() >>> >>> rather than disabling the check. >>> >>> >>> Patch 389: >>> >>> All this patch does is that it enables a check globally and then >>> disables it everywhere locally. >>> >>> IMO the patch should be retired for now, or make-lint should be >>> taught to automatically skip the check inside TestCase.assertRaises() >>> context. >>> >>> >>> Patch 390: ACK >>> >>> >>> Patch 391: >>> >>> default_from uses argument names of the specified function as param >>> names from which to create the default. >>> >>> These changes break default_from: >>> >>> - default_from=lambda name_from_ip: >>> _reverse_zone_name(name_from_ip), >>> + default_from=_reverse_zone_name, >>> >>> - default_from=lambda idnsname: >>> default_zone_update_policy(idnsname), >>> + default_from=default_zone_update_policy, >>> >>> This change adds dependency on non-existent param: >>> >>> - default_from=lambda: krb_utils.get_principal(), >>> + default_from=krb_utils.get_principal, >>> >>> The result of this check is misleading for default_from, so IMO the >>> patch should be retired for now, or make-lint should be taught to >>> automatically disable the check for the default_from argument in >>> param definitions. >>> >>> >>> Patch 392: >>> >>> I think the existing occurences of exec()/eval() should be removed >>> before this check can be enabled. >>> >>> >>> Patch 393: ACK >>> >>> >>> I would like to see a patch which enables the useless-suppression >>> info message, so that we can catch dangling "# pylint: disable=" >>> comments (there are some in the code). >>> >>> (Also, it would be nice to finally rewrite make-lint to pylint config >>> file + plugins.) >>> >> Updated patches attached, please apply patch 394 first. >> >> Patches 389, 391, 392 have been removed, issues will be addressed later. OK. >> >> useless-suppression, unbalanced-tuple-unpacking, >> unpacking-non-sequence, and python3 checks are on my TODO list >> >> I can try to rewrite pylint to plugins and config file. Good! >> >> Martin^2 >> >> > I forgot to attach patch 379, all patches again (394 should go first) Thanks, ACK. Pushed to master: 00fd28e02689d49f89429b663b16ce4ca484e7d6 -- Jan Cholasta From akasurde at redhat.com Mon Dec 28 07:06:12 2015 From: akasurde at redhat.com (Abhijeet Kasurde) Date: Mon, 28 Dec 2015 12:36:12 +0530 Subject: [Freeipa-devel] Fixed login error message box in LoginScreen page Message-ID: <5680DF64.4060808@redhat.com> Hi All, Please review patches attached. Thanks, Abhijeet Kasurde -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-akasurde-0001-Fixed-login-error-message-box-in-LoginScreen-page-ipa-4-2.patch Type: text/x-patch Size: 1510 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-akasurde-0001-Fixed-login-error-message-box-in-LoginScreen-page.patch Type: text/x-patch Size: 1509 bytes Desc: not available URL: