From jcholast at redhat.com Tue Nov 1 12:28:02 2011 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 01 Nov 2011 13:28:02 +0100 Subject: [Freeipa-devel] [PATCH] 54 Fix attempted write to attribute of read-only object In-Reply-To: <20111014081859.GB8783@redhat.com> References: <4E96B5AB.4030800@redhat.com> <4E970565.6040009@redhat.com> <4E97EC5B.90309@redhat.com> <20111014081859.GB8783@redhat.com> Message-ID: <4EAFE5D2.4010607@redhat.com> Dne 14.10.2011 10:19, Alexander Bokovoy napsal(a): > On Fri, 14 Oct 2011, Jan Cholasta wrote: >> Perform an HTTP request. >> """ >> - if self.ca_host == None: >> - self.ca_host = self._select_ca() >> + if self.ca_host is None: >> + object.__setattr__(self, 'ca_host', self._select_ca()) >> return dogtag.http_request(self.ca_host, port, url, **kw) > I don't like this approach as well. A better way would be to have a > class CaCache that is mutable and allow changing its properties. Then > you would create an instance of CaCache in ca.__init__() and ask for > its properties later. I don't like the idea of introducing a new class every time we need a ReadOnly attribute to be writable. There's quite a few places in the code where we do object.__setattr__ on ReadOnly objects. IMO the right thing to do would be to add means of whitelisting ReadOnly attributes that need to stay writable after locking. > > You can move those _select_ca(), _select_any_master(), > _host_has_service() to CaCache as they seem to not depend on anything > in class ca but rather use global api.env. > > This way you will get is a fairly simple CaCache class reusable both > in ca and ra classes. Honza -- Jan Cholasta From pvoborni at redhat.com Tue Nov 1 12:37:48 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 01 Nov 2011 13:37:48 +0100 Subject: [Freeipa-devel] [PATCH] 029 Page is cleared before it is visible In-Reply-To: <4EAF2365.8000005@redhat.com> References: <4EA92AF7.8080609@redhat.com> <4EAF2365.8000005@redhat.com> Message-ID: <4EAFE81C.4030004@redhat.com> On 10/31/2011 11:38 PM, Endi Sukma Dewata wrote: > On 10/27/2011 4:57 AM, Petr Vobornik wrote: >> https://fedorahosted.org/freeipa/ticket/1459 > Some issues: > > 1. The new clear() method is called during refresh(), so the facet with > the old data is still shown for a brief moment before it's cleared. > The clear() should be called before show(). However, if the pkey/filter > is unchanged (check using needs_update()) we just need to show() the > facet, no need to clear() and refresh() again. The above logic needs to > be fixed. Changed. > > The we will need to override the needs_update() for search and > association facets because the default one always returns true. Done > 2. The following code in association.js and search.js will call clear() > if there's no old pkey/filter, is this intentional? No old pkey/filter > means the page is just loaded, so it probably doesn't need clearing. > > if (!old_pkey || old_pkey[0] !== pkey[0]) { > that.clear(); > } > > if (!old_filter || old_filter[0] !== filter[0]) { > that.clear(); > } It seems unnecessary. But probably it was intentional (don't remember) - IDRC if there is a widget - maybe keytab or certificate status, which has some default state worth cleaning. Anyway in current implementation this logic is part of need_update and it is a must. IMHO we should avoid implementing special need_cleaning method. Cleaning at first display doesn't do any harm and it is one less method to maintain in couple classes. > 3) Fixed bad implementation of clear method in radio_widget. 4) Changed direct/indirect radio names in association facets - radios form different facets were interfering. 5) Added ID generator, using in radio_widget, same reason as #4. 6) Added clearing of header in details facet and association facets - refreshes of member counts were confusing. 7) Removed setting that.pkey in create method in details, association facet (it broke need_update, didn't found purpose). 8) Maybe we should add a refresh button to search facet. It doesn't reflect concurrent usage. Refresh by 2 changes of filter doesn't seem nice. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0029-1-Page-is-cleared-before-it-is-visible.patch Type: text/x-patch Size: 16384 bytes Desc: not available URL: From ayoung at redhat.com Tue Nov 1 16:12:08 2011 From: ayoung at redhat.com (Adam Young) Date: Tue, 01 Nov 2011 12:12:08 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances Message-ID: <4EB01A58.6070705@redhat.com> We had a brief discussion on unifying the PKI and IPA Directory Server instances. Here are my notes from it. Please fill out the details and correct me if I've mis-stated anything below. Issues: 1. Both make changes to Config. One identified conflict is he configuration of the Uniqueness plugin 2. PKI uses Directory Manager. This is insecure. Can it use a differen, limited admin? 3. Index strategies are different 4. make sure we have a union of the required sets of plugins 5. PKI needs to set D.S. Default Name context 6. If PKI uses the IPA datastore for users, it needs to creat the user with all the right prerequisites (object class, defaults) 7. PKI puts users in groups using "member of" so that should still work for the IPA tree -------------- next part -------------- An HTML attachment was scrubbed... URL: From ayoung at redhat.com Tue Nov 1 16:34:51 2011 From: ayoung at redhat.com (Adam Young) Date: Tue, 01 Nov 2011 12:34:51 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <4EB01A58.6070705@redhat.com> References: <4EB01A58.6070705@redhat.com> Message-ID: <4EB01FAB.50700@redhat.com> On 11/01/2011 12:12 PM, Adam Young wrote: > > We had a brief discussion on unifying the PKI and IPA Directory Server > instances. Here are my notes from it. Please fill out the details > and correct me if I've mis-stated anything below. > > Issues: > > > 1. > > Both make changes to Config. One identified conflict is he > configuration of the Uniqueness plugin > > 2. > > PKI uses Directory Manager. This is insecure. Can it use a > differen, limited admin? > > 3. > > Index strategies are different > > 4. > > make sure we have a union of the required sets of plugins > > 5. > > PKI needs to set D.S. Default Name context > > 6. > > If PKI uses the IPA datastore for users, it needs to creat the > user with all the right prerequisites (object class, defaults) > > 7. > > PKI puts users in groups using "member of" so that should still > work for the IPA tree > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel One additional point: 8. make sure that Certificate Server and IPA upgrade mechanisms for DirSrv don't conflict -------------- next part -------------- An HTML attachment was scrubbed... URL: From rmeggins at redhat.com Tue Nov 1 16:40:22 2011 From: rmeggins at redhat.com (Richard Megginson) Date: Tue, 01 Nov 2011 12:40:22 -0400 (EDT) Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <4EB01A58.6070705@redhat.com> Message-ID: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> ----- Original Message ----- > > > > We had a brief discussion on unifying the PKI and IPA Directory > Server instances. Here are my notes from it. Please fill out the > details and correct me if I've mis-stated anything below. > > > Issues: > > > Do IPA and PKI use different suffixes? > > 1. > > Both make changes to Config. One identified conflict is he > configuration of the Uniqueness plugin It may be easy to enhance this plugin and other plugins to allow different configuration per subtree. > 2. > > PKI uses Directory Manager. This is insecure. Can it use a differen, > limited admin? Or use ldapi? I don't think ldapjdk can use ldapi. > 3. > > Index strategies are different Use a union? e.g. if ipa needs attribute "a" indexed for equality only, but PKI needs it indexed for presence and substring only, then we can just index it for eq, sub, and pres. > 4. > > make sure we have a union of the required sets of plugins > 5. > > PKI needs to set D.S. Default Name context What is this? > 6. > > If PKI uses the IPA datastore for users, it needs to creat the user > with all the right prerequisites (object class, defaults) If both PKI and IPA use structural objectclasses, we may have to create corresponding auxiliary objectclasses so that you can mix-in both sets of objectclasses while having only one structural objectclass per entry. > 7. > > PKI puts users in groups using ?member of? so that should still work > for the IPA tree > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From simo at redhat.com Tue Nov 1 16:43:49 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 01 Nov 2011 12:43:49 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <4EB01A58.6070705@redhat.com> References: <4EB01A58.6070705@redhat.com> Message-ID: <1320165829.7734.274.camel@willson.li.ssimo.org> On Tue, 2011-11-01 at 12:12 -0400, Adam Young wrote: > We had a brief discussion on unifying the PKI and IPA Directory Server > instances. Here are my notes from it. Please fill out the details > and correct me if I've mis-stated anything below. > > > Issues: > > > 1. Both make changes to Config. One identified conflict is he > configuration of the Uniqueness plugin > > 2. PKI uses Directory Manager. This is insecure. Can it use a > differen, limited admin? Not only insecure but we do not want necessarily trust PKI to touch stuff that is IPA specific. > > 3. Index strategies are different > > 4. make sure we have a union of the required sets of plugins > > 5. PKI needs to set D.S. Default Name context This is a DS/IPA feature, not a PKI feature. In an IPA install we will need to be able to tell DS that the IPA namingContext is the default one. > 6. If PKI uses the IPA datastore for users, it needs to creat > the user with all the right prerequisites (object class, > defaults) No it should never be allowed to create users, it should just use existing users I think. > 7. PKI puts users in groups using ?member of? so that should > still work for the IPA tree PKI is currently using groupOfUniqueUsers and uniqueMember. We will need it to use groupOfNames (as modified in 389DS to not require members) and use member (which will automatically create memberof attributes). Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Tue Nov 1 16:49:49 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 01 Nov 2011 12:49:49 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> Message-ID: <1320166189.7734.280.camel@willson.li.ssimo.org> On Tue, 2011-11-01 at 12:40 -0400, Richard Megginson wrote: > ----- Original Message ----- > > > > > > > > We had a brief discussion on unifying the PKI and IPA Directory > > Server instances. Here are my notes from it. Please fill out the > > details and correct me if I've mis-stated anything below. > > > > > > Issues: > > > > > > > > Do IPA and PKI use different suffixes? Currently not as we use completely separate instances, but we will be able to use different suffixes for some stuff. > > > > 1. > > > > Both make changes to Config. One identified conflict is he > > configuration of the Uniqueness plugin > > It may be easy to enhance this plugin and other plugins to allow different configuration per subtree. If we confirm this conflict this will become a requirement before we can proceed. > > 2. > > > > PKI uses Directory Manager. This is insecure. Can it use a differen, > > limited admin? > > Or use ldapi? I don't think ldapjdk can use ldapi. It's a matter of trust for me. I do not want to trust PKI to have free reign on all data. I want it to be confined to only what it needs. So we can use ldapi and user mapping, but we wouldn't map the user to DM anyway. > > 3. > > > > Index strategies are different > > Use a union? e.g. if ipa needs attribute "a" indexed for equality only, but PKI needs it indexed for presence and substring only, then we can just index it for eq, sub, and pres. The problem here is finding out and how to make sure pki vs ds/ipa install and upgrade scripts do not stomp on each other. > > 4. > > > > make sure we have a union of the required sets of plugins > > 5. > > > > PKI needs to set D.S. Default Name context > > What is this? See my other mail, we need DS to support setting defaultNamingContext in rootdse. > > 6. > > > > If PKI uses the IPA datastore for users, it needs to creat the user > > with all the right prerequisites (object class, defaults) > > If both PKI and IPA use structural objectclasses, we may have to create corresponding auxiliary objectclasses so that you can mix-in both sets of objectclasses while having only one structural objectclass per entry. The problem here is much bigger, PKI simply do not have enough information to create a proper IPA user, so it should not be allowed to. This is an example of why I want to tightly control through ACIs what PKI can do and prevent it from causing "issues". Simo. -- Simo Sorce * Red Hat, Inc * New York From ayoung at redhat.com Tue Nov 1 16:53:35 2011 From: ayoung at redhat.com (Adam Young) Date: Tue, 01 Nov 2011 12:53:35 -0400 Subject: [Freeipa-devel] [PATCH] 0293-Add-priority-to-pwpolicy-list Message-ID: <4EB0240F.1010809@redhat.com> -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-admiyo-0293-Add-priority-to-pwpolicy-list.patch Type: text/x-patch Size: 862 bytes Desc: not available URL: From alee at redhat.com Tue Nov 1 17:08:18 2011 From: alee at redhat.com (Ade Lee) Date: Tue, 01 Nov 2011 13:08:18 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <1320166189.7734.280.camel@willson.li.ssimo.org> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> Message-ID: <1320167298.17206.126.camel@localhost.localdomain> On Tue, 2011-11-01 at 12:49 -0400, Simo Sorce wrote: > On Tue, 2011-11-01 at 12:40 -0400, Richard Megginson wrote: > > ----- Original Message ----- > > > > > > > > > > > > We had a brief discussion on unifying the PKI and IPA Directory > > > Server instances. Here are my notes from it. Please fill out the > > > details and correct me if I've mis-stated anything below. > > > > > > > > > Issues: > > > > > > > > > > > > > Do IPA and PKI use different suffixes? > > Currently not as we use completely separate instances, but we will be > able to use different suffixes for some stuff. > I would suggest that if we use the same database, then we use different suffixes. For one thing, we will want to be able to set ACIs so that the information here is not publicly browsable. It will also be much easier to limit the pki users ability to touch the rest of the db, and visa versa. It also makes it less likely that upgrade scripts will stomp on each other. > > > > > > 1. > > > > > > Both make changes to Config. One identified conflict is he > > > configuration of the Uniqueness plugin > > > > It may be easy to enhance this plugin and other plugins to allow different configuration per subtree. > > If we confirm this conflict this will become a requirement before we can > proceed. > > > > 2. > > > > > > PKI uses Directory Manager. This is insecure. Can it use a differen, > > > limited admin? > > > > Or use ldapi? I don't think ldapjdk can use ldapi. > > It's a matter of trust for me. I do not want to trust PKI to have free > reign on all data. I want it to be confined to only what it needs. > > So we can use ldapi and user mapping, but we wouldn't map the user to DM > anyway. > > > > 3. > > > > > > Index strategies are different > > > > Use a union? e.g. if ipa needs attribute "a" indexed for equality only, but PKI needs it indexed for presence and substring only, then we can just index it for eq, sub, and pres. > > The problem here is finding out and how to make sure pki vs ds/ipa > install and upgrade scripts do not stomp on each other. > > > 4. > > > > > > make sure we have a union of the required sets of plugins > > > 5. > > > > > > PKI needs to set D.S. Default Name context > > > > What is this? > > See my other mail, we need DS to support setting defaultNamingContext in > rootdse. > > > > 6. > > > > > > If PKI uses the IPA datastore for users, it needs to creat the user > > > with all the right prerequisites (object class, defaults) > > > > If both PKI and IPA use structural objectclasses, we may have to create corresponding auxiliary objectclasses so that you can mix-in both sets of objectclasses while having only one structural objectclass per entry. > > The problem here is much bigger, PKI simply do not have enough > information to create a proper IPA user, so it should not be allowed to. > This is an example of why I want to tightly control through ACIs what > PKI can do and prevent it from causing "issues". > If we do this integration, then I'm OK with IPA creating the users. > > Simo. > From edewata at redhat.com Tue Nov 1 17:10:47 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 01 Nov 2011 12:10:47 -0500 Subject: [Freeipa-devel] [PATCH] 0293-Add-priority-to-pwpolicy-list In-Reply-To: <4EB0240F.1010809@redhat.com> References: <4EB0240F.1010809@redhat.com> Message-ID: <4EB02817.4010005@redhat.com> On 11/1/2011 11:53 AM, Adam Young wrote: > ACK and pushed to master. -- Endi S. Dewata From edewata at redhat.com Tue Nov 1 22:31:13 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 01 Nov 2011 17:31:13 -0500 Subject: [Freeipa-devel] [PATCH] 029 Page is cleared before it is visible In-Reply-To: <4EAFE81C.4030004@redhat.com> References: <4EA92AF7.8080609@redhat.com> <4EAF2365.8000005@redhat.com> <4EAFE81C.4030004@redhat.com> Message-ID: <4EB07331.9070606@redhat.com> On 11/1/2011 7:37 AM, Petr Vobornik wrote: >> 1. The new clear() method is called during refresh(), so the facet with >> the old data is still shown for a brief moment before it's cleared. > >> The clear() should be called before show(). However, if the pkey/filter >> is unchanged (check using needs_update()) we just need to show() the >> facet, no need to clear() and refresh() again. The above logic needs to >> be fixed. > > Changed. > >> The we will need to override the needs_update() for search and >> association facets because the default one always returns true. > > Done On the second thought, this might not be sufficient to detect the changes in the list page. Try changing an attribute in an entry, then go back to the list page, the list page will not show the updated attribute because the filter is not changed. I think we should remove the needs_update() from the search facet so it will always refresh, but we probably can keep it for association facet. What do you think? Sorry about that. There are probably better solutions, but let's do this separately. > 5) Added ID generator, using in radio_widget, same reason as #4. The get_id() method (might be better called get_next_id() or generate_id()) doesn't really need to take a widget parameter. The id_count should be unique enough. If you want, it can take an optional prefix so the ID will be like '-'. It will make it more usable for other things not just widgets. > 8) Maybe we should add a refresh button to search facet. It doesn't > reflect concurrent usage. Refresh by 2 changes of filter doesn't seem nice. You can reload the page too, but I agree we probably need a refresh button, and possibly for other facets too. I'll open a ticket. 9. The facet header's clear() calls load() with empty data. The load() will display the facet group label using facet's pkey. Since this is called before refresh(), sometimes you'll see 'undefined' or the old pkey. I think the code in entity.js:351-354 should check if the data is empty it should clear the label. 10. Instead of emptying button label in host.js:731-732, it's probably better to reset it to its initial value: var password_label = $('.button-label', that.set_password_button); password_label.text(IPA.messages.objects.host.password_set_button); -- Endi S. Dewata From edewata at redhat.com Wed Nov 2 01:11:53 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 01 Nov 2011 20:11:53 -0500 Subject: [Freeipa-devel] Nesting widgets In-Reply-To: <4EA01D9F.60705@redhat.com> References: <4E83BE0F.2020502@redhat.com> <4E9ED74D.2020902@redhat.com> <4EA01D9F.60705@redhat.com> Message-ID: <4EB098D9.2010802@redhat.com> > >>> So I decided to try to get an IP Address widget working. See the > >>> attached patch. It was fairly trivial. > >>> > >>> However, this widget is not really all that useful by itself. It would > >>> need to work as a part of a multivalued_text widget in order to > replace > >>> the widget used on the dnsrecord page. And looking at the multivalued > >>> text widget, I think you will agree that is going to be tricky. > >> > >> I think we can create an extend point for validation logic (EG > >> validator object in field's spec) instead of inheriting the widget. To summarize the recent discussion on IRC and other threads, we're planning to split the existing widget concept into (logical) fields and (visual) widgets. See: https://fedorahosted.org/freeipa/ticket/2040 The new widgets will be used mainly for input/output like the existing widgets, but they can also be any other UI components such as header or separator. The widgets will support nesting. For example, the multivalued widget will become a composite widget which has an Add and Undo All links and can contain other widgets: widgets: [ { factory: IPA.multivalued_widget, name: 'fullName', widget: IPA.text_widget } ] The widgets can be arranged inside a section, which is also a composite widget that has a header and is collapsible. widgets: [ { factory: IPA.section_widget, name: 'identity', widgets: [ { factory: IPA.text_widget, name: 'fullName' } ] } ] For IP address we could either use a text widget with an IP address validator, or we could also implement a custom IP address widget. Either solution can be used inside multivalued widget. widgets: [ { factory: IPA.multivalued_widget, name: 'ipaddress1', widget: { factory: IPA.text_widget, validator: IPA.ipaddress_validator } }, { factory: IPA.multivalued_widget, name: 'ipaddress2', widget: IPA.ipaddress_widget, }, ] > > Interesting concept. Yes, this would work too, and solve the issue for > > IP addresses validation. But there are places where we may want to > > validate only a single IP address, and we may end up with code > > duplication, although more likely they will both just call the same > > utility function. Since the multivalued widget is completely separate, we can reuse the IP address widget for a single valued attribute. widgets: [ { factory: IPA.ipaddress_widget, name: 'ipaddress' } ] > >>> In order to make the widget scheme more nestable, the section > >>> section.load and save can do more work, such as scoping down the piece > >>> of the request record to just that portion required by the widget. > >>> Bascially, it can do what widget.load does, just externally > >> > >> I don't think this would help. This would limit the widget. Currently > >> most widget works with the record.[widget name]. But we can override > >> load and save method to break the 1:1 mapping between widget and > >> record. Widget can consist of several widgets and supply them custom > >> record object (it can assume that the simple widget would fetch the > >> value of its name. The name is defined by the master widget (no > >> problem here).) > >> > >> The concept you mentioned could be beneficial if we abandon flat > >> records and introduce structured ones. But I think there is no will > >> for it. Event then I don't know if sections should be responsible for > >> this. It's not their purpose. > > > > Records are already structured, just that by the time we get to the > > individual fields, they tend to be flat. But the record is JSON and is a > > tree structure. > > > > But I like what you are saying. I agree that the interface for Widget > > and Section should be the same. Right now Section is: > > that.save = function(record) {}; > > that.load = function(record) {); > > > > and load is > > that.load = function(record) {}; > > that.save = function() {}; > > > > It is this last function that needs to be changed, but it will have far > > reaching effects. We use save to extract the value or values from the > > field for many uses. I think we can do this: For all widgets, rename > > save() to get_value(); and then add a save(record) method that calls > > get_value(); Then widget and record have the same interface, which is a > > big first step. The fields will be used to load/store attributes. Each field usually will have a corresponding widget, except hidden fields: fields: [ { name: 'krbprincipalname' // hidden }, { name: 'cn', widget: 'identity.fullName' // widget inside a section }, { name: 'mail', widget: 'contact.email' // multivalued widget } ] During load the fields will be used to get the attributes from the records returned by the server and then pass them to the corresponding widgets to be displayed. During update the fields will be used to save the values from the corresponding widgets and they will be used to generate the update commands. This will be covered in this ticket: https://fedorahosted.org/freeipa/ticket/2041 We might be able to do nested fields too to handle hierarchical records, but that's not needed now. > >> Another (maybe bigger) problem is layout. For multivalued textbox with > >> IP address widget one of the problems can be the placement (or even > >> behaviour) of 'undo' and 'error-link'. Possible solution could be: > >> specifying container for these parts by its container - the containing > >> widget could control where they would be rendered. Another one can be > >> API notification of the state change (no visual by the widget itself) > >> and handling it by the parent widget. > >> > >> For more layout demanding widgets section layout can be limiting (like > >> in host adder dialog which force to implement custom section instead > >> of widget). Which in my opinion is bad, but we don't have any other > >> solution for this yet. I think there is a ticket for it. > > > > Perhaps for more complex widgets, we can extract the layout into its own > > object, and pass it to the constructore. We'll provide one by default, > > but allow overloading. Yes, by default all containers (facets, dialogs, or composite widgets) will use 1-column layout. So the nested widgets will be arranged vertically with the same width. Each widget will be able to implement custom layout. This is probably sufficient for now. See also: https://fedorahosted.org/freeipa/ticket/1501 Other related tickets: https://fedorahosted.org/freeipa/ticket/2042 https://fedorahosted.org/freeipa/ticket/2043 https://fedorahosted.org/freeipa/ticket/2052 -- Endi S. Dewata From abokovoy at redhat.com Wed Nov 2 10:42:06 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 2 Nov 2011 12:42:06 +0200 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <4EAEB8A8.7010805@redhat.com> References: <4EA6C7E8.1000301@redhat.com> <4EA71C71.6070500@redhat.com> <4EA7D54E.3010300@redhat.com> <1319636472.27433.57.camel@balmora.brq.redhat.com> <4EA810B0.5020307@redhat.com> <1319639963.5652.3.camel@balmora.brq.redhat.com> <4EA81E29.4030007@redhat.com> <4EAE8416.4060401@redhat.com> <20111031121904.GA9412@redhat.com> <4EAEB8A8.7010805@redhat.com> Message-ID: <20111102104206.GB14373@redhat.com> On Mon, 31 Oct 2011, Jan Cholasta wrote: > Dne 31.10.2011 13:19, Alexander Bokovoy napsal(a): > >On Mon, 31 Oct 2011, Jan Cholasta wrote: > >>Added finalization for __call__ and the check for CLI. Patch attached. > >ACK from my side but see below. > > > >>+ def __getattribute__(self, name): > >>+ if not name.startswith('_Plugin__') and not name.startswith('_ReadOnly__') and name != 'finalize_late': > >>+ self.finalize_late() > >>+ return object.__getattribute__(self, name) > >Could you get faster than three string comparisons? As > >__getattribute__ is fairly often called it would make sense to keep > >these operations to absolute minimum. > > > > Is there any noticable slowdown? Yes. Now I have different patch to solve this issue that avoids using __getattribute__. Instead, it sets a trap on attributes that are changed by finalization process and when they are accessed first time, the trap forces instance to finalize. As result, the attributes get their proper values and traps are removed, no performance costs anymore. For Commands one additional check is done on __call__() method to verify that we are indeed finalized before execution proceeds. It is a safety net here. Performance is not bad: 1. Before the patch [root at vm-114 ipalib]# time ipa >/dev/null real 0m1.101s user 0m0.930s sys 0m0.151s [root at vm-114 ipalib]# time ipa user-find>/dev/null real 0m3.132s user 0m0.983s sys 0m0.135s 2. With patch [root at vm-114 ipalib]# patch -p2 <~/speedup.patch patching file frontend.py patching file plugable.py [root at vm-114 ipalib]# time ipa >/dev/null real 0m0.563s user 0m0.438s sys 0m0.098s [root at vm-114 ipalib]# time ipa >/dev/null real 0m0.521s user 0m0.412s sys 0m0.100s [root at vm-114 ipalib]# time ipa user-find>/dev/null real 0m1.069s user 0m0.445s sys 0m0.111s [root at vm-114 ipalib]# time ipa user-find>/dev/null real 0m0.840s user 0m0.425s sys 0m0.126s [root at vm-114 ipalib]# time ipa user-find>/dev/null real 0m0.816s user 0m0.432s sys 0m0.119s Patch is attached. -- / Alexander Bokovoy -------------- next part -------------- >From d28b13f9de7d41b25c51aa7c26ca2b09f8671e6b Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Wed, 2 Nov 2011 12:24:20 +0200 Subject: [PATCH] Perform late initialization of FreeIPA plugins https://fedorahosted.org/freeipa/ticket/1336 When plugins are loaded, instances of the provided objects and commands are registered in the API instance. The patch changes finalization process to apply on first access to the Command instance that would cause either access to properties (args, options, params) or execution of the command itself. The patch gives 2x boost for client-side commands like help and 3x boost for commands that go to the server side. All performance numbers are approximate and may vary a lot. --- ipalib/frontend.py | 20 ++++++++++++++++++++ ipalib/plugable.py | 6 ++---- 2 files changed, 22 insertions(+), 4 deletions(-) diff --git a/ipalib/frontend.py b/ipalib/frontend.py index 61e7f493f8a8e30a1a189d06cd6a69893319deaf..a3fed2bbf0fb4631e238fad9892fb8129dbd6ca1 100644 --- a/ipalib/frontend.py +++ b/ipalib/frontend.py @@ -404,6 +404,22 @@ class Command(HasParam): msg_summary = None msg_truncated = _('Results are truncated, try a more specific search') + def __init__(self): + class Finalizer(object): + def __init__(self, master, attr): + self.master = master + self.attr = attr + + def __call__(self): + self.master.finalize() + # At this point master.attr points to proper object + return self.master.__dict__[self.attr]() + + self.args = Finalizer(self, 'args') + self.options = Finalizer(self, 'options') + self.params = Finalizer(self, 'params') + super(Command, self).__init__() + def __call__(self, *args, **options): """ Perform validation and then execute the command. @@ -411,6 +427,10 @@ class Command(HasParam): If not in a server context, the call will be forwarded over XML-RPC and the executed an the nearest IPA server. """ + # Plugin instance must be finalized before we get to execution + if not self.__dict__['_Plugin__finalized']: + self.finalize() + params = self.args_options_2_params(*args, **options) self.debug( 'raw: %s(%s)', self.name, ', '.join(self._repr_iter(**params)) diff --git a/ipalib/plugable.py b/ipalib/plugable.py index b0e415656e0428eb164c35a2862fcfbf50883381..ee29929c0a6de776724659e03194973937111868 100644 --- a/ipalib/plugable.py +++ b/ipalib/plugable.py @@ -207,6 +207,7 @@ class Plugin(ReadOnly): self.label ) ) + self.__finalized = False def __get_api(self): """ @@ -220,6 +221,7 @@ class Plugin(ReadOnly): def finalize(self): """ """ + self.__finalized = True if not is_production_mode(self): lock(self) @@ -637,10 +639,6 @@ class API(DictProxy): if not production_mode: assert p.instance.api is self - for p in plugins.itervalues(): - p.instance.finalize() - if not production_mode: - assert islocked(p.instance) is True object.__setattr__(self, '_API__finalized', True) tuple(PluginInfo(p) for p in plugins.itervalues()) object.__setattr__(self, 'plugins', -- 1.7.7 From pvoborni at redhat.com Wed Nov 2 13:33:34 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 02 Nov 2011 14:33:34 +0100 Subject: [Freeipa-devel] [PATCH] 029 Page is cleared before it is visible In-Reply-To: <4EB07331.9070606@redhat.com> References: <4EA92AF7.8080609@redhat.com> <4EAF2365.8000005@redhat.com> <4EAFE81C.4030004@redhat.com> <4EB07331.9070606@redhat.com> Message-ID: <4EB146AE.9010609@redhat.com> On 11/01/2011 11:31 PM, Endi Sukma Dewata wrote: > On 11/1/2011 7:37 AM, Petr Vobornik wrote: >>> 1. The new clear() method is called during refresh(), so the facet with >>> the old data is still shown for a brief moment before it's cleared. >> >>> The clear() should be called before show(). However, if the pkey/filter >>> is unchanged (check using needs_update()) we just need to show() the >>> facet, no need to clear() and refresh() again. The above logic needs to >>> be fixed. >> >> Changed. >> >>> The we will need to override the needs_update() for search and >>> association facets because the default one always returns true. >> >> Done > > On the second thought, this might not be sufficient to detect the > changes in the list page. Try changing an attribute in an entry, then go > back to the list page, the list page will not show the updated attribute > because the filter is not changed. > > I think we should remove the needs_update() from the search facet so it > will always refresh, but we probably can keep it for association facet. > What do you think? Sorry about that. Changed to refresh always. Clearing always or not clearing at all seems wrong. When the pkey doesn't change, only small portion of values can/will change, so it's probably not so wrong to show the previous list, but when the filter is changed the results will be probably much different and the clear is needed. What do you think? -> Added needs_clear method which clears if the filter changes (basically renamed needs_update). > There are probably better solutions, but let's do this separately. In future we can build a mechanism for subscribing to events from different facets and doing appropriate actions. Something like: var refresh_search_on_save = function(spec) { var that = {}; that.register = function() { that.entity = IPA.get_entity(spec.entity); that.details_facet = that.entity.get_facet(spec.facet || 'details'); that.search_facet = that.entity.get_facet(spec.search_facet || 'search'); that.details_facet.on_save.attach(that.on_save, that); }; that.on_save = function() { that.search_facet.set_needs_refresh(true); }; return that; }; So the facets won't be dependent on each other. > >> 5) Added ID generator, using in radio_widget, same reason as #4. > > The get_id() method (might be better called get_next_id() or > generate_id()) doesn't really need to take a widget parameter. The > id_count should be unique enough. If you want, it can take an optional > prefix so the ID will be like '-'. It will make it more > usable for other things not just widgets. changed to get_next_id(prefix). > > 9. The facet header's clear() calls load() with empty data. The load() > will display the facet group label using facet's pkey. Since this is > called before refresh(), sometimes you'll see 'undefined' or the old > pkey. I think the code in entity.js:351-354 should check if the data is > empty it should clear the label. Fixed > > 10. Instead of emptying button label in host.js:731-732, it's probably > better to reset it to its initial value: > > var password_label = $('.button-label', that.set_password_button); > password_label.text(IPA.messages.objects.host.password_set_button); Seems more proper to clean the label. If the label is set to ...host.password_set_button it will always shows before load "set OTP" after load it can change to "reset OTP" which is wrong behavior. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0029-2-Page-is-cleared-before-it-is-visible.patch Type: text/x-patch Size: 17747 bytes Desc: not available URL: From mkosek at redhat.com Wed Nov 2 13:41:43 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 02 Nov 2011 14:41:43 +0100 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <20111102104206.GB14373@redhat.com> References: <4EA6C7E8.1000301@redhat.com> <4EA71C71.6070500@redhat.com> <4EA7D54E.3010300@redhat.com> <1319636472.27433.57.camel@balmora.brq.redhat.com> <4EA810B0.5020307@redhat.com> <1319639963.5652.3.camel@balmora.brq.redhat.com> <4EA81E29.4030007@redhat.com> <4EAE8416.4060401@redhat.com> <20111031121904.GA9412@redhat.com> <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> Message-ID: <1320241305.3724.43.camel@balmora.brq.redhat.com> On Wed, 2011-11-02 at 12:42 +0200, Alexander Bokovoy wrote: > On Mon, 31 Oct 2011, Jan Cholasta wrote: > > Dne 31.10.2011 13:19, Alexander Bokovoy napsal(a): > > >On Mon, 31 Oct 2011, Jan Cholasta wrote: > > >>Added finalization for __call__ and the check for CLI. Patch attached. > > >ACK from my side but see below. > > > > > >>+ def __getattribute__(self, name): > > >>+ if not name.startswith('_Plugin__') and not name.startswith('_ReadOnly__') and name != 'finalize_late': > > >>+ self.finalize_late() > > >>+ return object.__getattribute__(self, name) > > >Could you get faster than three string comparisons? As > > >__getattribute__ is fairly often called it would make sense to keep > > >these operations to absolute minimum. > > > > > > > Is there any noticable slowdown? > Yes. Now I have different patch to solve this issue that avoids using > __getattribute__. Instead, it sets a trap on attributes that are > changed by finalization process and when they are accessed first time, > the trap forces instance to finalize. As result, the attributes get > their proper values and traps are removed, no performance costs > anymore. > > For Commands one additional check is done on __call__() method to > verify that we are indeed finalized before execution proceeds. It is a > safety net here. > > Performance is not bad: > > 1. Before the patch > [root at vm-114 ipalib]# time ipa >/dev/null > > real 0m1.101s > user 0m0.930s > sys 0m0.151s > [root at vm-114 ipalib]# time ipa user-find>/dev/null > > real 0m3.132s > user 0m0.983s > sys 0m0.135s > > 2. With patch > [root at vm-114 ipalib]# patch -p2 <~/speedup.patch > patching file frontend.py > patching file plugable.py > [root at vm-114 ipalib]# time ipa >/dev/null > > real 0m0.563s > user 0m0.438s > sys 0m0.098s > [root at vm-114 ipalib]# time ipa >/dev/null > > real 0m0.521s > user 0m0.412s > sys 0m0.100s > [root at vm-114 ipalib]# time ipa user-find>/dev/null > > real 0m1.069s > user 0m0.445s > sys 0m0.111s > [root at vm-114 ipalib]# time ipa user-find>/dev/null > > real 0m0.840s > user 0m0.425s > sys 0m0.126s > [root at vm-114 ipalib]# time ipa user-find>/dev/null > > real 0m0.816s > user 0m0.432s > sys 0m0.119s > > Patch is attached. Hmm, this looks impressive! It would brings us considerably faster CLI calls. And again, I would feel much safer if we do these optimizations only for CLI and let server do all the initialization right on start. Unit tests show few errors though. But these may not be the actual problems in the patch, it may be just wrong test assumptions: ====================================================================== FAIL: Doctest: ipalib.crud ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib64/python2.7/doctest.py", line 2166, in runTest raise self.failureException(self.format_failure(new.getvalue())) AssertionError: Failed doctest test for ipalib.crud File "/home/mkosek/freeipa/ipalib/crud.py", line 19, in crud ---------------------------------------------------------------------- File "/home/mkosek/freeipa/ipalib/crud.py", line 76, in ipalib.crud Failed example: list(api.Command.user_add.args) Exception raised: Traceback (most recent call last): File "/usr/lib64/python2.7/doctest.py", line 1254, in __run compileflags, 1) in test.globs File "", line 1, in list(api.Command.user_add.args) TypeError: 'Finalizer' object is not iterable ---------------------------------------------------------------------- File "/home/mkosek/freeipa/ipalib/crud.py", line 78, in ipalib.crud Failed example: list(api.Command.user_add.options) Exception raised: Traceback (most recent call last): File "/usr/lib64/python2.7/doctest.py", line 1254, in __run compileflags, 1) in test.globs File "", line 1, in list(api.Command.user_add.options) TypeError: 'Finalizer' object is not iterable ---------------------------------------------------------------------- File "/home/mkosek/freeipa/ipalib/crud.py", line 94, in ipalib.crud Failed example: list(api.Command.user_show.args) Exception raised: Traceback (most recent call last): File "/usr/lib64/python2.7/doctest.py", line 1254, in __run compileflags, 1) in test.globs File "", line 1, in list(api.Command.user_show.args) TypeError: 'Finalizer' object is not iterable ---------------------------------------------------------------------- File "/home/mkosek/freeipa/ipalib/crud.py", line 96, in ipalib.crud Failed example: list(api.Command.user_show.options) Exception raised: Traceback (most recent call last): File "/usr/lib64/python2.7/doctest.py", line 1254, in __run compileflags, 1) in test.globs File "", line 1, in list(api.Command.user_show.options) TypeError: 'Finalizer' object is not iterable ---------------------------------------------------------------------- File "/home/mkosek/freeipa/ipalib/crud.py", line 114, in ipalib.crud Failed example: list(api.Command.user_add.output_params) Exception raised: Traceback (most recent call last): File "/usr/lib64/python2.7/doctest.py", line 1254, in __run compileflags, 1) in test.globs File "", line 1, in list(api.Command.user_add.output_params) TypeError: 'NoneType' object is not iterable ---------------------------------------------------------------------- File "/home/mkosek/freeipa/ipalib/crud.py", line 116, in ipalib.crud Failed example: list(api.Command.user_show.output_params) Exception raised: Traceback (most recent call last): File "/usr/lib64/python2.7/doctest.py", line 1254, in __run compileflags, 1) in test.globs File "", line 1, in list(api.Command.user_show.output_params) TypeError: 'NoneType' object is not iterable ====================================================================== FAIL: Test the ``ipalib.frontend.Command.args`` instance attribute. ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nose/case.py", line 187, in runTest self.test(*self.arg) File "/home/mkosek/freeipa/tests/test_ipalib/test_frontend.py", line 255, in test_args assert self.cls().args is None AssertionError ====================================================================== FAIL: Test the `ipalib.frontend.Command.get_output_params` method. ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nose/case.py", line 187, in runTest self.test(*self.arg) File "/home/mkosek/freeipa/tests/test_ipalib/test_frontend.py", line 710, in test_get_output_params assert list(inst.get_output_params()) == ['one', 'two', 'three'] AssertionError ====================================================================== FAIL: Test the ``ipalib.frontend.Command.options`` instance attribute. ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nose/case.py", line 187, in runTest self.test(*self.arg) File "/home/mkosek/freeipa/tests/test_ipalib/test_frontend.py", line 304, in test_options assert self.cls().options is None AssertionError I found one more issue with our ./makeapi script. $ ./makeapi Writing API to API.txt Traceback (most recent call last): File "./makeapi", line 392, in sys.exit(main()) File "./makeapi", line 376, in main rval |= make_api() File "./makeapi", line 167, in make_api fd.write('args: %d,%d,%d\n' % (len(cmd.args), len(cmd.options), len(cmd.output))) TypeError: object of type 'Finalizer' has no len() Martin From abokovoy at redhat.com Wed Nov 2 14:01:53 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 2 Nov 2011 16:01:53 +0200 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <1320241305.3724.43.camel@balmora.brq.redhat.com> References: <4EA7D54E.3010300@redhat.com> <1319636472.27433.57.camel@balmora.brq.redhat.com> <4EA810B0.5020307@redhat.com> <1319639963.5652.3.camel@balmora.brq.redhat.com> <4EA81E29.4030007@redhat.com> <4EAE8416.4060401@redhat.com> <20111031121904.GA9412@redhat.com> <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <1320241305.3724.43.camel@balmora.brq.redhat.com> Message-ID: <20111102140152.GC14373@redhat.com> On Wed, 02 Nov 2011, Martin Kosek wrote: > Hmm, this looks impressive! It would brings us considerably faster CLI > calls. > > And again, I would feel much safer if we do these optimizations only > for CLI and let server do all the initialization right on start. Well, that could be done in API.finalize() as we would know context at that point already. > Unit tests show few errors though. But these may not be the actual > problems in the patch, it may be just wrong test assumptions: > > ====================================================================== > FAIL: Doctest: ipalib.crud > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/lib64/python2.7/doctest.py", line 2166, in runTest > raise self.failureException(self.format_failure(new.getvalue())) > AssertionError: Failed doctest test for ipalib.crud > File "/home/mkosek/freeipa/ipalib/crud.py", line 19, in crud these are examples that doctest extracts and tries to run. I can make Finalizer class iterable and that will solve all these: > ---------------------------------------------------------------------- > File "/home/mkosek/freeipa/ipalib/crud.py", line 76, in ipalib.crud > Failed example: > list(api.Command.user_add.args) > Exception raised: > Traceback (most recent call last): > File "/usr/lib64/python2.7/doctest.py", line 1254, in __run > compileflags, 1) in test.globs > File "", line 1, in > list(api.Command.user_add.args) > TypeError: 'Finalizer' object is not iterable > ---------------------------------------------------------------------- > File "/home/mkosek/freeipa/ipalib/crud.py", line 78, in ipalib.crud > Failed example: > list(api.Command.user_add.options) > Exception raised: > Traceback (most recent call last): > File "/usr/lib64/python2.7/doctest.py", line 1254, in __run > compileflags, 1) in test.globs > File "", line 1, in > list(api.Command.user_add.options) > TypeError: 'Finalizer' object is not iterable > ---------------------------------------------------------------------- > File "/home/mkosek/freeipa/ipalib/crud.py", line 94, in ipalib.crud > Failed example: > list(api.Command.user_show.args) > Exception raised: > Traceback (most recent call last): > File "/usr/lib64/python2.7/doctest.py", line 1254, in __run > compileflags, 1) in test.globs > File "", line 1, in > list(api.Command.user_show.args) > TypeError: 'Finalizer' object is not iterable > ---------------------------------------------------------------------- > File "/home/mkosek/freeipa/ipalib/crud.py", line 96, in ipalib.crud > Failed example: > list(api.Command.user_show.options) > Exception raised: > Traceback (most recent call last): > File "/usr/lib64/python2.7/doctest.py", line 1254, in __run > compileflags, 1) in test.globs > File "", line 1, in > list(api.Command.user_show.options) > TypeError: 'Finalizer' object is not iterable > ---------------------------------------------------------------------- > File "/home/mkosek/freeipa/ipalib/crud.py", line 114, in ipalib.crud > Failed example: > list(api.Command.user_add.output_params) > Exception raised: > Traceback (most recent call last): > File "/usr/lib64/python2.7/doctest.py", line 1254, in __run > compileflags, 1) in test.globs > File "", line 1, in > list(api.Command.user_add.output_params) > TypeError: 'NoneType' object is not iterable Output params could be run through finalizer as well. > ---------------------------------------------------------------------- > File "/home/mkosek/freeipa/ipalib/crud.py", line 116, in ipalib.crud > Failed example: > list(api.Command.user_show.output_params) > Exception raised: > Traceback (most recent call last): > File "/usr/lib64/python2.7/doctest.py", line 1254, in __run > compileflags, 1) in test.globs > File "", line 1, in > list(api.Command.user_show.output_params) > TypeError: 'NoneType' object is not iterable Same here. Errors below are about assumptions that before finalizing args, options > ====================================================================== > FAIL: Test the ``ipalib.frontend.Command.args`` instance attribute. > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/nose/case.py", line 187, in > runTest > self.test(*self.arg) > File "/home/mkosek/freeipa/tests/test_ipalib/test_frontend.py", line > 255, in test_args > assert self.cls().args is None > AssertionError > > ====================================================================== > FAIL: Test the `ipalib.frontend.Command.get_output_params` method. > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/nose/case.py", line 187, in > runTest > self.test(*self.arg) > File "/home/mkosek/freeipa/tests/test_ipalib/test_frontend.py", line > 710, in test_get_output_params > assert list(inst.get_output_params()) == ['one', 'two', 'three'] > AssertionError > > ====================================================================== > FAIL: Test the ``ipalib.frontend.Command.options`` instance attribute. > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/nose/case.py", line 187, in > runTest > self.test(*self.arg) > File "/home/mkosek/freeipa/tests/test_ipalib/test_frontend.py", line > 304, in test_options > assert self.cls().options is None > AssertionError > > > > I found one more issue with our ./makeapi script. > > $ ./makeapi > Writing API to API.txt > Traceback (most recent call last): > File "./makeapi", line 392, in > sys.exit(main()) > File "./makeapi", line 376, in main > rval |= make_api() > File "./makeapi", line 167, in make_api > fd.write('args: %d,%d,%d\n' % (len(cmd.args), len(cmd.options), > len(cmd.output))) > TypeError: object of type 'Finalizer' has no len() This is expectation that Finalizer is iteratable. I think these all can be fixed in two relatively simple shots. Let me look at that later today. -- / Alexander Bokovoy From jcholast at redhat.com Wed Nov 2 14:24:39 2011 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 02 Nov 2011 15:24:39 +0100 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <20111102104206.GB14373@redhat.com> References: <4EA6C7E8.1000301@redhat.com> <4EA71C71.6070500@redhat.com> <4EA7D54E.3010300@redhat.com> <1319636472.27433.57.camel@balmora.brq.redhat.com> <4EA810B0.5020307@redhat.com> <1319639963.5652.3.camel@balmora.brq.redhat.com> <4EA81E29.4030007@redhat.com> <4EAE8416.4060401@redhat.com> <20111031121904.GA9412@redhat.com> <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> Message-ID: <4EB152A7.5010303@redhat.com> Dne 2.11.2011 11:42, Alexander Bokovoy napsal(a): > On Mon, 31 Oct 2011, Jan Cholasta wrote: >> Dne 31.10.2011 13:19, Alexander Bokovoy napsal(a): >>> On Mon, 31 Oct 2011, Jan Cholasta wrote: >>>> Added finalization for __call__ and the check for CLI. Patch attached. >>> ACK from my side but see below. >>> >>>> + def __getattribute__(self, name): >>>> + if not name.startswith('_Plugin__') and not name.startswith('_ReadOnly__') and name != 'finalize_late': >>>> + self.finalize_late() >>>> + return object.__getattribute__(self, name) >>> Could you get faster than three string comparisons? As >>> __getattribute__ is fairly often called it would make sense to keep >>> these operations to absolute minimum. >>> >> >> Is there any noticable slowdown? > Yes. Now I have different patch to solve this issue that avoids using > __getattribute__. Instead, it sets a trap on attributes that are > changed by finalization process and when they are accessed first time, > the trap forces instance to finalize. As result, the attributes get > their proper values and traps are removed, no performance costs > anymore. > > For Commands one additional check is done on __call__() method to > verify that we are indeed finalized before execution proceeds. It is a > safety net here. > > Performance is not bad: > > 1. Before the patch > [root at vm-114 ipalib]# time ipa>/dev/null > > real 0m1.101s > user 0m0.930s > sys 0m0.151s > [root at vm-114 ipalib]# time ipa user-find>/dev/null > > real 0m3.132s > user 0m0.983s > sys 0m0.135s > > 2. With patch > [root at vm-114 ipalib]# patch -p2<~/speedup.patch > patching file frontend.py > patching file plugable.py > [root at vm-114 ipalib]# time ipa>/dev/null > > real 0m0.563s > user 0m0.438s > sys 0m0.098s > [root at vm-114 ipalib]# time ipa>/dev/null > > real 0m0.521s > user 0m0.412s > sys 0m0.100s > [root at vm-114 ipalib]# time ipa user-find>/dev/null > > real 0m1.069s > user 0m0.445s > sys 0m0.111s > [root at vm-114 ipalib]# time ipa user-find>/dev/null > > real 0m0.840s > user 0m0.425s > sys 0m0.126s > [root at vm-114 ipalib]# time ipa user-find>/dev/null > > real 0m0.816s > user 0m0.432s > sys 0m0.119s > > Patch is attached. I see a number of problems with the patch: * Only Command subclasses are finalized (that might be the reason why it is so fast) * You assume that the first operation on a plugin instance is a call on the args/options/params namespaces, but there is no guarantee that it will be * The whole approach is error-prone, as it requires the programmer to carefully setup the attributes so that the plugin can be properly finalized (this should be done automatically, so that every plugin is guaranteed to be finalized without complicated setup) Honza -- Jan Cholasta From abokovoy at redhat.com Wed Nov 2 15:11:15 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 2 Nov 2011 17:11:15 +0200 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <4EB152A7.5010303@redhat.com> References: <4EA7D54E.3010300@redhat.com> <1319636472.27433.57.camel@balmora.brq.redhat.com> <4EA810B0.5020307@redhat.com> <1319639963.5652.3.camel@balmora.brq.redhat.com> <4EA81E29.4030007@redhat.com> <4EAE8416.4060401@redhat.com> <20111031121904.GA9412@redhat.com> <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <4EB152A7.5010303@redhat.com> Message-ID: <20111102151114.GD14373@redhat.com> On Wed, 02 Nov 2011, Jan Cholasta wrote: > I see a number of problems with the patch: > > * Only Command subclasses are finalized (that might be the reason > why it is so fast) > * You assume that the first operation on a plugin instance is a > call on the args/options/params namespaces, but there is no > guarantee that it will be > * The whole approach is error-prone, as it requires the programmer > to carefully setup the attributes so that the plugin can be properly > finalized (this should be done automatically, so that every plugin > is guaranteed to be finalized without complicated setup) Perhaps you need to look at why finalize() is there at all. We have certain properties that are set only during finalize() method execution. If something cannot be performed without accessing those properties, it cannot be done before finalize() is performed. Anything else is allowed. So it boils down to being able to intercept the access to those properties that change their value in effect of finalize() call. What this all means is if a plugin writer has something in mind when changing properties in his/her re-implementation of finalize() method, it is simple to mark up those properties as such and this patch gives you a simple process and means to do so. Callable instances are a consequence of the above -- Command.__call__() does use properties that are changed due to finalize() being run. In fact, there are so many places in FreeIPA where we simply expect that foo.args/params/output_params/output is both iterable and callable instance that it is not even funny. Now, the process established by the proposed patch is actually dependent only on the fact that bound instance has finalize() method. This is quite generic and can be used for all other types of objects. -- / Alexander Bokovoy From jcholast at redhat.com Wed Nov 2 15:14:59 2011 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 02 Nov 2011 16:14:59 +0100 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <20111102151114.GD14373@redhat.com> References: <4EA7D54E.3010300@redhat.com> <1319636472.27433.57.camel@balmora.brq.redhat.com> <4EA810B0.5020307@redhat.com> <1319639963.5652.3.camel@balmora.brq.redhat.com> <4EA81E29.4030007@redhat.com> <4EAE8416.4060401@redhat.com> <20111031121904.GA9412@redhat.com> <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <4EB152A7.5010303@redhat.com> <20111102151114.GD14373@redhat.com> Message-ID: <4EB15E73.8070402@redhat.com> Dne 2.11.2011 16:11, Alexander Bokovoy napsal(a): > On Wed, 02 Nov 2011, Jan Cholasta wrote: >> I see a number of problems with the patch: >> >> * Only Command subclasses are finalized (that might be the reason >> why it is so fast) >> * You assume that the first operation on a plugin instance is a >> call on the args/options/params namespaces, but there is no >> guarantee that it will be >> * The whole approach is error-prone, as it requires the programmer >> to carefully setup the attributes so that the plugin can be properly >> finalized (this should be done automatically, so that every plugin >> is guaranteed to be finalized without complicated setup) > Perhaps you need to look at why finalize() is there at all. We have > certain properties that are set only during finalize() method > execution. If something cannot be performed without accessing those > properties, it cannot be done before finalize() is performed. > Anything else is allowed. So it boils down to being able to intercept > the access to those properties that change their value in effect of > finalize() call. > > What this all means is if a plugin writer has something in mind > when changing properties in his/her re-implementation of finalize() > method, it is simple to mark up those properties as such and this > patch gives you a simple process and means to do so. > > Callable instances are a consequence of the above -- > Command.__call__() does use properties that are changed due to > finalize() being run. In fact, there are so many places in FreeIPA > where we simply expect that foo.args/params/output_params/output is > both iterable and callable instance that it is not even funny. > > Now, the process established by the proposed patch is actually > dependent only on the fact that bound instance has finalize() method. > This is quite generic and can be used for all other types of objects. > The real problem is in the API class so it should be fixed there, not worked around in Plugin subclasses. -- Jan Cholasta From edewata at redhat.com Wed Nov 2 15:41:53 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 02 Nov 2011 10:41:53 -0500 Subject: [Freeipa-devel] [PATCH] 029 Page is cleared before it is visible In-Reply-To: <4EB146AE.9010609@redhat.com> References: <4EA92AF7.8080609@redhat.com> <4EAF2365.8000005@redhat.com> <4EAFE81C.4030004@redhat.com> <4EB07331.9070606@redhat.com> <4EB146AE.9010609@redhat.com> Message-ID: <4EB164C1.4090308@redhat.com> On 11/2/2011 8:33 AM, Petr Vobornik wrote: >> On the second thought, this might not be sufficient to detect the >> changes in the list page. Try changing an attribute in an entry, then go >> back to the list page, the list page will not show the updated attribute >> because the filter is not changed. >> >> I think we should remove the needs_update() from the search facet so it >> will always refresh, but we probably can keep it for association facet. >> What do you think? Sorry about that. > > Changed to refresh always. Clearing always or not clearing at all seems > wrong. When the pkey doesn't change, only small portion of values > can/will change, so it's probably not so wrong to show the previous > list, but when the filter is changed the results will be probably much > different and the clear is needed. What do you think? -> Added > needs_clear method which clears if the filter changes (basically renamed > needs_update). OK. The search page feels more responsive this way. The term needs_clear() is a bit confusing because during facet.refresh() it also calls table.empty() which essentially the same as table.clear(). How about clear_before_show()? It's not important, we can do this later. >> There are probably better solutions, but let's do this separately. > > In future we can build a mechanism for subscribing to events from > different facets and doing appropriate actions. > > Something like: > > var refresh_search_on_save = function(spec) { > var that = {}; > > that.register = function() { > > that.entity = IPA.get_entity(spec.entity); > that.details_facet = that.entity.get_facet(spec.facet || 'details'); > that.search_facet = that.entity.get_facet(spec.search_facet || 'search'); > > that.details_facet.on_save.attach(that.on_save, that); > }; > > that.on_save = function() { > that.search_facet.set_needs_refresh(true); > }; > > return that; > }; > > So the facets won't be dependent on each other. Yes, that would be better, but I'd put the registration somewhere inside the entity class (to be created). This is also going to force early creation of the facets as opposed to lazy loading. >> 9. The facet header's clear() calls load() with empty data. The load() >> will display the facet group label using facet's pkey. Since this is >> called before refresh(), sometimes you'll see 'undefined' or the old >> pkey. I think the code in entity.js:351-354 should check if the data is >> empty it should clear the label. > > Fixed I think displaying no label at all would be better than showing incomplete label (without primary key), but we can do this later. >> 10. Instead of emptying button label in host.js:731-732, it's probably >> better to reset it to its initial value: >> >> var password_label = $('.button-label', that.set_password_button); >> password_label.text(IPA.messages.objects.host.password_set_button); > > Seems more proper to clean the label. If the label is set to > ...host.password_set_button it will always shows before load "set OTP" > after load it can change to "reset OTP" which is wrong behavior. I see your point. It might be better to just hide the button, but we can do this later. Question about this one: > 4) Changed direct/indirect radio names in association facets - radios > form different facets were interfering. Did you notice any problem with the old radio name? The name was supposed to be used locally by the facet itself so it should not interfere with radios in other facets, whereas ID is global so it needs to be unique. Regardless, ACK and pushed to master. -- Endi S. Dewata From ayoung at redhat.com Wed Nov 2 15:53:04 2011 From: ayoung at redhat.com (Adam Young) Date: Wed, 02 Nov 2011 11:53:04 -0400 Subject: [Freeipa-devel] Nesting widgets In-Reply-To: <4EB098D9.2010802@redhat.com> References: <4E83BE0F.2020502@redhat.com> <4E9ED74D.2020902@redhat.com> <4EA01D9F.60705@redhat.com> <4EB098D9.2010802@redhat.com> Message-ID: <4EB16760.4010606@redhat.com> This sounds pretty good. I think it is the right approach. On 11/01/2011 09:11 PM, Endi Sukma Dewata wrote: >> >>> So I decided to try to get an IP Address widget working. See the >> >>> attached patch. It was fairly trivial. >> >>> >> >>> However, this widget is not really all that useful by itself. It >> would >> >>> need to work as a part of a multivalued_text widget in order to >> replace >> >>> the widget used on the dnsrecord page. And looking at the >> multivalued >> >>> text widget, I think you will agree that is going to be tricky. >> >> >> >> I think we can create an extend point for validation logic (EG >> >> validator object in field's spec) instead of inheriting the widget. > > To summarize the recent discussion on IRC and other threads, we're > planning to split the existing widget concept into (logical) fields > and (visual) widgets. See: https://fedorahosted.org/freeipa/ticket/2040 > > The new widgets will be used mainly for input/output like the existing > widgets, but they can also be any other UI components such as header > or separator. > > The widgets will support nesting. For example, the multivalued widget > will become a composite widget which has an Add and Undo All links and > can contain other widgets: > > widgets: [ > { > factory: IPA.multivalued_widget, > name: 'fullName', > widget: IPA.text_widget > } > ] > > The widgets can be arranged inside a section, which is also a > composite widget that has a header and is collapsible. > > widgets: [ > { > factory: IPA.section_widget, > name: 'identity', > widgets: [ > { > factory: IPA.text_widget, > name: 'fullName' > } > ] > } > ] > > For IP address we could either use a text widget with an IP address > validator, or we could also implement a custom IP address widget. > Either solution can be used inside multivalued widget. > > widgets: [ > { > factory: IPA.multivalued_widget, > name: 'ipaddress1', > widget: { > factory: IPA.text_widget, > validator: IPA.ipaddress_validator > } > }, > { > factory: IPA.multivalued_widget, > name: 'ipaddress2', > widget: IPA.ipaddress_widget, > }, > ] > >> > Interesting concept. Yes, this would work too, and solve the issue for >> > IP addresses validation. But there are places where we may want to >> > validate only a single IP address, and we may end up with code >> > duplication, although more likely they will both just call the same >> > utility function. > > Since the multivalued widget is completely separate, we can reuse the > IP address widget for a single valued attribute. > > widgets: [ > { > factory: IPA.ipaddress_widget, > name: 'ipaddress' > } > ] > >> >>> In order to make the widget scheme more nestable, the section >> >>> section.load and save can do more work, such as scoping down the >> piece >> >>> of the request record to just that portion required by the widget. >> >>> Bascially, it can do what widget.load does, just externally >> >> >> >> I don't think this would help. This would limit the widget. Currently >> >> most widget works with the record.[widget name]. But we can override >> >> load and save method to break the 1:1 mapping between widget and >> >> record. Widget can consist of several widgets and supply them custom >> >> record object (it can assume that the simple widget would fetch the >> >> value of its name. The name is defined by the master widget (no >> >> problem here).) >> >> >> >> The concept you mentioned could be beneficial if we abandon flat >> >> records and introduce structured ones. But I think there is no will >> >> for it. Event then I don't know if sections should be responsible for >> >> this. It's not their purpose. >> > >> > Records are already structured, just that by the time we get to the >> > individual fields, they tend to be flat. But the record is JSON and >> is a >> > tree structure. >> > >> > But I like what you are saying. I agree that the interface for Widget >> > and Section should be the same. Right now Section is: >> > that.save = function(record) {}; >> > that.load = function(record) {); >> > >> > and load is >> > that.load = function(record) {}; >> > that.save = function() {}; >> > >> > It is this last function that needs to be changed, but it will have >> far >> > reaching effects. We use save to extract the value or values from the >> > field for many uses. I think we can do this: For all widgets, rename >> > save() to get_value(); and then add a save(record) method that calls >> > get_value(); Then widget and record have the same interface, which >> is a >> > big first step. > > The fields will be used to load/store attributes. Each field usually > will have a corresponding widget, except hidden fields: > > fields: [ > { > name: 'krbprincipalname' // hidden > }, > { > name: 'cn', > widget: 'identity.fullName' // widget inside a section > }, > { > name: 'mail', > widget: 'contact.email' // multivalued widget > } > ] > > During load the fields will be used to get the attributes from the > records returned by the server and then pass them to the corresponding > widgets to be displayed. > > During update the fields will be used to save the values from the > corresponding widgets and they will be used to generate the update > commands. This will be covered in this ticket: > https://fedorahosted.org/freeipa/ticket/2041 > > We might be able to do nested fields too to handle hierarchical > records, but that's not needed now. > >> >> Another (maybe bigger) problem is layout. For multivalued textbox >> with >> >> IP address widget one of the problems can be the placement (or even >> >> behaviour) of 'undo' and 'error-link'. Possible solution could be: >> >> specifying container for these parts by its container - the >> containing >> >> widget could control where they would be rendered. Another one can be >> >> API notification of the state change (no visual by the widget itself) >> >> and handling it by the parent widget. >> >> >> >> For more layout demanding widgets section layout can be limiting >> (like >> >> in host adder dialog which force to implement custom section instead >> >> of widget). Which in my opinion is bad, but we don't have any other >> >> solution for this yet. I think there is a ticket for it. >> > >> > Perhaps for more complex widgets, we can extract the layout into >> its own >> > object, and pass it to the constructore. We'll provide one by default, >> > but allow overloading. > > Yes, by default all containers (facets, dialogs, or composite widgets) > will use 1-column layout. So the nested widgets will be arranged > vertically with the same width. Each widget will be able to implement > custom layout. This is probably sufficient for now. See also: > https://fedorahosted.org/freeipa/ticket/1501 > > Other related tickets: > https://fedorahosted.org/freeipa/ticket/2042 > https://fedorahosted.org/freeipa/ticket/2043 > https://fedorahosted.org/freeipa/ticket/2052 > From pvoborni at redhat.com Wed Nov 2 16:01:04 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 02 Nov 2011 17:01:04 +0100 Subject: [Freeipa-devel] [PATCH] 029 Page is cleared before it is visible In-Reply-To: <4EB164C1.4090308@redhat.com> References: <4EA92AF7.8080609@redhat.com> <4EAF2365.8000005@redhat.com> <4EAFE81C.4030004@redhat.com> <4EB07331.9070606@redhat.com> <4EB146AE.9010609@redhat.com> <4EB164C1.4090308@redhat.com> Message-ID: <4EB16940.4040708@redhat.com> On 11/02/2011 04:41 PM, Endi Sukma Dewata wrote: > On 11/2/2011 8:33 AM, Petr Vobornik wrote: > > Question about this one: > >> 4) Changed direct/indirect radio names in association facets - radios >> form different facets were interfering. > > Did you notice any problem with the old radio name? The name was > supposed to be used locally by the facet itself so it should not > interfere with radios in other facets, whereas ID is global so it needs > to be unique. > In sudo and hbac rule enable radio button. Radios with same name are interfering on whole page. If you click at one, others gets unset. This wasn't an issue before because we have always reloaded the data. Radios in single facet wasn't interfering because they had different names. Gathering data wasn't a problem, because jquery selector was constraint to widget's container. > Regardless, ACK and pushed to master. -- Petr Vobornik From abokovoy at redhat.com Wed Nov 2 16:37:21 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 2 Nov 2011 18:37:21 +0200 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <4EB15E73.8070402@redhat.com> References: <4EA810B0.5020307@redhat.com> <1319639963.5652.3.camel@balmora.brq.redhat.com> <4EA81E29.4030007@redhat.com> <4EAE8416.4060401@redhat.com> <20111031121904.GA9412@redhat.com> <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <4EB152A7.5010303@redhat.com> <20111102151114.GD14373@redhat.com> <4EB15E73.8070402@redhat.com> Message-ID: <20111102163720.GA23092@redhat.com> On Wed, 02 Nov 2011, Jan Cholasta wrote: > >Callable instances are a consequence of the above -- > >Command.__call__() does use properties that are changed due to > >finalize() being run. In fact, there are so many places in FreeIPA > >where we simply expect that foo.args/params/output_params/output is > >both iterable and callable instance that it is not even funny. > > > >Now, the process established by the proposed patch is actually > >dependent only on the fact that bound instance has finalize() method. > >This is quite generic and can be used for all other types of objects. > > > > The real problem is in the API class so it should be fixed there, > not worked around in Plugin subclasses. Please explain why you do think real problem is in API class. Finalize() is needed purely to being able to split plugins' property initialization into stages before loading plugins and after due to the fact that each and every plugin could contribute commands and additions to the same objects. While these stages are separate, plugins can access properties and commands they want and it is duty of a plugin to get its property working right. We have finalize() method exactly for this purpose. API class' attempt to call finalize() on each plugin's instance at once is a poor replacement to detecting access to not-yet-initialized properties. We can implement that access like you have proposed with __getattribute__ but it has additional cost for all other properties that don't need post-initialization (finalization) and, what's more important, they incur these costs irrespective whether initialization has happened or not. That's bad and my patch shows it is actually noticeable as the performance difference, for example, of 'ipa dnsrecord-find example.com' between the two patches is about 2x in favor of my patch (1.624s vs 0.912s). Regarding other objects, here is the list of places where we define specific finalizers: $ git grep 'def finalize' ipalib/cli.py: def finalize(self): ipalib/frontend.py: def finalize(self): ipalib/plugable.py: def finalize(self): ipalib/plugable.py: def finalize(self): ipaserver/rpcserver.py: def finalize(self): ipaserver/rpcserver.py: def finalize(self): ipaserver/rpcserver.py: def finalize(self): tests/test_ipalib/test_cli.py: def finalize(self): tests/test_ipalib/test_config.py: def finalize_core(self, ctx, **defaults): tests/util.py: def finalize(self, *plugins, **kw): As you can see, there NO plugins that define their own finalizers, thus, all of them rely on API.finalize(), help.finalize(), and Plugin.finalize()/Command.finalize(). ipaserver/rpcserver.py finalizers are fine as well, I have checked them. Updated patch is attached. It addresses all comments from Martin. [root at vm-114 freeipa-2-1-speedup]# time ipa dnsrecord-find ipa.local >/dev/null real 0m0.883s user 0m0.504s sys 0m0.136s [root at vm-114 freeipa-2-1-speedup]# time ipa user-find >/dev/null real 0m0.887s user 0m0.486s sys 0m0.141s [root at vm-114 freeipa-2-1-speedup]# time ipa group-find >/dev/null real 0m0.933s user 0m0.446s sys 0m0.169s [root at vm-114 freeipa-2-1-speedup]# time ipa help >/dev/null real 0m0.624s user 0m0.479s sys 0m0.133s [root at vm-114 freeipa-2-1-speedup]# time ipa group >/dev/null real 0m0.612s user 0m0.475s sys 0m0.126s [root at vm-114 freeipa-2-1-speedup]# time ipa user >/dev/null real 0m0.617s user 0m0.472s sys 0m0.131s -- / Alexander Bokovoy From abokovoy at redhat.com Wed Nov 2 16:38:49 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 2 Nov 2011 18:38:49 +0200 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <20111102163720.GA23092@redhat.com> References: <1319639963.5652.3.camel@balmora.brq.redhat.com> <4EA81E29.4030007@redhat.com> <4EAE8416.4060401@redhat.com> <20111031121904.GA9412@redhat.com> <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <4EB152A7.5010303@redhat.com> <20111102151114.GD14373@redhat.com> <4EB15E73.8070402@redhat.com> <20111102163720.GA23092@redhat.com> Message-ID: <20111102163848.GB23092@redhat.com> (actual patch attached!) On Wed, 02 Nov 2011, Alexander Bokovoy wrote: > On Wed, 02 Nov 2011, Jan Cholasta wrote: > > >Callable instances are a consequence of the above -- > > >Command.__call__() does use properties that are changed due to > > >finalize() being run. In fact, there are so many places in FreeIPA > > >where we simply expect that foo.args/params/output_params/output is > > >both iterable and callable instance that it is not even funny. > > > > > >Now, the process established by the proposed patch is actually > > >dependent only on the fact that bound instance has finalize() method. > > >This is quite generic and can be used for all other types of objects. > > > > > > > The real problem is in the API class so it should be fixed there, > > not worked around in Plugin subclasses. > Please explain why you do think real problem is in API class. > Finalize() is needed purely to being able to split plugins' property > initialization into stages before loading plugins and after due to the > fact that each and every plugin could contribute commands and > additions to the same objects. While these stages are separate, > plugins can access properties and commands they want and it is duty of > a plugin to get its property working right. We have finalize() method > exactly for this purpose. > > API class' attempt to call finalize() on each plugin's instance at > once is a poor replacement to detecting access to > not-yet-initialized properties. We can implement that access like you > have proposed with __getattribute__ but it has additional cost for all > other properties that don't need post-initialization (finalization) > and, what's more important, they incur these costs irrespective > whether initialization has happened or not. That's bad and my patch > shows it is actually noticeable as the performance difference, for > example, of 'ipa dnsrecord-find example.com' between the two patches > is about 2x in favor of my patch (1.624s vs 0.912s). > > Regarding other objects, here is the list of places where we define > specific finalizers: > $ git grep 'def finalize' > ipalib/cli.py: def finalize(self): > ipalib/frontend.py: def finalize(self): > ipalib/plugable.py: def finalize(self): > ipalib/plugable.py: def finalize(self): > ipaserver/rpcserver.py: def finalize(self): > ipaserver/rpcserver.py: def finalize(self): > ipaserver/rpcserver.py: def finalize(self): > tests/test_ipalib/test_cli.py: def finalize(self): > tests/test_ipalib/test_config.py: def finalize_core(self, ctx, **defaults): > tests/util.py: def finalize(self, *plugins, **kw): > > As you can see, there NO plugins that define their own finalizers, > thus, all of them rely on API.finalize(), help.finalize(), and > Plugin.finalize()/Command.finalize(). ipaserver/rpcserver.py > finalizers are fine as well, I have checked them. > > Updated patch is attached. It addresses all comments from Martin. > > [root at vm-114 freeipa-2-1-speedup]# time ipa dnsrecord-find ipa.local >/dev/null > > real 0m0.883s > user 0m0.504s > sys 0m0.136s > [root at vm-114 freeipa-2-1-speedup]# time ipa user-find >/dev/null > > real 0m0.887s > user 0m0.486s > sys 0m0.141s > [root at vm-114 freeipa-2-1-speedup]# time ipa group-find >/dev/null > > real 0m0.933s > user 0m0.446s > sys 0m0.169s > [root at vm-114 freeipa-2-1-speedup]# time ipa help >/dev/null > > real 0m0.624s > user 0m0.479s > sys 0m0.133s > [root at vm-114 freeipa-2-1-speedup]# time ipa group >/dev/null > > real 0m0.612s > user 0m0.475s > sys 0m0.126s > [root at vm-114 freeipa-2-1-speedup]# time ipa user >/dev/null > > real 0m0.617s > user 0m0.472s > sys 0m0.131s > > -- > / Alexander Bokovoy -- / Alexander Bokovoy -------------- next part -------------- >From be4fc63a16e9fdd5fa1c58aa5a5267b8e06e44ce Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Wed, 2 Nov 2011 12:24:20 +0200 Subject: [PATCH] Perform late initialization of FreeIPA plugins https://fedorahosted.org/freeipa/ticket/1336 When plugins are loaded, instances of the provided objects and commands are registered in the API instance. The patch changes finalization process to apply on first access to the Command instance that would cause either access to properties (args, options, params) or execution of the command itself. The patch gives 2x boost for client-side commands like help and 3x boost for commands that go to the server side. All performance numbers are approximate and may vary a lot. --- ipalib/frontend.py | 50 ++++++++++++++++++++++++++++++++++++ ipalib/plugable.py | 12 ++++++--- tests/test_ipalib/test_frontend.py | 7 +++-- 3 files changed, 62 insertions(+), 7 deletions(-) diff --git a/ipalib/frontend.py b/ipalib/frontend.py index 61e7f493f8a8e30a1a189d06cd6a69893319deaf..22d77372b8cb58bb2e922b3fb158ee1025b0d242 100644 --- a/ipalib/frontend.py +++ b/ipalib/frontend.py @@ -64,6 +64,44 @@ def entry_count(entry): return num_entries +class _MirrorTrap(object): + """ + _MirrorTrap is a special class to support late initialization in FreeIPA. + _MirrorTrap instances can be used instead of None for properties that + change their state due to Plugin#finalize() method called. + + As Plugin#finalize() might be computationally expensive, objects that could not + act without such properties, should initialize them to _MirrorTrap() instance + in their __init__() method so that _MirrorTrap instance would force the object + to finalize() before returning the value of the property. + + Usage pattern is following: + self.args = _MirrorTrap(self, 'args') + + Pass the reference to the object holding the attribute and the name of the attribute. + On first access to the attribute, _MirrorTrip will try to call object's finalize() method + and expects that the attribute will be re-assigned with new (proper) value. + + In many places in FreeIPA, an attribute gets assigned with NameSpace() instance during + finalize() call. + """ + def __init__(self, master, attr): + self.master = master + self.attr = attr + + def __call__(self, **args): + self.master.finalize() + # At this point master.attr points to proper object + return getattr(self.master, self.attr)(**args) + + def __iter__(self): + self.master.finalize() + return getattr(self.master, self.attr).__iter__() + + def len(self): + self.master.finalize() + return getattr(self.master, self.attr).len() + class HasParam(Plugin): """ @@ -404,6 +442,14 @@ class Command(HasParam): msg_summary = None msg_truncated = _('Results are truncated, try a more specific search') + def __init__(self): + self.args = _MirrorTrap(self, 'args') + self.options = _MirrorTrap(self, 'options') + self.output_params = _MirrorTrap(self, 'output_params') + self.output = _MirrorTrap(self, 'output') + + super(Command, self).__init__() + def __call__(self, *args, **options): """ Perform validation and then execute the command. @@ -411,6 +457,10 @@ class Command(HasParam): If not in a server context, the call will be forwarded over XML-RPC and the executed an the nearest IPA server. """ + # Plugin instance must be finalized before we get to execution + if not self.__dict__['_Plugin__finalized']: + self.finalize() + params = self.args_options_2_params(*args, **options) self.debug( 'raw: %s(%s)', self.name, ', '.join(self._repr_iter(**params)) diff --git a/ipalib/plugable.py b/ipalib/plugable.py index b0e415656e0428eb164c35a2862fcfbf50883381..e0e1eacbb14a0d8278832bcbda358264bf233100 100644 --- a/ipalib/plugable.py +++ b/ipalib/plugable.py @@ -207,6 +207,7 @@ class Plugin(ReadOnly): self.label ) ) + self.__finalized = False def __get_api(self): """ @@ -220,6 +221,7 @@ class Plugin(ReadOnly): def finalize(self): """ """ + self.__finalized = True if not is_production_mode(self): lock(self) @@ -637,10 +639,12 @@ class API(DictProxy): if not production_mode: assert p.instance.api is self - for p in plugins.itervalues(): - p.instance.finalize() - if not production_mode: - assert islocked(p.instance) is True + if self.env.context != 'cli': + for p in plugins.itervalues(): + p.instance.finalize() + if not production_mode: + assert islocked(p.instance) + object.__setattr__(self, '_API__finalized', True) tuple(PluginInfo(p) for p in plugins.itervalues()) object.__setattr__(self, 'plugins', diff --git a/tests/test_ipalib/test_frontend.py b/tests/test_ipalib/test_frontend.py index 0f6aecb3d137872fd89e89f9db64e1d9aab7d8c1..261823c4ff90b05f4c24fa0bfc71fb3e139b6a4b 100644 --- a/tests/test_ipalib/test_frontend.py +++ b/tests/test_ipalib/test_frontend.py @@ -30,6 +30,7 @@ from ipalib import frontend, backend, plugable, errors, parameters, config from ipalib import output from ipalib.parameters import Str from ipapython.version import API_VERSION +import types def test_RULE_FLAG(): assert frontend.RULE_FLAG == 'validation_rule' @@ -252,7 +253,7 @@ class test_Command(ClassChecker): """ Test the ``ipalib.frontend.Command.args`` instance attribute. """ - assert self.cls().args is None + assert isinstance(self.cls().args, (types.NoneType, frontend._MirrorTrap)) o = self.cls() o.finalize() assert type(o.args) is plugable.NameSpace @@ -301,7 +302,7 @@ class test_Command(ClassChecker): """ Test the ``ipalib.frontend.Command.options`` instance attribute. """ - assert self.cls().options is None + assert isinstance(self.cls().options, (types.NoneType, frontend._MirrorTrap)) o = self.cls() o.finalize() assert type(o.options) is plugable.NameSpace @@ -323,7 +324,7 @@ class test_Command(ClassChecker): Test the ``ipalib.frontend.Command.output`` instance attribute. """ inst = self.cls() - assert inst.output is None + assert isinstance(inst.output, (types.NoneType, frontend._MirrorTrap)) inst.finalize() assert type(inst.output) is plugable.NameSpace assert list(inst.output) == ['result'] -- 1.7.7 From nkinder at redhat.com Wed Nov 2 16:59:59 2011 From: nkinder at redhat.com (Nathan Kinder) Date: Wed, 02 Nov 2011 09:59:59 -0700 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <1320167298.17206.126.camel@localhost.localdomain> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> Message-ID: <4EB1770F.5000109@redhat.com> On 11/01/2011 10:08 AM, Ade Lee wrote: > On Tue, 2011-11-01 at 12:49 -0400, Simo Sorce wrote: >> On Tue, 2011-11-01 at 12:40 -0400, Richard Megginson wrote: >>> ----- Original Message ----- >>>> >>>> >>>> We had a brief discussion on unifying the PKI and IPA Directory >>>> Server instances. Here are my notes from it. Please fill out the >>>> details and correct me if I've mis-stated anything below. >>>> >>>> >>>> Issues: >>>> >>>> >>>> >>> Do IPA and PKI use different suffixes? >> Currently not as we use completely separate instances, but we will be >> able to use different suffixes for some stuff. >> > I would suggest that if we use the same database, then we use different > suffixes. For one thing, we will want to be able to set ACIs so that > the information here is not publicly browsable. > > It will also be much easier to limit the pki users ability to touch the > rest of the db, and visa versa. > > It also makes it less likely that upgrade scripts will stomp on each > other. We really should use separate suffixes/backends for the reasons above in addition to other benefits. Replication is at the suffix level, so this adds the ability to not replicate PKI data is that is something we ever decide we need to do. Indexes are also configured at the suffix/backend level, so we can have separate indexes defined for the PKI and IPA user trees. >>>> 1. >>>> >>>> Both make changes to Config. One identified conflict is he >>>> configuration of the Uniqueness plugin >>> It may be easy to enhance this plugin and other plugins to allow different configuration per subtree. >> If we confirm this conflict this will become a requirement before we can >> proceed. What is this conflict? Many of our plug-ins have this ability already, and we should add it where it is missing. The RHDS documentation shows that the attribute uniqueness plug-in can be configured by subtree already. >>>> 2. >>>> >>>> PKI uses Directory Manager. This is insecure. Can it use a differen, >>>> limited admin? >>> Or use ldapi? I don't think ldapjdk can use ldapi. >> It's a matter of trust for me. I do not want to trust PKI to have free >> reign on all data. I want it to be confined to only what it needs. >> >> So we can use ldapi and user mapping, but we wouldn't map the user to DM >> anyway. Agreed. If the roles were reversed, we would not want IPA to be able to have free reign on the PKI data. It makes sense to have separate admin roles and only use DM where absolutely necessary. >>>> 3. >>>> >>>> Index strategies are different >>> Use a union? e.g. if ipa needs attribute "a" indexed for equality only, but PKI needs it indexed for presence and substring only, then we can just index it for eq, sub, and pres. If we use separate suffixes/backends, we can configure the indexes differently. >> The problem here is finding out and how to make sure pki vs ds/ipa >> install and upgrade scripts do not stomp on each other. >>>> 4. >>>> >>>> make sure we have a union of the required sets of plugins >>>> 5. >>>> >>>> PKI needs to set D.S. Default Name context >>> What is this? >> See my other mail, we need DS to support setting defaultNamingContext in >> rootdse. Is there a RFE opened against 389 for this already? It should be pretty easy to add the ability to configure a default naming context to be displayed in the rootDSE. >>>> 6. >>>> >>>> If PKI uses the IPA datastore for users, it needs to creat the user >>>> with all the right prerequisites (object class, defaults) >>> If both PKI and IPA use structural objectclasses, we may have to create corresponding auxiliary objectclasses so that you can mix-in both sets of objectclasses while having only one structural objectclass per entry. >> The problem here is much bigger, PKI simply do not have enough >> information to create a proper IPA user, so it should not be allowed to. >> This is an example of why I want to tightly control through ACIs what >> PKI can do and prevent it from causing "issues". >> > If we do this integration, then I'm OK with IPA creating the users. > >> Simo. >> > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From ayoung at redhat.com Wed Nov 2 20:03:42 2011 From: ayoung at redhat.com (Adam Young) Date: Wed, 02 Nov 2011 16:03:42 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <4EB1770F.5000109@redhat.com> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> <4EB1770F.5000109@redhat.com> Message-ID: <4EB1A21E.9020208@redhat.com> To clarify: there are two types of Data stored in the PKI CA DS instances. One is Users and groups (IdM), and the other is certificates and requests. The CA currently administers its own users: creates, add deletes, add privs and so forth. If we extract the IdM objects from the CA control, we will need to build another mechanism to manage them. The Certificates will stay in their own suffix. I don't think anyone disagrees about this. I'm trying to think through the steps of using the IPA user database for PKI. If I undertand the end state, the general flow will be driven from ipa-server-install and will look like this: 1. Create a unified DS instance. We can continue to name it the way that IPA does: All caps the hostname. 2. Apply the LDAP schema LDIFs to it. At this point, we probably want to create the whole IPA schema, and the cmsUser as well 3. Call PKISilent (or equivalent) passing the info for the Unified directory server instance, to include the IPA tree for the users. 4. PKISilent will create the PKI admin user, to include its certificiate in the IPA tree. This user will be half baked from an IPA perspective, but will allow administration of the CA. 5. Create a PKIDirectory Manager account that has complete control over only the PKI suffix, and not the IPA suffix. Modify the PKI code to use only this account. This account should be able to authenticate against the Dir Srv using a client certificate, stored in the PKI specific NSS database. 6. Restart the CA. 7. Continue the IPA server install from the. Fix up the Admin user so that it works for IPA. 8. Disable the Directory Manager's password. From here on out, access to the Dir Srv should be either certificate or Keytab only. After IPA is up and running, the management of the User Database is done by IPA. One thing that several people have asked for in IPA is the ability to manage external Roles: ones that don't map to ACIs. PKI has this requirement as well. There are a couple approaches we could take: 1. Attempt to use the current Role mechanism in IPA. This gets hidden to an outside user, but that should be OK. Checking if a user is a PKI Admin, Agent, or Auditor should be done only by a privileged account anyway. 2. Use a User Group: PKIAdmins, PKIAgents, and PKIAuditors. This is what I would suggest. 3. Create an external mechanism. Once IPA has assumed control of the IdM DB, we will still need to be able to get user certificates into the user records CMSUser field. I do not know CS well enough to say how that would work, but I assume it will be one of these two mechanisms: 1. Use the CA Administrative interface to modify an existing user to have the certificate or 2. Add a mechanism to IPA to request and apply the certificate to the IPA User. I'm guessing that the flow would be something like this: 1. Create a user (ipa user-add) 2. Assign them to the PKIAgents groups 3. Call the CA Admin interface to make the user an agent. If we do it this way, the PKI instance will need to be able to modify the IPA user record. Alternatively: 1. ipa user-add 2. ipa group-add-member 3. ipa user-cert As an added bonus, we get user certs. One discussion we've had on the side is about scalability. As the Databases increase in size, it might become impractical to fully synchroize the user database over to a machine that is dedicated to Certificate operations. For example, we might have a public facing machine in the DMZ that is servering CRLs and OCSP to the world. This machine would only need the subset of users that can act as PKI admins. In this case, we might build a custom script for replicating a subset of the IPA data one direction only: from one of the fully replicated instance to the DMZ instance. This is not something we foresee doing in a near term release, but should be discussed and fleshed out. From alee at redhat.com Wed Nov 2 20:44:47 2011 From: alee at redhat.com (Ade Lee) Date: Wed, 02 Nov 2011 16:44:47 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <4EB1A21E.9020208@redhat.com> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> <4EB1770F.5000109@redhat.com> <4EB1A21E.9020208@redhat.com> Message-ID: <1320266688.17206.179.camel@localhost.localdomain> On Wed, 2011-11-02 at 16:03 -0400, Adam Young wrote: > To clarify: there are two types of Data stored in the PKI CA DS > instances. One is Users and groups (IdM), and the other is > certificates and requests. > > The CA currently administers its own users: creates, add deletes, add > privs and so forth. If we extract the IdM objects from the CA > control, we will need to build another mechanism to manage them. > > The Certificates will stay in their own suffix. I don't think anyone > disagrees about this. > > I'm trying to think through the steps of using the IPA user database for > PKI. If I undertand the end state, the general flow will be driven from > ipa-server-install and will look like this: > > 1. Create a unified DS instance. We can continue to name it the way > that IPA does: All caps the hostname. > 2. Apply the LDAP schema LDIFs to it. At this point, we probably want > to create the whole IPA schema, and the cmsUser as well For clarification, cmsuser is just an object that is defined in the PKI schema, not the actual admin user created in the install itself. It may be advantageous to actually pre-create this user when applying schema LDIFS and just have pkisilent add the user certs. > 3. Call PKISilent (or equivalent) passing the info for the Unified > directory server instance, to include the IPA tree for the users. > 4. PKISilent will create the PKI admin user, to include its > certificiate in the IPA tree. This user will be half baked from an IPA > perspective, but will allow administration of the CA. Pre-creating this user may solve this problem. You can pre-create a user with all the required IPA and PKI attributes and just have pkisilent add the user cert needed. As we will be running as directory manager in this case, we will permissions to do this. > 5. Create a PKIDirectory Manager account that has complete control over > only the PKI suffix, and not the IPA suffix. Modify the PKI code to > use only this account. This account should be able to authenticate > against the Dir Srv using a client certificate, stored in the PKI > specific NSS database. This of course involves setting up the relevant ACIs. We may want to consider the reverse. That only certain IPA accounts should have access to the PKI data. > 6. Restart the CA. > 7. Continue the IPA server install from the. Fix up the Admin user so > that it works for IPA. Not needed if we pre-populate as mentioned above. > 8. Disable the Directory Manager's password. From here on out, access > to the Dir Srv should be either certificate or Keytab only. > > > After IPA is up and running, the management of the User Database is done > by IPA. > One thing that several people have asked for in IPA is the ability to > manage external Roles: ones that don't map to ACIs. PKI has this > requirement as well. There are a couple approaches we could take: > > 1. Attempt to use the current Role mechanism in IPA. This gets hidden > to an outside user, but that should be OK. Checking if a user is a PKI > Admin, Agent, or Auditor should be done only by a privileged account anyway. > > 2. Use a User Group: PKIAdmins, PKIAgents, and PKIAuditors. This > is what I would suggest. > > 3. Create an external mechanism. > > > Once IPA has assumed control of the IdM DB, we will still need to be > able to get user certificates into the user records CMSUser field. I > do not know CS well enough to say how that would work, but I assume it > will be one of these two mechanisms: > > 1. Use the CA Administrative interface to modify an existing user to > have the certificate > or > 2. Add a mechanism to IPA to request and apply the certificate to the > IPA User. > > I'm guessing that the flow would be something like this: > > 1. Create a user (ipa user-add) > 2. Assign them to the PKIAgents groups > 3. Call the CA Admin interface to make the user an agent. > > If we do it this way, the PKI instance will need to be able to modify > the IPA user record. Alternatively: > > 1. ipa user-add > 2. ipa group-add-member > 3. ipa user-cert So, a user becomes an agent on the ca by having a certificate in the user record and being a member of the relevant admin, agent or auditor group. I see this as follows: 1. ipa cms-user-add (add a user and add the auxilliary cmsuser object class) 2. ipa user-cert (contact the ca and get a certificate for this user, add this cert to the user record in the ipa database) 3. ipa group-add-member (add the user to the relevant group) At no point does PKI need to modify anything in the IPA database. > > As an added bonus, we get user certs. > > > One discussion we've had on the side is about scalability. As the > Databases increase in size, it might become impractical to fully > synchroize the user database over to a machine that is dedicated to > Certificate operations. For example, we might have a public facing > machine in the DMZ that is servering CRLs and OCSP to the world. This > machine would only need the subset of users that can act as PKI > admins. In this case, we might build a custom script for replicating a > subset of the IPA data one direction only: from one of the fully > replicated instance to the DMZ instance. This is not something we > foresee doing in a near term release, but should be discussed and > fleshed out. > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From simo at redhat.com Wed Nov 2 22:06:05 2011 From: simo at redhat.com (Simo Sorce) Date: Wed, 02 Nov 2011 18:06:05 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <1320266688.17206.179.camel@localhost.localdomain> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> <4EB1770F.5000109@redhat.com> <4EB1A21E.9020208@redhat.com> <1320266688.17206.179.camel@localhost.localdomain> Message-ID: <1320271565.7734.588.camel@willson.li.ssimo.org> On Wed, 2011-11-02 at 16:44 -0400, Ade Lee wrote: > On Wed, 2011-11-02 at 16:03 -0400, Adam Young wrote: [...] > So, a user becomes an agent on the ca by having a certificate in the > user record and being a member of the relevant admin, agent or auditor > group. > > I see this as follows: > 1. ipa cms-user-add (add a user and add the auxilliary cmsuser object > class) > 2. ipa user-cert (contact the ca and get a certificate for this user, > add this cert to the user record in the ipa database) > 3. ipa group-add-member (add the user to the relevant group) > > At no point does PKI need to modify anything in the IPA database. Sounds reasonable. Can you post a link to the schema that would be added to IPA objects ? Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Wed Nov 2 22:19:03 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 02 Nov 2011 18:19:03 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <1320271565.7734.588.camel@willson.li.ssimo.org> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> <4EB1770F.5000109@redhat.com> <4EB1A21E.9020208@redhat.com> <1320266688.17206.179.camel@localhost.localdomain> <1320271565.7734.588.camel@willson.li.ssimo.org> Message-ID: <4EB1C1D7.4020901@redhat.com> Simo Sorce wrote: > On Wed, 2011-11-02 at 16:44 -0400, Ade Lee wrote: >> On Wed, 2011-11-02 at 16:03 -0400, Adam Young wrote: > [...] >> So, a user becomes an agent on the ca by having a certificate in the >> user record and being a member of the relevant admin, agent or auditor >> group. >> >> I see this as follows: >> 1. ipa cms-user-add (add a user and add the auxilliary cmsuser object >> class) >> 2. ipa user-cert (contact the ca and get a certificate for this user, >> add this cert to the user record in the ipa database) >> 3. ipa group-add-member (add the user to the relevant group) >> >> At no point does PKI need to modify anything in the IPA database. > > Sounds reasonable. > Can you post a link to the schema that would be added to IPA objects ? > > Simo. > IIRC the user we create in CS now has the description attribute set up in a very specific way. Is that still required? rob From ayoung at redhat.com Thu Nov 3 00:25:44 2011 From: ayoung at redhat.com (Adam Young) Date: Wed, 02 Nov 2011 20:25:44 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <4EB1C1D7.4020901@redhat.com> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> <4EB1770F.5000109@redhat.com> <4EB1A21E.9020208@redhat.com> <1320266688.17206.179.camel@localhost.localdomain> <1320271565.7734.588.camel@willson.li.ssimo.org> <4EB1C1D7.4020901@redhat.com> Message-ID: <4EB1DF88.6020304@redhat.com> On 11/02/2011 06:19 PM, Rob Crittenden wrote: > Simo Sorce wrote: >> On Wed, 2011-11-02 at 16:44 -0400, Ade Lee wrote: >>> On Wed, 2011-11-02 at 16:03 -0400, Adam Young wrote: >> [...] >>> So, a user becomes an agent on the ca by having a certificate in the >>> user record and being a member of the relevant admin, agent or auditor >>> group. >>> >>> I see this as follows: >>> 1. ipa cms-user-add (add a user and add the auxilliary cmsuser object >>> class) >>> 2. ipa user-cert (contact the ca and get a certificate for this user, >>> add this cert to the user record in the ipa database) >>> 3. ipa group-add-member (add the user to the relevant group) >>> >>> At no point does PKI need to modify anything in the IPA database. >> >> Sounds reasonable. >> Can you post a link to the schema that would be added to IPA objects ? >> >> Simo. >> I think this is it: http://svn.fedorahosted.org/svn/pki/trunk/pki/base/ca/shared/conf/schema.ldif Look for cmsuser. The cert seems to comes from 05rfc4523.ldif and is added in 06inetorgperson.ldif Which is already in our user record. CMS only seems to "require" usertype, which is a string, and "allows" userstate which is an integer. > IIRC the user we create in CS now has the description attribute set up > in a very specific way. Is that still required? > > rob > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From simo at redhat.com Thu Nov 3 04:56:35 2011 From: simo at redhat.com (Simo Sorce) Date: Thu, 03 Nov 2011 00:56:35 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <4EB1DF88.6020304@redhat.com> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> <4EB1770F.5000109@redhat.com> <4EB1A21E.9020208@redhat.com> <1320266688.17206.179.camel@localhost.localdomain> <1320271565.7734.588.camel@willson.li.ssimo.org> <4EB1C1D7.4020901@redhat.com> <4EB1DF88.6020304@redhat.com> Message-ID: <1320296195.7734.599.camel@willson.li.ssimo.org> On Wed, 2011-11-02 at 20:25 -0400, Adam Young wrote: > On 11/02/2011 06:19 PM, Rob Crittenden wrote: > > Simo Sorce wrote: > >> On Wed, 2011-11-02 at 16:44 -0400, Ade Lee wrote: > >>> On Wed, 2011-11-02 at 16:03 -0400, Adam Young wrote: > >> [...] > >>> So, a user becomes an agent on the ca by having a certificate in the > >>> user record and being a member of the relevant admin, agent or auditor > >>> group. > >>> > >>> I see this as follows: > >>> 1. ipa cms-user-add (add a user and add the auxilliary cmsuser object > >>> class) > >>> 2. ipa user-cert (contact the ca and get a certificate for this user, > >>> add this cert to the user record in the ipa database) > >>> 3. ipa group-add-member (add the user to the relevant group) > >>> > >>> At no point does PKI need to modify anything in the IPA database. > >> > >> Sounds reasonable. > >> Can you post a link to the schema that would be added to IPA objects ? > >> > >> Simo. > >> > I think this is it: > > http://svn.fedorahosted.org/svn/pki/trunk/pki/base/ca/shared/conf/schema.ldif > > Look for cmsuser. Unfortunately it looks like the cmsuser objectclass is of type structural, which means it cannot be added to existing records. > The cert seems to comes from > > 05rfc4523.ldif > > and is added in > > 06inetorgperson.ldif > > Which is already in our user record. > > CMS only seems to "require" usertype, which is a string, and "allows" > userstate which is an integer. I wonder if we can convince PKI to use a different schema to reprsent this information. We can use Roles or Groups to tell what type of user a user is, not sure about the state as that schema file has exactly the same comment for both usertype and userstate, seems a bug. > > IIRC the user we create in CS now has the description attribute set up > > in a very specific way. Is that still required? What is description used for ? Simo. -- Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Thu Nov 3 08:30:27 2011 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 03 Nov 2011 09:30:27 +0100 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <20111102163848.GB23092@redhat.com> References: <1319639963.5652.3.camel@balmora.brq.redhat.com> <4EA81E29.4030007@redhat.com> <4EAE8416.4060401@redhat.com> <20111031121904.GA9412@redhat.com> <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <4EB152A7.5010303@redhat.com> <20111102151114.GD14373@redhat.com> <4EB15E73.8070402@redhat.com> <20111102163720.GA23092@redhat.com> <20111102163848.GB23092@redhat.com> Message-ID: <1320309030.23357.3.camel@balmora.brq.redhat.com> On Wed, 2011-11-02 at 18:38 +0200, Alexander Bokovoy wrote: > (actual patch attached!) > > On Wed, 02 Nov 2011, Alexander Bokovoy wrote: > > On Wed, 02 Nov 2011, Jan Cholasta wrote: > > > >Callable instances are a consequence of the above -- > > > >Command.__call__() does use properties that are changed due to > > > >finalize() being run. In fact, there are so many places in FreeIPA > > > >where we simply expect that foo.args/params/output_params/output is > > > >both iterable and callable instance that it is not even funny. > > > > > > > >Now, the process established by the proposed patch is actually > > > >dependent only on the fact that bound instance has finalize() method. > > > >This is quite generic and can be used for all other types of objects. > > > > > > > > > > The real problem is in the API class so it should be fixed there, > > > not worked around in Plugin subclasses. > > Please explain why you do think real problem is in API class. > > Finalize() is needed purely to being able to split plugins' property > > initialization into stages before loading plugins and after due to the > > fact that each and every plugin could contribute commands and > > additions to the same objects. While these stages are separate, > > plugins can access properties and commands they want and it is duty of > > a plugin to get its property working right. We have finalize() method > > exactly for this purpose. > > > > API class' attempt to call finalize() on each plugin's instance at > > once is a poor replacement to detecting access to > > not-yet-initialized properties. We can implement that access like you > > have proposed with __getattribute__ but it has additional cost for all > > other properties that don't need post-initialization (finalization) > > and, what's more important, they incur these costs irrespective > > whether initialization has happened or not. That's bad and my patch > > shows it is actually noticeable as the performance difference, for > > example, of 'ipa dnsrecord-find example.com' between the two patches > > is about 2x in favor of my patch (1.624s vs 0.912s). > > > > Regarding other objects, here is the list of places where we define > > specific finalizers: > > $ git grep 'def finalize' > > ipalib/cli.py: def finalize(self): > > ipalib/frontend.py: def finalize(self): > > ipalib/plugable.py: def finalize(self): > > ipalib/plugable.py: def finalize(self): > > ipaserver/rpcserver.py: def finalize(self): > > ipaserver/rpcserver.py: def finalize(self): > > ipaserver/rpcserver.py: def finalize(self): > > tests/test_ipalib/test_cli.py: def finalize(self): > > tests/test_ipalib/test_config.py: def finalize_core(self, ctx, **defaults): > > tests/util.py: def finalize(self, *plugins, **kw): > > > > As you can see, there NO plugins that define their own finalizers, > > thus, all of them rely on API.finalize(), help.finalize(), and > > Plugin.finalize()/Command.finalize(). ipaserver/rpcserver.py > > finalizers are fine as well, I have checked them. > > > > Updated patch is attached. It addresses all comments from Martin. > > > > [root at vm-114 freeipa-2-1-speedup]# time ipa dnsrecord-find ipa.local >/dev/null > > > > real 0m0.883s > > user 0m0.504s > > sys 0m0.136s > > [root at vm-114 freeipa-2-1-speedup]# time ipa user-find >/dev/null > > > > real 0m0.887s > > user 0m0.486s > > sys 0m0.141s > > [root at vm-114 freeipa-2-1-speedup]# time ipa group-find >/dev/null > > > > real 0m0.933s > > user 0m0.446s > > sys 0m0.169s > > [root at vm-114 freeipa-2-1-speedup]# time ipa help >/dev/null > > > > real 0m0.624s > > user 0m0.479s > > sys 0m0.133s > > [root at vm-114 freeipa-2-1-speedup]# time ipa group >/dev/null > > > > real 0m0.612s > > user 0m0.475s > > sys 0m0.126s > > [root at vm-114 freeipa-2-1-speedup]# time ipa user >/dev/null > > > > real 0m0.617s > > user 0m0.472s > > sys 0m0.131s > > > > -- > > / Alexander Bokovoy > Good, this indeed addresses all my previous concerns but one :-) $ ./makeapi Writing API to API.txt Traceback (most recent call last): File "./makeapi", line 392, in sys.exit(main()) File "./makeapi", line 376, in main rval |= make_api() File "./makeapi", line 167, in make_api fd.write('args: %d,%d,%d\n' % (len(cmd.args), len(cmd.options), len(cmd.output))) But if you change "len" functions to "__len__" it works fine. Martin From abokovoy at redhat.com Thu Nov 3 09:01:20 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 3 Nov 2011 11:01:20 +0200 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <1320309030.23357.3.camel@balmora.brq.redhat.com> References: <4EAE8416.4060401@redhat.com> <20111031121904.GA9412@redhat.com> <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <4EB152A7.5010303@redhat.com> <20111102151114.GD14373@redhat.com> <4EB15E73.8070402@redhat.com> <20111102163720.GA23092@redhat.com> <20111102163848.GB23092@redhat.com> <1320309030.23357.3.camel@balmora.brq.redhat.com> Message-ID: <20111103090119.GA26667@redhat.com> On Thu, 03 Nov 2011, Martin Kosek wrote: > Good, this indeed addresses all my previous concerns but one :-) > > $ ./makeapi > Writing API to API.txt > Traceback (most recent call last): > File "./makeapi", line 392, in > sys.exit(main()) > File "./makeapi", line 376, in main > rval |= make_api() > File "./makeapi", line 167, in make_api > fd.write('args: %d,%d,%d\n' % (len(cmd.args), len(cmd.options), len(cmd.output))) > > But if you change "len" functions to "__len__" it works fine. I suspected this. :) Ok, that and I protected self.__finalized reassignment in case Plugin#finalize() got called twice -- second time the class is locked already so self.__finalized = True will blow exception. I made it no-op for next passes. New patch attached. Survived fresh re-install. -- / Alexander Bokovoy -------------- next part -------------- >From 050e75b18a2b6856d0626edbdcabed10aa841ad3 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Wed, 2 Nov 2011 12:24:20 +0200 Subject: [PATCH] Perform late initialization of FreeIPA plugins https://fedorahosted.org/freeipa/ticket/1336 When plugins are loaded, instances of the provided objects and commands are registered in the API instance. The patch changes finalization process to apply on first access to the Command instance that would cause either access to properties (args, options, params) or execution of the command itself. The patch gives 2x boost for client-side commands like help and 3x boost for commands that go to the server side. All performance numbers are approximate and may vary a lot. --- ipalib/frontend.py | 50 ++++++++++++++++++++++++++++++++++++ ipalib/plugable.py | 13 ++++++--- tests/test_ipalib/test_frontend.py | 7 +++-- 3 files changed, 63 insertions(+), 7 deletions(-) diff --git a/ipalib/frontend.py b/ipalib/frontend.py index 61e7f493f8a8e30a1a189d06cd6a69893319deaf..45d254be23d6c8b0c359c8a4e84d0658a5bf4fe7 100644 --- a/ipalib/frontend.py +++ b/ipalib/frontend.py @@ -64,6 +64,44 @@ def entry_count(entry): return num_entries +class _MirrorTrap(object): + """ + _MirrorTrap is a special class to support late initialization in FreeIPA. + _MirrorTrap instances can be used instead of None for properties that + change their state due to Plugin#finalize() method called. + + As Plugin#finalize() might be computationally expensive, objects that could not + act without such properties, should initialize them to _MirrorTrap() instance + in their __init__() method so that _MirrorTrap instance would force the object + to finalize() before returning the value of the property. + + Usage pattern is following: + self.args = _MirrorTrap(self, 'args') + + Pass the reference to the object holding the attribute and the name of the attribute. + On first access to the attribute, _MirrorTrip will try to call object's finalize() method + and expects that the attribute will be re-assigned with new (proper) value. + + In many places in FreeIPA, an attribute gets assigned with NameSpace() instance during + finalize() call. + """ + def __init__(self, master, attr): + self.master = master + self.attr = attr + + def __call__(self, **args): + self.master.finalize() + # At this point master.attr points to proper object + return getattr(self.master, self.attr)(**args) + + def __iter__(self): + self.master.finalize() + return getattr(self.master, self.attr).__iter__() + + def __len__(self): + self.master.finalize() + return getattr(self.master, self.attr).__len__() + class HasParam(Plugin): """ @@ -404,6 +442,14 @@ class Command(HasParam): msg_summary = None msg_truncated = _('Results are truncated, try a more specific search') + def __init__(self): + self.args = _MirrorTrap(self, 'args') + self.options = _MirrorTrap(self, 'options') + self.output_params = _MirrorTrap(self, 'output_params') + self.output = _MirrorTrap(self, 'output') + + super(Command, self).__init__() + def __call__(self, *args, **options): """ Perform validation and then execute the command. @@ -411,6 +457,10 @@ class Command(HasParam): If not in a server context, the call will be forwarded over XML-RPC and the executed an the nearest IPA server. """ + # Plugin instance must be finalized before we get to execution + if not self.__dict__['_Plugin__finalized']: + self.finalize() + params = self.args_options_2_params(*args, **options) self.debug( 'raw: %s(%s)', self.name, ', '.join(self._repr_iter(**params)) diff --git a/ipalib/plugable.py b/ipalib/plugable.py index b0e415656e0428eb164c35a2862fcfbf50883381..d1411de32fc1a888db35e1dafdf6bf4fe8d0634b 100644 --- a/ipalib/plugable.py +++ b/ipalib/plugable.py @@ -207,6 +207,7 @@ class Plugin(ReadOnly): self.label ) ) + self.__finalized = False def __get_api(self): """ @@ -220,6 +221,8 @@ class Plugin(ReadOnly): def finalize(self): """ """ + if not self.__finalized: + self.__finalized = True if not is_production_mode(self): lock(self) @@ -637,10 +640,12 @@ class API(DictProxy): if not production_mode: assert p.instance.api is self - for p in plugins.itervalues(): - p.instance.finalize() - if not production_mode: - assert islocked(p.instance) is True + if self.env.context != 'cli': + for p in plugins.itervalues(): + p.instance.finalize() + if not production_mode: + assert islocked(p.instance) + object.__setattr__(self, '_API__finalized', True) tuple(PluginInfo(p) for p in plugins.itervalues()) object.__setattr__(self, 'plugins', diff --git a/tests/test_ipalib/test_frontend.py b/tests/test_ipalib/test_frontend.py index 0f6aecb3d137872fd89e89f9db64e1d9aab7d8c1..261823c4ff90b05f4c24fa0bfc71fb3e139b6a4b 100644 --- a/tests/test_ipalib/test_frontend.py +++ b/tests/test_ipalib/test_frontend.py @@ -30,6 +30,7 @@ from ipalib import frontend, backend, plugable, errors, parameters, config from ipalib import output from ipalib.parameters import Str from ipapython.version import API_VERSION +import types def test_RULE_FLAG(): assert frontend.RULE_FLAG == 'validation_rule' @@ -252,7 +253,7 @@ class test_Command(ClassChecker): """ Test the ``ipalib.frontend.Command.args`` instance attribute. """ - assert self.cls().args is None + assert isinstance(self.cls().args, (types.NoneType, frontend._MirrorTrap)) o = self.cls() o.finalize() assert type(o.args) is plugable.NameSpace @@ -301,7 +302,7 @@ class test_Command(ClassChecker): """ Test the ``ipalib.frontend.Command.options`` instance attribute. """ - assert self.cls().options is None + assert isinstance(self.cls().options, (types.NoneType, frontend._MirrorTrap)) o = self.cls() o.finalize() assert type(o.options) is plugable.NameSpace @@ -323,7 +324,7 @@ class test_Command(ClassChecker): Test the ``ipalib.frontend.Command.output`` instance attribute. """ inst = self.cls() - assert inst.output is None + assert isinstance(inst.output, (types.NoneType, frontend._MirrorTrap)) inst.finalize() assert type(inst.output) is plugable.NameSpace assert list(inst.output) == ['result'] -- 1.7.7 From jcholast at redhat.com Thu Nov 3 09:39:26 2011 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 03 Nov 2011 10:39:26 +0100 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <20111102163848.GB23092@redhat.com> References: <1319639963.5652.3.camel@balmora.brq.redhat.com> <4EA81E29.4030007@redhat.com> <4EAE8416.4060401@redhat.com> <20111031121904.GA9412@redhat.com> <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <4EB152A7.5010303@redhat.com> <20111102151114.GD14373@redhat.com> <4EB15E73.8070402@redhat.com> <20111102163720.GA23092@redhat.com> <20111102163848.GB23092@redhat.com> Message-ID: <4EB2614E.8090307@redhat.com> Dne 2.11.2011 17:38, Alexander Bokovoy napsal(a): > (actual patch attached!) > > On Wed, 02 Nov 2011, Alexander Bokovoy wrote: >> On Wed, 02 Nov 2011, Jan Cholasta wrote: >>>> Callable instances are a consequence of the above -- >>>> Command.__call__() does use properties that are changed due to >>>> finalize() being run. In fact, there are so many places in FreeIPA >>>> where we simply expect that foo.args/params/output_params/output is >>>> both iterable and callable instance that it is not even funny. >>>> >>>> Now, the process established by the proposed patch is actually >>>> dependent only on the fact that bound instance has finalize() method. >>>> This is quite generic and can be used for all other types of objects. >>>> >>> >>> The real problem is in the API class so it should be fixed there, >>> not worked around in Plugin subclasses. >> Please explain why you do think real problem is in API class. >> Finalize() is needed purely to being able to split plugins' property >> initialization into stages before loading plugins and after due to the >> fact that each and every plugin could contribute commands and >> additions to the same objects. While these stages are separate, >> plugins can access properties and commands they want and it is duty of >> a plugin to get its property working right. We have finalize() method >> exactly for this purpose. >> >> API class' attempt to call finalize() on each plugin's instance at >> once is a poor replacement to detecting access to >> not-yet-initialized properties. We can implement that access like you >> have proposed with __getattribute__ but it has additional cost for all >> other properties that don't need post-initialization (finalization) >> and, what's more important, they incur these costs irrespective >> whether initialization has happened or not. That's bad and my patch >> shows it is actually noticeable as the performance difference, for >> example, of 'ipa dnsrecord-find example.com' between the two patches >> is about 2x in favor of my patch (1.624s vs 0.912s). >> >> Regarding other objects, here is the list of places where we define >> specific finalizers: >> $ git grep 'def finalize' >> ipalib/cli.py: def finalize(self): >> ipalib/frontend.py: def finalize(self): >> ipalib/plugable.py: def finalize(self): >> ipalib/plugable.py: def finalize(self): >> ipaserver/rpcserver.py: def finalize(self): >> ipaserver/rpcserver.py: def finalize(self): >> ipaserver/rpcserver.py: def finalize(self): >> tests/test_ipalib/test_cli.py: def finalize(self): >> tests/test_ipalib/test_config.py: def finalize_core(self, ctx, **defaults): >> tests/util.py: def finalize(self, *plugins, **kw): >> >> As you can see, there NO plugins that define their own finalizers, >> thus, all of them rely on API.finalize(), help.finalize(), and >> Plugin.finalize()/Command.finalize(). ipaserver/rpcserver.py >> finalizers are fine as well, I have checked them. >> >> Updated patch is attached. It addresses all comments from Martin. >> >> [root at vm-114 freeipa-2-1-speedup]# time ipa dnsrecord-find ipa.local>/dev/null >> >> real 0m0.883s >> user 0m0.504s >> sys 0m0.136s >> [root at vm-114 freeipa-2-1-speedup]# time ipa user-find>/dev/null >> >> real 0m0.887s >> user 0m0.486s >> sys 0m0.141s >> [root at vm-114 freeipa-2-1-speedup]# time ipa group-find>/dev/null >> >> real 0m0.933s >> user 0m0.446s >> sys 0m0.169s >> [root at vm-114 freeipa-2-1-speedup]# time ipa help>/dev/null >> >> real 0m0.624s >> user 0m0.479s >> sys 0m0.133s >> [root at vm-114 freeipa-2-1-speedup]# time ipa group>/dev/null >> >> real 0m0.612s >> user 0m0.475s >> sys 0m0.126s >> [root at vm-114 freeipa-2-1-speedup]# time ipa user>/dev/null >> >> real 0m0.617s >> user 0m0.472s >> sys 0m0.131s >> >> -- >> / Alexander Bokovoy > The problem with the API class is that it creates all the plugin instances every time. Both mine and your patch don't change that, they deal only with finalization, which is the easier part of the lazy initialization problem. Additionally, in your patch finalize() is called only on Command subclasses, so non-Command plugins are not ReadOnly-locked like they should (currently this is done only when production mode is off). We could change API so that the plugins are initialized on-demand. That way we could probably get rid off finalize() on plugins altogether and move the initialization code into __init__() (because all the required information would be available on-demand) and the locking into API. The flaw of this approach is that it would require a number of fundamental changes, namely changing the way API class handles plugin initialization and moving the initialization of "static" Plugin properties (name, module, fullname, bases, label, ...) from instance level to class level. (Nonetheless, I think API would't suffer from a facelift - currently it is messy, hard to read code, with bits nobody ever used or uses anymore, some of the docstrings and comments aren't correct, etc.) Anyway, if we decide to go with your solution, please make sure *all* the plugins that are used are finalized (because of the locking) and add a new api.env attribute which controls the lazy finalization, so that the behavior can be overriden (actually I have already done that - see attached patch - just use "api.env.plugins_on_demand" instead of "api.env.context == 'cli'"). Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: lazycontrol.patch Type: text/x-patch Size: 1384 bytes Desc: not available URL: From mkosek at redhat.com Thu Nov 3 10:21:04 2011 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 03 Nov 2011 11:21:04 +0100 Subject: [Freeipa-devel] [PATCH] 158 Fix ipa-server-install answer cache Message-ID: <1320315666.23357.4.camel@balmora.brq.redhat.com> Current Answer Cache storing mechanism is not ideal for storing non-trivial Python types like arrays, custom classes, etc. RawConfigParser just translates values to string, which are not correctly decoded when the Answer Cache is parsed and restored in the installer. This patch replaces RawConfigParser with Python's standard pickle module, which is a recommended way for serialization in Python. https://fedorahosted.org/freeipa/ticket/2054 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-158-fix-ipa-server-install-answer-cache.patch Type: text/x-patch Size: 5301 bytes Desc: not available URL: From abokovoy at redhat.com Thu Nov 3 11:43:53 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 3 Nov 2011 13:43:53 +0200 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <4EB2614E.8090307@redhat.com> References: <4EAE8416.4060401@redhat.com> <20111031121904.GA9412@redhat.com> <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <4EB152A7.5010303@redhat.com> <20111102151114.GD14373@redhat.com> <4EB15E73.8070402@redhat.com> <20111102163720.GA23092@redhat.com> <20111102163848.GB23092@redhat.com> <4EB2614E.8090307@redhat.com> Message-ID: <20111103114352.GB26667@redhat.com> On Thu, 03 Nov 2011, Jan Cholasta wrote: > The problem with the API class is that it creates all the plugin > instances every time. Both mine and your patch don't change that, > they deal only with finalization, which is the easier part of the > lazy initialization problem. Additionally, in your patch finalize() > is called only on Command subclasses, so non-Command plugins are not > ReadOnly-locked like they should (currently this is done only when > production mode is off). I ended up with explicitly finalizing Object, Property, and Backend objects. This, by the way, proved that major slowdown is brought by the Command and Method instances as I've got statistically negligible variations of execution time, within less than 1%. > We could change API so that the plugins are initialized on-demand. > That way we could probably get rid off finalize() on plugins > altogether and move the initialization code into __init__() (because > all the required information would be available on-demand) and the > locking into API. The flaw of this approach is that it would require > a number of fundamental changes, namely changing the way API class > handles plugin initialization and moving the initialization of > "static" Plugin properties (name, module, fullname, bases, label, > ...) from instance level to class level. (Nonetheless, I think API > would't suffer from a facelift - currently it is messy, hard to read > code, with bits nobody ever used or uses anymore, some of the > docstrings and comments aren't correct, etc.) A review of API class is a good idea. However, I think it is currently enough what is in proposed patch as it gets you 2-3x speed increase already. One important feature I'd like to still keep in FreeIPA is ability to extend any object and action during plugin import phase regardless of the python source file it comes from. So far our split between 'user methods' and 'dnsrecord methods' is mostly utilitarian as they are implemented in their own source code files. However, nobody prevents you from implementing all needed modifications to all classes in the single source file and I expect this to be fairly typical to site-specific cases. This is something I'd like to keep as it has great value for all end-users and it requires loading all Python files in ipalib/plugins (and in ipaserver/plugins for server side). > Anyway, if we decide to go with your solution, please make sure > *all* the plugins that are used are finalized (because of the > locking) and add a new api.env attribute which controls the lazy > finalization, so that the behavior can be overriden (actually I have > already done that - see attached patch - just use > "api.env.plugins_on_demand" instead of "api.env.context == 'cli'"). Done. -- / Alexander Bokovoy -------------- next part -------------- >From 44ebebc2fede6f001a826fa4047abfeb02195cac Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Wed, 2 Nov 2011 12:24:20 +0200 Subject: [PATCH] Perform late initialization of FreeIPA plugins https://fedorahosted.org/freeipa/ticket/1336 When plugins are loaded, instances of the provided objects and commands are registered in the API instance. The patch changes finalization process to apply on first access to the Command instance that would cause either access to properties (args, options, params) or execution of the command itself. The patch gives 2x boost for client-side commands like help and 3x boost for commands that go to the server side. All performance numbers are approximate and may vary a lot. --- ipalib/config.py | 4 ++ ipalib/constants.py | 1 + ipalib/frontend.py | 57 ++++++++++++++++++++++++++++++++++++ ipalib/plugable.py | 28 +++++++++++++++-- makeapi | 1 + tests/test_ipalib/test_frontend.py | 6 ++-- 6 files changed, 90 insertions(+), 7 deletions(-) diff --git a/ipalib/config.py b/ipalib/config.py index 410e5f0b252fc423e9c673e4f5b62e50eaf3602e..5e3ef8d9bc529dea59890bc8974b1d455113b14c 100644 --- a/ipalib/config.py +++ b/ipalib/config.py @@ -492,6 +492,10 @@ class Env(object): if 'conf_default' not in self: self.conf_default = self._join('confdir', 'default.conf') + # Set plugins_on_demand: + if 'plugins_on_demand' not in self: + self.plugins_on_demand = (self.context == 'cli') + def _finalize_core(self, **defaults): """ Complete initialization of standard IPA environment. diff --git a/ipalib/constants.py b/ipalib/constants.py index 6d246288b0bb6ad3509fdb62616a03d678312319..7ec897b58786b5d7047d5fbdafb655b7d71a5400 100644 --- a/ipalib/constants.py +++ b/ipalib/constants.py @@ -188,6 +188,7 @@ DEFAULT_CONFIG = ( ('confdir', object), # Directory containing config files ('conf', object), # File containing context specific config ('conf_default', object), # File containing context independent config + ('plugins_on_demand', object), # Whether to finalize plugins on-demand (bool) # Set in Env._finalize_core(): ('in_server', object), # Whether or not running in-server (bool) diff --git a/ipalib/frontend.py b/ipalib/frontend.py index 61e7f493f8a8e30a1a189d06cd6a69893319deaf..92380b0b0ce59b6888b394dd2595faa0b1107e33 100644 --- a/ipalib/frontend.py +++ b/ipalib/frontend.py @@ -23,6 +23,7 @@ Base classes for all front-end plugins. import re import inspect +import types from base import lock, check_name, NameSpace from plugable import Plugin, is_production_mode from parameters import create_param, parse_param_spec, Param, Str, Flag, Password @@ -64,6 +65,46 @@ def entry_count(entry): return num_entries +class _MirrorTrap(object): + """ + _MirrorTrap is a special class to support late initialization in FreeIPA. + _MirrorTrap instances can be used instead of None for properties that + change their state due to Plugin#finalize() method called. + + As Plugin#finalize() might be computationally expensive, objects that could not + act without such properties, should initialize them to _MirrorTrap() instance + in their __init__() method so that _MirrorTrap instance would force the object + to finalize() before returning the value of the property. + + Usage pattern is following: + self.args = _MirrorTrap(self, 'args') + + Pass the reference to the object holding the attribute and the name of the attribute. + On first access to the attribute, _MirrorTrip will try to call object's finalize() method + and expects that the attribute will be re-assigned with new (proper) value. + + In many places in FreeIPA, an attribute gets assigned with NameSpace() instance during + finalize() call. + """ + def __init__(self, master, attr): + self.master = master + self.attr = attr + + def __call__(self, **args): + self.master.finalize() + # At this point master.attr points to proper object + return getattr(self.master, self.attr)(**args) + + def __iter__(self): + self.master.finalize() + return getattr(self.master, self.attr).__iter__() + + def __len__(self): + self.master.finalize() + return getattr(self.master, self.attr).__len__() + +def is_not_initialized(attr): + return isinstance(attr, (types.NoneType, _MirrorTrap)) class HasParam(Plugin): """ @@ -404,6 +445,14 @@ class Command(HasParam): msg_summary = None msg_truncated = _('Results are truncated, try a more specific search') + def __init__(self): + self.args = _MirrorTrap(self, 'args') + self.options = _MirrorTrap(self, 'options') + self.output_params = _MirrorTrap(self, 'output_params') + self.output = _MirrorTrap(self, 'output') + + super(Command, self).__init__() + def __call__(self, *args, **options): """ Perform validation and then execute the command. @@ -411,6 +460,10 @@ class Command(HasParam): If not in a server context, the call will be forwarded over XML-RPC and the executed an the nearest IPA server. """ + # Plugin instance must be finalized before we get to execution + if not self.__dict__['_Plugin__finalized']: + self.finalize() + params = self.args_options_2_params(*args, **options) self.debug( 'raw: %s(%s)', self.name, ', '.join(self._repr_iter(**params)) @@ -769,6 +822,10 @@ class Command(HasParam): loaded in self.api to determine what their custom `Command.get_args` and `Command.get_options` methods should yield. """ + if self.__dict__['_Plugin__finalized']: + # Already finalized, instance is locked and modifications will + # cause exceptions. + return self._create_param_namespace('args') if len(self.args) == 0 or not self.args[-1].multivalue: self.max_args = len(self.args) diff --git a/ipalib/plugable.py b/ipalib/plugable.py index b0e415656e0428eb164c35a2862fcfbf50883381..3e7305a9c5d2919e8c7a3f928871b0698c4f3af0 100644 --- a/ipalib/plugable.py +++ b/ipalib/plugable.py @@ -207,6 +207,7 @@ class Plugin(ReadOnly): self.label ) ) + self.__finalized = False def __get_api(self): """ @@ -220,6 +221,10 @@ class Plugin(ReadOnly): def finalize(self): """ """ + if not self.__finalized: + self.__finalized = True + else: + return if not is_production_mode(self): lock(self) @@ -637,10 +642,25 @@ class API(DictProxy): if not production_mode: assert p.instance.api is self - for p in plugins.itervalues(): - p.instance.finalize() - if not production_mode: - assert islocked(p.instance) is True + # Always finalize objects, their properties, and backends + namespaces = [] + for i in ('Object', 'Property', 'Backend'): + if i in self: + namespaces.append(self[i]) + for n in namespaces: + for p in n: + n[p].finalize() + + # Finalize remaining plugins only if forced to do so + # Otherwise they will get finalized on first access to + # properties that are affected by finalization + if not self.env.plugins_on_demand: + for p in plugins.itervalues(): + if not p.instance.__dict__['_Plugin__finalized']: + p.instance.finalize() + if not production_mode: + assert islocked(p.instance) + object.__setattr__(self, '_API__finalized', True) tuple(PluginInfo(p) for p in plugins.itervalues()) object.__setattr__(self, 'plugins', diff --git a/makeapi b/makeapi index af3b0f96bbb6cc1a705c407561a44d8d25d26485..f68cd6254f278d945e22e092d7f3610e5638c02b 100755 --- a/makeapi +++ b/makeapi @@ -357,6 +357,7 @@ def main(): validate_api=True, enable_ra=True, mode='developer', + plugins_on_demand=False, ) api.bootstrap(**cfg) diff --git a/tests/test_ipalib/test_frontend.py b/tests/test_ipalib/test_frontend.py index 0f6aecb3d137872fd89e89f9db64e1d9aab7d8c1..206e1f1f2ed759893019b9679393ded78d1e0452 100644 --- a/tests/test_ipalib/test_frontend.py +++ b/tests/test_ipalib/test_frontend.py @@ -252,7 +252,7 @@ class test_Command(ClassChecker): """ Test the ``ipalib.frontend.Command.args`` instance attribute. """ - assert self.cls().args is None + assert frontend.is_not_initialized(self.cls().args) is True o = self.cls() o.finalize() assert type(o.args) is plugable.NameSpace @@ -301,7 +301,7 @@ class test_Command(ClassChecker): """ Test the ``ipalib.frontend.Command.options`` instance attribute. """ - assert self.cls().options is None + assert frontend.is_not_initialized(self.cls().options) is True o = self.cls() o.finalize() assert type(o.options) is plugable.NameSpace @@ -323,7 +323,7 @@ class test_Command(ClassChecker): Test the ``ipalib.frontend.Command.output`` instance attribute. """ inst = self.cls() - assert inst.output is None + assert frontend.is_not_initialized(inst.output) is True inst.finalize() assert type(inst.output) is plugable.NameSpace assert list(inst.output) == ['result'] -- 1.7.7 From pvoborni at redhat.com Thu Nov 3 12:35:06 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 03 Nov 2011 13:35:06 +0100 Subject: [Freeipa-devel] [PATCH] 030 Extending facet's mechanism of gathering changes Message-ID: <4EB28A7A.3080905@redhat.com> https://fedorahosted.org/freeipa/ticket/2041 I'm not sure if update_info and other new classes should be in details.js. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0030-Extending-facet-s-mechanism-of-gathering-changes.patch Type: text/x-patch Size: 16545 bytes Desc: not available URL: From ayoung at redhat.com Thu Nov 3 13:20:22 2011 From: ayoung at redhat.com (Adam Young) Date: Thu, 03 Nov 2011 09:20:22 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <1320296195.7734.599.camel@willson.li.ssimo.org> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> <4EB1770F.5000109@redhat.com> <4EB1A21E.9020208@redhat.com> <1320266688.17206.179.camel@localhost.localdomain> <1320271565.7734.588.camel@willson.li.ssimo.org> <4EB1C1D7.4020901@redhat.com> <4EB1DF88.6020304@redhat.com> <1320296195.7734.599.camel@willson.li.ssimo.org> Message-ID: <4EB29516.5040006@redhat.com> On 11/03/2011 12:56 AM, Simo Sorce wrote: > On Wed, 2011-11-02 at 20:25 -0400, Adam Young wrote: >> On 11/02/2011 06:19 PM, Rob Crittenden wrote: >>> Simo Sorce wrote: >>>> On Wed, 2011-11-02 at 16:44 -0400, Ade Lee wrote: >>>>> On Wed, 2011-11-02 at 16:03 -0400, Adam Young wrote: >>>> [...] >>>>> So, a user becomes an agent on the ca by having a certificate in the >>>>> user record and being a member of the relevant admin, agent or auditor >>>>> group. >>>>> >>>>> I see this as follows: >>>>> 1. ipa cms-user-add (add a user and add the auxilliary cmsuser object >>>>> class) >>>>> 2. ipa user-cert (contact the ca and get a certificate for this user, >>>>> add this cert to the user record in the ipa database) >>>>> 3. ipa group-add-member (add the user to the relevant group) >>>>> >>>>> At no point does PKI need to modify anything in the IPA database. >>>> Sounds reasonable. >>>> Can you post a link to the schema that would be added to IPA objects ? >>>> >>>> Simo. >>>> >> I think this is it: >> >> http://svn.fedorahosted.org/svn/pki/trunk/pki/base/ca/shared/conf/schema.ldif >> >> Look for cmsuser. > Unfortunately it looks like the cmsuser objectclass is of type > structural, which means it cannot be added to existing records. > >> The cert seems to comes from >> >> 05rfc4523.ldif >> >> and is added in >> >> 06inetorgperson.ldif >> >> Which is already in our user record. >> >> CMS only seems to "require" usertype, which is a string, and "allows" >> userstate which is an integer. > I wonder if we can convince PKI to use a different schema to reprsent > this information. We can use Roles or Groups to tell what type of user a > user is, not sure about the state as that schema file has exactly the > same comment for both usertype and userstate, seems a bug. I think so. I do not know if CMSuser is really needed, as it looks like everything in there is accounted for some other way. My guess is the "type" field is used to enforce that someone in one group, say agents, cannot also be in a different group, say "auditors" but they do use groups for most roles in the system. I think that there are going to be severl layers for the permissions in the IPA approach: For CAAdmins, we create a group CAdmins, and add a Role to to CAAdminRole which contains the appropriate Permissions. Same for Auditors and agents. No one should have the ACI to change these roles. We probably want to put in an RFE for IPA Roles that state two roles are mutually exclusive. userstate is, I think, to disable an account, which is handled using nsaccount lock in IPA. I think we should agree on a single mechanism, and it should be the one in the most standard schema. > >>> IIRC the user we create in CS now has the description attribute set up >>> in a very specific way. Is that still required? > What is description used for ? > > Simo. > > From rcritten at redhat.com Thu Nov 3 13:22:23 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 03 Nov 2011 09:22:23 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <1320266688.17206.179.camel@localhost.localdomain> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> <4EB1770F.5000109@redhat.com> <4EB1A21E.9020208@redhat.com> <1320266688.17206.179.camel@localhost.localdomain> Message-ID: <4EB2958F.1070903@redhat.com> Ade Lee wrote: > On Wed, 2011-11-02 at 16:03 -0400, Adam Young wrote: >> To clarify: there are two types of Data stored in the PKI CA DS >> instances. One is Users and groups (IdM), and the other is >> certificates and requests. >> >> The CA currently administers its own users: creates, add deletes, add >> privs and so forth. If we extract the IdM objects from the CA >> control, we will need to build another mechanism to manage them. >> >> The Certificates will stay in their own suffix. I don't think anyone >> disagrees about this. >> >> I'm trying to think through the steps of using the IPA user database for >> PKI. If I undertand the end state, the general flow will be driven from >> ipa-server-install and will look like this: >> >> 1. Create a unified DS instance. We can continue to name it the way >> that IPA does: All caps the hostname. >> 2. Apply the LDAP schema LDIFs to it. At this point, we probably want >> to create the whole IPA schema, and the cmsUser as well > > For clarification, cmsuser is just an object that is defined in the PKI > schema, not the actual admin user created in the install itself. > > It may be advantageous to actually pre-create this user when applying > schema LDIFS and just have pkisilent add the user certs. The description attribute needs to store per-instance specific information such as the requestId. Unless you mean just adding userstate and usertype. >> 3. Call PKISilent (or equivalent) passing the info for the Unified >> directory server instance, to include the IPA tree for the users. >> 4. PKISilent will create the PKI admin user, to include its >> certificiate in the IPA tree. This user will be half baked from an IPA >> perspective, but will allow administration of the CA. > > Pre-creating this user may solve this problem. You can pre-create a > user with all the required IPA and PKI attributes and just have > pkisilent add the user cert needed. > > As we will be running as directory manager in this case, we will > permissions to do this. > >> 5. Create a PKIDirectory Manager account that has complete control over >> only the PKI suffix, and not the IPA suffix. Modify the PKI code to >> use only this account. This account should be able to authenticate >> against the Dir Srv using a client certificate, stored in the PKI >> specific NSS database. > > This of course involves setting up the relevant ACIs. We may want to > consider the reverse. That only certain IPA accounts should have access > to the PKI data. Which still involves ACIs, right? >> 6. Restart the CA. >> 7. Continue the IPA server install from the. Fix up the Admin user so >> that it works for IPA. > > Not needed if we pre-populate as mentioned above. > >> 8. Disable the Directory Manager's password. From here on out, access >> to the Dir Srv should be either certificate or Keytab only. >> >> >> After IPA is up and running, the management of the User Database is done >> by IPA. >> One thing that several people have asked for in IPA is the ability to >> manage external Roles: ones that don't map to ACIs. PKI has this >> requirement as well. There are a couple approaches we could take: >> >> 1. Attempt to use the current Role mechanism in IPA. This gets hidden >> to an outside user, but that should be OK. Checking if a user is a PKI >> Admin, Agent, or Auditor should be done only by a privileged account anyway. >> >> 2. Use a User Group: PKIAdmins, PKIAgents, and PKIAuditors. This >> is what I would suggest. >> >> 3. Create an external mechanism. >> >> >> Once IPA has assumed control of the IdM DB, we will still need to be >> able to get user certificates into the user records CMSUser field. I >> do not know CS well enough to say how that would work, but I assume it >> will be one of these two mechanisms: >> >> 1. Use the CA Administrative interface to modify an existing user to >> have the certificate >> or >> 2. Add a mechanism to IPA to request and apply the certificate to the >> IPA User. >> >> I'm guessing that the flow would be something like this: >> >> 1. Create a user (ipa user-add) >> 2. Assign them to the PKIAgents groups >> 3. Call the CA Admin interface to make the user an agent. >> >> If we do it this way, the PKI instance will need to be able to modify >> the IPA user record. Alternatively: >> >> 1. ipa user-add >> 2. ipa group-add-member >> 3. ipa user-cert > > So, a user becomes an agent on the ca by having a certificate in the > user record and being a member of the relevant admin, agent or auditor > group. > > I see this as follows: > 1. ipa cms-user-add (add a user and add the auxilliary cmsuser object > class) > 2. ipa user-cert (contact the ca and get a certificate for this user, > add this cert to the user record in the ipa database) > 3. ipa group-add-member (add the user to the relevant group) > > At no point does PKI need to modify anything in the IPA database. And it never should IMHO. We need to maintain the idea that the CA is some black box that we can poke at from time to time to get data but I'd prefer to keep them discrete. >> >> As an added bonus, we get user certs. >> >> >> One discussion we've had on the side is about scalability. As the >> Databases increase in size, it might become impractical to fully >> synchroize the user database over to a machine that is dedicated to >> Certificate operations. For example, we might have a public facing >> machine in the DMZ that is servering CRLs and OCSP to the world. This >> machine would only need the subset of users that can act as PKI >> admins. In this case, we might build a custom script for replicating a >> subset of the IPA data one direction only: from one of the fully >> replicated instance to the DMZ instance. This is not something we >> foresee doing in a near term release, but should be discussed and >> fleshed out. >> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From simo at redhat.com Thu Nov 3 13:53:40 2011 From: simo at redhat.com (Simo Sorce) Date: Thu, 03 Nov 2011 09:53:40 -0400 Subject: [Freeipa-devel] [PATCH] fix memleak in ipa-kdb Message-ID: <1320328420.7734.608.camel@willson.li.ssimo.org> Pushed the attached patch under the oneliner rule. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-ipa-kdb-Fix-memory-leak.patch Type: text/x-patch Size: 754 bytes Desc: not available URL: From pvoborni at redhat.com Thu Nov 3 14:26:53 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 03 Nov 2011 15:26:53 +0100 Subject: [Freeipa-devel] [PATCH] 031 Field for DNS SOA class changed to combobox with options Message-ID: <4EB2A4AD.1030603@redhat.com> https://fedorahosted.org/freeipa/ticket/602 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0031-Field-for-DNS-SOA-class-changed-to-combobox-with-opt.patch Type: text/x-patch Size: 2816 bytes Desc: not available URL: From alee at redhat.com Thu Nov 3 15:00:05 2011 From: alee at redhat.com (Ade Lee) Date: Thu, 03 Nov 2011 11:00:05 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <4EB29516.5040006@redhat.com> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> <4EB1770F.5000109@redhat.com> <4EB1A21E.9020208@redhat.com> <1320266688.17206.179.camel@localhost.localdomain> <1320271565.7734.588.camel@willson.li.ssimo.org> <4EB1C1D7.4020901@redhat.com> <4EB1DF88.6020304@redhat.com> <1320296195.7734.599.camel@willson.li.ssimo.org> <4EB29516.5040006@redhat.com> Message-ID: <1320332406.17206.240.camel@localhost.localdomain> On Thu, 2011-11-03 at 09:20 -0400, Adam Young wrote: > On 11/03/2011 12:56 AM, Simo Sorce wrote: > > On Wed, 2011-11-02 at 20:25 -0400, Adam Young wrote: > >> On 11/02/2011 06:19 PM, Rob Crittenden wrote: > >>> Simo Sorce wrote: > >>>> On Wed, 2011-11-02 at 16:44 -0400, Ade Lee wrote: > >>>>> On Wed, 2011-11-02 at 16:03 -0400, Adam Young wrote: > >>>> [...] > >>>>> So, a user becomes an agent on the ca by having a certificate in the > >>>>> user record and being a member of the relevant admin, agent or auditor > >>>>> group. > >>>>> > >>>>> I see this as follows: > >>>>> 1. ipa cms-user-add (add a user and add the auxilliary cmsuser object > >>>>> class) > >>>>> 2. ipa user-cert (contact the ca and get a certificate for this user, > >>>>> add this cert to the user record in the ipa database) > >>>>> 3. ipa group-add-member (add the user to the relevant group) > >>>>> > >>>>> At no point does PKI need to modify anything in the IPA database. > >>>> Sounds reasonable. > >>>> Can you post a link to the schema that would be added to IPA objects ? > >>>> > >>>> Simo. > >>>> > >> I think this is it: > >> > >> http://svn.fedorahosted.org/svn/pki/trunk/pki/base/ca/shared/conf/schema.ldif > >> > >> Look for cmsuser. > > Unfortunately it looks like the cmsuser objectclass is of type > > structural, which means it cannot be added to existing records. > > > >> The cert seems to comes from > >> > >> 05rfc4523.ldif > >> > >> and is added in > >> > >> 06inetorgperson.ldif > >> > >> Which is already in our user record. > >> > >> CMS only seems to "require" usertype, which is a string, and "allows" > >> userstate which is an integer. > > I wonder if we can convince PKI to use a different schema to reprsent > > this information. We can use Roles or Groups to tell what type of user a > > user is, not sure about the state as that schema file has exactly the > > same comment for both usertype and userstate, seems a bug. > > I think so. I do not know if CMSuser is really needed, as it looks like > everything in there is accounted for some other way. > > My guess is the "type" field is used to enforce that someone in one > group, say agents, cannot also be in a different group, say "auditors" > but they do use groups for most roles in the system. > > I think that there are going to be severl layers for the permissions in > the IPA approach: For CAAdmins, we create a group CAdmins, and add a > Role to to CAAdminRole which contains the appropriate Permissions. Same > for Auditors and agents. No one should have the ACI to change these roles. > > We probably want to put in an RFE for IPA Roles that state two roles > are mutually exclusive. > > > userstate is, I think, to disable an account, which is handled using > nsaccount lock in IPA. I think we should agree on a single mechanism, > and it should be the one in the most standard schema. > > With just an initial look at the dogtag code, it appears that userState and userType are no longer used in any meaningful way. We'll have to do a more exhaustive search to be sure, but initial indications are that we may no longer need this object class. Adam does bring up a good point, which is that - as a common criteria requirement, users were required to have no more than one of the following roles: agent, admin, auditor. How would this be enforced in the IPA database? > > > > > >>> IIRC the user we create in CS now has the description attribute set up > >>> in a very specific way. Is that still required? > > What is description used for ? > > > > Simo. > > > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From ayoung at redhat.com Thu Nov 3 15:01:38 2011 From: ayoung at redhat.com (Adam Young) Date: Thu, 03 Nov 2011 11:01:38 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <1320332406.17206.240.camel@localhost.localdomain> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> <4EB1770F.5000109@redhat.com> <4EB1A21E.9020208@redhat.com> <1320266688.17206.179.camel@localhost.localdomain> <1320271565.7734.588.camel@willson.li.ssimo.org> <4EB1C1D7.4020901@redhat.com> <4EB1DF88.6020304@redhat.com> <1320296195.7734.599.camel@willson.li.ssimo.org> <4EB29516.5040006@redhat.com> <1320332406.17206.240.camel@localhost.localdomain> Message-ID: <4EB2ACD2.6080005@redhat.com> On 11/03/2011 11:00 AM, Ade Lee wrote: > On Thu, 2011-11-03 at 09:20 -0400, Adam Young wrote: >> On 11/03/2011 12:56 AM, Simo Sorce wrote: >>> On Wed, 2011-11-02 at 20:25 -0400, Adam Young wrote: >>>> On 11/02/2011 06:19 PM, Rob Crittenden wrote: >>>>> Simo Sorce wrote: >>>>>> On Wed, 2011-11-02 at 16:44 -0400, Ade Lee wrote: >>>>>>> On Wed, 2011-11-02 at 16:03 -0400, Adam Young wrote: >>>>>> [...] >>>>>>> So, a user becomes an agent on the ca by having a certificate in the >>>>>>> user record and being a member of the relevant admin, agent or auditor >>>>>>> group. >>>>>>> >>>>>>> I see this as follows: >>>>>>> 1. ipa cms-user-add (add a user and add the auxilliary cmsuser object >>>>>>> class) >>>>>>> 2. ipa user-cert (contact the ca and get a certificate for this user, >>>>>>> add this cert to the user record in the ipa database) >>>>>>> 3. ipa group-add-member (add the user to the relevant group) >>>>>>> >>>>>>> At no point does PKI need to modify anything in the IPA database. >>>>>> Sounds reasonable. >>>>>> Can you post a link to the schema that would be added to IPA objects ? >>>>>> >>>>>> Simo. >>>>>> >>>> I think this is it: >>>> >>>> http://svn.fedorahosted.org/svn/pki/trunk/pki/base/ca/shared/conf/schema.ldif >>>> >>>> Look for cmsuser. >>> Unfortunately it looks like the cmsuser objectclass is of type >>> structural, which means it cannot be added to existing records. >>> >>>> The cert seems to comes from >>>> >>>> 05rfc4523.ldif >>>> >>>> and is added in >>>> >>>> 06inetorgperson.ldif >>>> >>>> Which is already in our user record. >>>> >>>> CMS only seems to "require" usertype, which is a string, and "allows" >>>> userstate which is an integer. >>> I wonder if we can convince PKI to use a different schema to reprsent >>> this information. We can use Roles or Groups to tell what type of user a >>> user is, not sure about the state as that schema file has exactly the >>> same comment for both usertype and userstate, seems a bug. >> I think so. I do not know if CMSuser is really needed, as it looks like >> everything in there is accounted for some other way. >> >> My guess is the "type" field is used to enforce that someone in one >> group, say agents, cannot also be in a different group, say "auditors" >> but they do use groups for most roles in the system. >> >> I think that there are going to be severl layers for the permissions in >> the IPA approach: For CAAdmins, we create a group CAdmins, and add a >> Role to to CAAdminRole which contains the appropriate Permissions. Same >> for Auditors and agents. No one should have the ACI to change these roles. >> >> We probably want to put in an RFE for IPA Roles that state two roles >> are mutually exclusive. >> >> >> userstate is, I think, to disable an account, which is handled using >> nsaccount lock in IPA. I think we should agree on a single mechanism, >> and it should be the one in the most standard schema. >> >> > With just an initial look at the dogtag code, it appears that userState > and userType are no longer used in any meaningful way. We'll have to do > a more exhaustive search to be sure, but initial indications are that we > may no longer need this object class. > > Adam does bring up a good point, which is that - as a common criteria > requirement, users were required to have no more than one of the > following roles: agent, admin, auditor. How would this be enforced in > the IPA database? I think that we need to make it an RFE for IPA. > >> >>>>> IIRC the user we create in CS now has the description attribute set up >>>>> in a very specific way. Is that still required? >>> What is description used for ? >>> >>> Simo. >>> >>> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > From simo at redhat.com Thu Nov 3 15:05:58 2011 From: simo at redhat.com (Simo Sorce) Date: Thu, 03 Nov 2011 11:05:58 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <1320332406.17206.240.camel@localhost.localdomain> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> <4EB1770F.5000109@redhat.com> <4EB1A21E.9020208@redhat.com> <1320266688.17206.179.camel@localhost.localdomain> <1320271565.7734.588.camel@willson.li.ssimo.org> <4EB1C1D7.4020901@redhat.com> <4EB1DF88.6020304@redhat.com> <1320296195.7734.599.camel@willson.li.ssimo.org> <4EB29516.5040006@redhat.com> <1320332406.17206.240.camel@localhost.localdomain> Message-ID: <1320332759.7734.645.camel@willson.li.ssimo.org> On Thu, 2011-11-03 at 11:00 -0400, Ade Lee wrote: > On Thu, 2011-11-03 at 09:20 -0400, Adam Young wrote: > > On 11/03/2011 12:56 AM, Simo Sorce wrote: > > > On Wed, 2011-11-02 at 20:25 -0400, Adam Young wrote: > > >> On 11/02/2011 06:19 PM, Rob Crittenden wrote: > > >>> Simo Sorce wrote: > > >>>> On Wed, 2011-11-02 at 16:44 -0400, Ade Lee wrote: > > >>>>> On Wed, 2011-11-02 at 16:03 -0400, Adam Young wrote: > > >>>> [...] > > >>>>> So, a user becomes an agent on the ca by having a certificate in the > > >>>>> user record and being a member of the relevant admin, agent or auditor > > >>>>> group. > > >>>>> > > >>>>> I see this as follows: > > >>>>> 1. ipa cms-user-add (add a user and add the auxilliary cmsuser object > > >>>>> class) > > >>>>> 2. ipa user-cert (contact the ca and get a certificate for this user, > > >>>>> add this cert to the user record in the ipa database) > > >>>>> 3. ipa group-add-member (add the user to the relevant group) > > >>>>> > > >>>>> At no point does PKI need to modify anything in the IPA database. > > >>>> Sounds reasonable. > > >>>> Can you post a link to the schema that would be added to IPA objects ? > > >>>> > > >>>> Simo. > > >>>> > > >> I think this is it: > > >> > > >> http://svn.fedorahosted.org/svn/pki/trunk/pki/base/ca/shared/conf/schema.ldif > > >> > > >> Look for cmsuser. > > > Unfortunately it looks like the cmsuser objectclass is of type > > > structural, which means it cannot be added to existing records. > > > > > >> The cert seems to comes from > > >> > > >> 05rfc4523.ldif > > >> > > >> and is added in > > >> > > >> 06inetorgperson.ldif > > >> > > >> Which is already in our user record. > > >> > > >> CMS only seems to "require" usertype, which is a string, and "allows" > > >> userstate which is an integer. > > > I wonder if we can convince PKI to use a different schema to reprsent > > > this information. We can use Roles or Groups to tell what type of user a > > > user is, not sure about the state as that schema file has exactly the > > > same comment for both usertype and userstate, seems a bug. > > > > I think so. I do not know if CMSuser is really needed, as it looks like > > everything in there is accounted for some other way. > > > > My guess is the "type" field is used to enforce that someone in one > > group, say agents, cannot also be in a different group, say "auditors" > > but they do use groups for most roles in the system. > > > > I think that there are going to be severl layers for the permissions in > > the IPA approach: For CAAdmins, we create a group CAdmins, and add a > > Role to to CAAdminRole which contains the appropriate Permissions. Same > > for Auditors and agents. No one should have the ACI to change these roles. > > > > We probably want to put in an RFE for IPA Roles that state two roles > > are mutually exclusive. > > > > > > userstate is, I think, to disable an account, which is handled using > > nsaccount lock in IPA. I think we should agree on a single mechanism, > > and it should be the one in the most standard schema. > > > > > > With just an initial look at the dogtag code, it appears that userState > and userType are no longer used in any meaningful way. We'll have to do > a more exhaustive search to be sure, but initial indications are that we > may no longer need this object class. > > Adam does bring up a good point, which is that - as a common criteria > requirement, users were required to have no more than one of the > following roles: agent, admin, auditor. How would this be enforced in > the IPA database? At the moment it can't be enforced, but I guess we could build a special plugin that will work similarly to the uniqueness plugin but will work on one or more pools of mutually exclusive roles to be enforced on all entries. I guess this is something that could be useful outside of CA as well if roles turns out to be used more. Simo. -- Simo Sorce * Red Hat, Inc * New York From alee at redhat.com Thu Nov 3 15:18:12 2011 From: alee at redhat.com (Ade Lee) Date: Thu, 03 Nov 2011 11:18:12 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <4EB2958F.1070903@redhat.com> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> <4EB1770F.5000109@redhat.com> <4EB1A21E.9020208@redhat.com> <1320266688.17206.179.camel@localhost.localdomain> <4EB2958F.1070903@redhat.com> Message-ID: <1320333492.17206.245.camel@localhost.localdomain> On Thu, 2011-11-03 at 09:22 -0400, Rob Crittenden wrote: > Ade Lee wrote: > > On Wed, 2011-11-02 at 16:03 -0400, Adam Young wrote: > >> To clarify: there are two types of Data stored in the PKI CA DS > >> instances. One is Users and groups (IdM), and the other is > >> certificates and requests. > >> > >> The CA currently administers its own users: creates, add deletes, add > >> privs and so forth. If we extract the IdM objects from the CA > >> control, we will need to build another mechanism to manage them. > >> > >> The Certificates will stay in their own suffix. I don't think anyone > >> disagrees about this. > >> > >> I'm trying to think through the steps of using the IPA user database for > >> PKI. If I undertand the end state, the general flow will be driven from > >> ipa-server-install and will look like this: > >> > >> 1. Create a unified DS instance. We can continue to name it the way > >> that IPA does: All caps the hostname. > >> 2. Apply the LDAP schema LDIFs to it. At this point, we probably want > >> to create the whole IPA schema, and the cmsUser as well > > > > For clarification, cmsuser is just an object that is defined in the PKI > > schema, not the actual admin user created in the install itself. > > > > It may be advantageous to actually pre-create this user when applying > > schema LDIFS and just have pkisilent add the user certs. > > The description attribute needs to store per-instance specific > information such as the requestId. Unless you mean just adding userstate > and usertype. > In dogtag, I believe we have used the description attribute to store whatever the user provided to describe the user/group. This is more relevant to the console. As IPA will be managing users and groups, then it can also manage this attribute. > >> 3. Call PKISilent (or equivalent) passing the info for the Unified > >> directory server instance, to include the IPA tree for the users. > >> 4. PKISilent will create the PKI admin user, to include its > >> certificiate in the IPA tree. This user will be half baked from an IPA > >> perspective, but will allow administration of the CA. > > > > Pre-creating this user may solve this problem. You can pre-create a > > user with all the required IPA and PKI attributes and just have > > pkisilent add the user cert needed. > > > > As we will be running as directory manager in this case, we will > > permissions to do this. > > > >> 5. Create a PKIDirectory Manager account that has complete control over > >> only the PKI suffix, and not the IPA suffix. Modify the PKI code to > >> use only this account. This account should be able to authenticate > >> against the Dir Srv using a client certificate, stored in the PKI > >> specific NSS database. > > > > This of course involves setting up the relevant ACIs. We may want to > > consider the reverse. That only certain IPA accounts should have access > > to the PKI data. > > Which still involves ACIs, right? Right > > >> 6. Restart the CA. > >> 7. Continue the IPA server install from the. Fix up the Admin user so > >> that it works for IPA. > > > > Not needed if we pre-populate as mentioned above. > > > >> 8. Disable the Directory Manager's password. From here on out, access > >> to the Dir Srv should be either certificate or Keytab only. > >> > >> > >> After IPA is up and running, the management of the User Database is done > >> by IPA. > >> One thing that several people have asked for in IPA is the ability to > >> manage external Roles: ones that don't map to ACIs. PKI has this > >> requirement as well. There are a couple approaches we could take: > >> > >> 1. Attempt to use the current Role mechanism in IPA. This gets hidden > >> to an outside user, but that should be OK. Checking if a user is a PKI > >> Admin, Agent, or Auditor should be done only by a privileged account anyway. > >> > >> 2. Use a User Group: PKIAdmins, PKIAgents, and PKIAuditors. This > >> is what I would suggest. > >> > >> 3. Create an external mechanism. > >> > >> > >> Once IPA has assumed control of the IdM DB, we will still need to be > >> able to get user certificates into the user records CMSUser field. I > >> do not know CS well enough to say how that would work, but I assume it > >> will be one of these two mechanisms: > >> > >> 1. Use the CA Administrative interface to modify an existing user to > >> have the certificate > >> or > >> 2. Add a mechanism to IPA to request and apply the certificate to the > >> IPA User. > >> > >> I'm guessing that the flow would be something like this: > >> > >> 1. Create a user (ipa user-add) > >> 2. Assign them to the PKIAgents groups > >> 3. Call the CA Admin interface to make the user an agent. > >> > >> If we do it this way, the PKI instance will need to be able to modify > >> the IPA user record. Alternatively: > >> > >> 1. ipa user-add > >> 2. ipa group-add-member > >> 3. ipa user-cert > > > > So, a user becomes an agent on the ca by having a certificate in the > > user record and being a member of the relevant admin, agent or auditor > > group. > > > > I see this as follows: > > 1. ipa cms-user-add (add a user and add the auxilliary cmsuser object > > class) > > 2. ipa user-cert (contact the ca and get a certificate for this user, > > add this cert to the user record in the ipa database) > > 3. ipa group-add-member (add the user to the relevant group) > > > > At no point does PKI need to modify anything in the IPA database. > > And it never should IMHO. We need to maintain the idea that the CA is > some black box that we can poke at from time to time to get data but I'd > prefer to keep them discrete. > Yes - me too. > >> > >> As an added bonus, we get user certs. > >> > >> > >> One discussion we've had on the side is about scalability. As the > >> Databases increase in size, it might become impractical to fully > >> synchroize the user database over to a machine that is dedicated to > >> Certificate operations. For example, we might have a public facing > >> machine in the DMZ that is servering CRLs and OCSP to the world. This > >> machine would only need the subset of users that can act as PKI > >> admins. In this case, we might build a custom script for replicating a > >> subset of the IPA data one direction only: from one of the fully > >> replicated instance to the DMZ instance. This is not something we > >> foresee doing in a near term release, but should be discussed and > >> fleshed out. > >> > >> _______________________________________________ > >> Freeipa-devel mailing list > >> Freeipa-devel at redhat.com > >> https://www.redhat.com/mailman/listinfo/freeipa-devel > > > > > > _______________________________________________ > > Freeipa-devel mailing list > > Freeipa-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/freeipa-devel > From awnuk at redhat.com Thu Nov 3 15:30:06 2011 From: awnuk at redhat.com (Andrew Wnuk) Date: Thu, 03 Nov 2011 08:30:06 -0700 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <4EB1C1D7.4020901@redhat.com> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> <4EB1770F.5000109@redhat.com> <4EB1A21E.9020208@redhat.com> <1320266688.17206.179.camel@localhost.localdomain> <1320271565.7734.588.camel@willson.li.ssimo.org> <4EB1C1D7.4020901@redhat.com> Message-ID: <4EB2B37E.2050002@redhat.com> On 11/02/2011 03:19 PM, Rob Crittenden wrote: > Simo Sorce wrote: >> On Wed, 2011-11-02 at 16:44 -0400, Ade Lee wrote: >>> On Wed, 2011-11-02 at 16:03 -0400, Adam Young wrote: >> [...] >>> So, a user becomes an agent on the ca by having a certificate in the >>> user record and being a member of the relevant admin, agent or auditor >>> group. >>> >>> I see this as follows: >>> 1. ipa cms-user-add (add a user and add the auxilliary cmsuser object >>> class) >>> 2. ipa user-cert (contact the ca and get a certificate for this user, >>> add this cert to the user record in the ipa database) >>> 3. ipa group-add-member (add the user to the relevant group) >>> >>> At no point does PKI need to modify anything in the IPA database. >> >> Sounds reasonable. >> Can you post a link to the schema that would be added to IPA objects ? >> >> Simo. >> > > IIRC the user we create in CS now has the description attribute set up > in a very specific way. Is that still required? > > rob Steps 1 to 3 should have an option to be performed only by CS admins with certificate client authentication, otherwise we will break rules of secure CS configuration including separation of roles. Andrew From ayoung at redhat.com Thu Nov 3 15:46:23 2011 From: ayoung at redhat.com (Adam Young) Date: Thu, 03 Nov 2011 11:46:23 -0400 Subject: [Freeipa-devel] Unifying the PKI and IPA Directory Server instances In-Reply-To: <4EB2B37E.2050002@redhat.com> References: <4d43ea86-7b74-4a4a-adec-47cdea058e78@zmail01.collab.prod.int.phx2.redhat.com> <1320166189.7734.280.camel@willson.li.ssimo.org> <1320167298.17206.126.camel@localhost.localdomain> <4EB1770F.5000109@redhat.com> <4EB1A21E.9020208@redhat.com> <1320266688.17206.179.camel@localhost.localdomain> <1320271565.7734.588.camel@willson.li.ssimo.org> <4EB1C1D7.4020901@redhat.com> <4EB2B37E.2050002@redhat.com> Message-ID: <4EB2B74F.1020509@redhat.com> On 11/03/2011 11:30 AM, Andrew Wnuk wrote: > On 11/02/2011 03:19 PM, Rob Crittenden wrote: >> Simo Sorce wrote: >>> On Wed, 2011-11-02 at 16:44 -0400, Ade Lee wrote: >>>> On Wed, 2011-11-02 at 16:03 -0400, Adam Young wrote: >>> [...] >>>> So, a user becomes an agent on the ca by having a certificate in the >>>> user record and being a member of the relevant admin, agent or auditor >>>> group. >>>> >>>> I see this as follows: >>>> 1. ipa cms-user-add (add a user and add the auxilliary cmsuser object >>>> class) >>>> 2. ipa user-cert (contact the ca and get a certificate for this user, >>>> add this cert to the user record in the ipa database) >>>> 3. ipa group-add-member (add the user to the relevant group) >>>> >>>> At no point does PKI need to modify anything in the IPA database. >>> >>> Sounds reasonable. >>> Can you post a link to the schema that would be added to IPA objects ? >>> >>> Simo. >>> >> >> IIRC the user we create in CS now has the description attribute set >> up in a very specific way. Is that still required? >> >> rob > > Steps 1 to 3 should have an option to be performed only by CS admins > with certificate client authentication, otherwise we will break rules > of secure CS configuration including separation of roles. We had a long talk about that on the IPA call this morning. In order to add someone to the PKIAdmin user-group, you need to have the appropriate ACIs. We'd like to lock thos in, so that someone messing around with IPA can't mess them up. I'm not certain that the specific authentication mechanism is the issue so much as you need to have a guarantee of authentication no less than what Client Cert auth gives you. Kerberos authentication should actually be as good: it will be enforced not just by the application, but all the way down to the DS instance via ACIs. > > Andrew > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From simo at redhat.com Thu Nov 3 18:06:36 2011 From: simo at redhat.com (Simo Sorce) Date: Thu, 03 Nov 2011 14:06:36 -0400 Subject: [Freeipa-devel] [PATCHES] #2036 Fix coverity bugs Message-ID: <1320343596.7734.675.camel@willson.li.ssimo.org> Just some unchecked returns, we do not care much, but will keep Coverity happy. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-CID-10742-Unchecked-return-value.patch Type: text/x-patch Size: 1345 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Fix-CID-10743-Unchecked-return-value.patch Type: text/x-patch Size: 1454 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0003-Fix-CID-10745-Unchecked-return-value.patch Type: text/x-patch Size: 1342 bytes Desc: not available URL: From simo at redhat.com Thu Nov 3 18:09:42 2011 From: simo at redhat.com (Simo Sorce) Date: Thu, 03 Nov 2011 14:09:42 -0400 Subject: [Freeipa-devel] [PATCHES] #2037 Coverity issues Message-ID: <1320343782.7734.677.camel@willson.li.ssimo.org> These are actual issues. Most are resource leaks. And one is a bad sizeof() computation that will cause us later to overwrite out of bound memory, so potentially a segfault. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-CID-11019-Resource-leak.patch Type: text/x-patch Size: 1126 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Fix-CID-11020-Resource-leak.patch Type: text/x-patch Size: 974 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0003-Fix-CID-11021-Resource-leak.patch Type: text/x-patch Size: 1013 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0004-Fix-CID-11022-Resource-leak.patch Type: text/x-patch Size: 1353 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0005-Fix-CID-11023-Resource-leak.patch Type: text/x-patch Size: 835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0006-Fix-CID-11024-Resource-leak.patch Type: text/x-patch Size: 846 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0007-Fix-CID-11025-Resource-leak.patch Type: text/x-patch Size: 1211 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0008-Fix-CID-11026-Resource-leak.patch Type: text/x-patch Size: 1144 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0009-Fix-CID-11027-Wrong-sizeof-argument.patch Type: text/x-patch Size: 919 bytes Desc: not available URL: From rcritten at redhat.com Thu Nov 3 19:02:23 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 03 Nov 2011 15:02:23 -0400 Subject: [Freeipa-devel] [PATCH] 158 Fix ipa-server-install answer cache In-Reply-To: <1320315666.23357.4.camel@balmora.brq.redhat.com> References: <1320315666.23357.4.camel@balmora.brq.redhat.com> Message-ID: <4EB2E53F.1090908@redhat.com> Martin Kosek wrote: > Current Answer Cache storing mechanism is not ideal for storing > non-trivial Python types like arrays, custom classes, etc. > RawConfigParser just translates values to string, which > are not correctly decoded when the Answer Cache is parsed and > restored in the installer. > > This patch replaces RawConfigParser with Python's standard pickle > module, which is a recommended way for serialization in Python. > > https://fedorahosted.org/freeipa/ticket/2054 ack From edewata at redhat.com Thu Nov 3 21:22:15 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 03 Nov 2011 16:22:15 -0500 Subject: [Freeipa-devel] [PATCH] 029 Page is cleared before it is visible In-Reply-To: <4EB16940.4040708@redhat.com> References: <4EA92AF7.8080609@redhat.com> <4EAF2365.8000005@redhat.com> <4EAFE81C.4030004@redhat.com> <4EB07331.9070606@redhat.com> <4EB146AE.9010609@redhat.com> <4EB164C1.4090308@redhat.com> <4EB16940.4040708@redhat.com> Message-ID: <4EB30607.1080807@redhat.com> On 11/2/2011 11:01 AM, Petr Vobornik wrote: >> Regardless, ACK and pushed to master. Found another problem, the krbtpolicy & config need to be forced to update. See the attached patch. -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-edewata-0305-Fixed-blank-krbtpolicy-and-config-pages.patch Type: text/x-patch Size: 5093 bytes Desc: not available URL: From mkosek at redhat.com Thu Nov 3 21:51:05 2011 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 03 Nov 2011 22:51:05 +0100 Subject: [Freeipa-devel] [PATCH] 158 Fix ipa-server-install answer cache In-Reply-To: <4EB2E53F.1090908@redhat.com> References: <1320315666.23357.4.camel@balmora.brq.redhat.com> <4EB2E53F.1090908@redhat.com> Message-ID: <1320357071.7196.0.camel@priserak> On Thu, 2011-11-03 at 15:02 -0400, Rob Crittenden wrote: > Martin Kosek wrote: > > Current Answer Cache storing mechanism is not ideal for storing > > non-trivial Python types like arrays, custom classes, etc. > > RawConfigParser just translates values to string, which > > are not correctly decoded when the Answer Cache is parsed and > > restored in the installer. > > > > This patch replaces RawConfigParser with Python's standard pickle > > module, which is a recommended way for serialization in Python. > > > > https://fedorahosted.org/freeipa/ticket/2054 > > ack > Pushed to master, ipa-2-1. Martin From simo at redhat.com Thu Nov 3 22:26:15 2011 From: simo at redhat.com (Simo Sorce) Date: Thu, 03 Nov 2011 18:26:15 -0400 Subject: [Freeipa-devel] [PATCH] #2038 modify salt creation Message-ID: <1320359175.7734.699.camel@willson.li.ssimo.org> As stated in the bug in order to attain better interoperability with Windows clients we need to change the way we generate the random salt. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Modify-random-salt-creation-for-interoperability.patch Type: text/x-patch Size: 2037 bytes Desc: not available URL: From edewata at redhat.com Thu Nov 3 22:51:42 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 03 Nov 2011 17:51:42 -0500 Subject: [Freeipa-devel] [PATCH] 030 Extending facet's mechanism of gathering changes In-Reply-To: <4EB28A7A.3080905@redhat.com> References: <4EB28A7A.3080905@redhat.com> Message-ID: <4EB31AFE.5010401@redhat.com> On 11/3/2011 7:35 AM, Petr Vobornik wrote: > https://fedorahosted.org/freeipa/ticket/2041 > > I'm not sure if update_info and other new classes should be in details.js. It's probably ok now, but in the future we might want to move them somewhere else. How about creating command.js and move all command-related code there? Other issues: 1. In details.js:650 we don't use param_info anymore, it should be metadata. 2. The add_field_option(command, field_name, param_info, values, join) probably can be simplified into add_field_option(field, values). The IPA.command_builder can store the command internally: var command_builder = IPA.command_builder({ command: command }); ... command_builder.add_field_option(field_info.field, values); The add_field_option() can get the name, metadata, and join from the field object. 3. The add_record_to_command() takes a list of sections, but it might not be necessary because (so far) it's only used to access the details facet's own list of sections. Do you expect this to be used differently? 4. The create_fields_update_command() is essentially the same as create_standard_update_command(). When the command_mode is 'save' is it possible to generate an update_info from records so we can just call create_fields_update_command()? This patch I think can be ACKed after fixing #1, the rest can be dealt with later. Some of the new codes are not used anywhere yet so it's a bit difficult to review and things might change again later. I'm curious to see how it's going to be used in HBAC/sudo rule and DNS zone that involves multiple commands: how the commands are going to be defined, how to associate certain fields with certain commands, how to assign priorities, etc. Do you have any preview patch for ticket #1515? -- Endi S. Dewata From edewata at redhat.com Fri Nov 4 03:34:01 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 03 Nov 2011 22:34:01 -0500 Subject: [Freeipa-devel] [PATCH] 031 Field for DNS SOA class changed to combobox with options In-Reply-To: <4EB2A4AD.1030603@redhat.com> References: <4EB2A4AD.1030603@redhat.com> Message-ID: <4EB35D29.5090800@redhat.com> On 11/3/2011 9:26 AM, Petr Vobornik wrote: > https://fedorahosted.org/freeipa/ticket/602 ACK and pushed to master. -- Endi S. Dewata From edewata at redhat.com Fri Nov 4 03:35:38 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 03 Nov 2011 22:35:38 -0500 Subject: [Freeipa-devel] [PATCH] 306 Moved facet code into facet.js. Message-ID: <4EB35D8A.80405@redhat.com> Facet-related code has been moved from entity.js into a new facet.js because the file is getting too big. -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-edewata-0306-Moved-facet-code-into-facet.js.patch Type: text/x-patch Size: 46872 bytes Desc: not available URL: From edewata at redhat.com Fri Nov 4 03:37:38 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 03 Nov 2011 22:37:38 -0500 Subject: [Freeipa-devel] [PATCH] 307 Added extensible UI framework. Message-ID: <4EB35E02.6020700@redhat.com> The entity definitions have been converted into classes. The entity init() method will use the builder to construct the facets and dialogs. The UI can be customized by creating a subclass of the original entity in extension.js and then overriding the init() method. Ticket #2043 -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-edewata-0307-Added-extensible-UI-framework.patch Type: text/x-patch Size: 29562 bytes Desc: not available URL: From mkosek at redhat.com Fri Nov 4 07:55:50 2011 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 04 Nov 2011 08:55:50 +0100 Subject: [Freeipa-devel] [PATCH] 159 Fix ipa-replica-conncheck port labels Message-ID: <1320393352.13980.2.camel@balmora.brq.redhat.com> Pushed under the one-liner (two-liner in this case) rule to master, ipa-2-1. Quick self-verification after the change: # ipa-replica-conncheck -m vm-050.idm.lab.bos.redhat.com Check connection from replica to remote master 'vm-050.idm.lab.bos.redhat.com': Directory Service: Unsecure port (389): OK Directory Service: Secure port (636): OK Kerberos KDC: TCP (88): OK Kerberos KDC: UDP (88): OK Kerberos Kpasswd: TCP (464): OK Kerberos Kpasswd: UDP (464): OK HTTP Server: Unsecure port (80): OK HTTP Server: Secure port (443): OK -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-159-fix-ipa-replica-conncheck-port-labels.patch Type: text/x-patch Size: 1170 bytes Desc: not available URL: From abokovoy at redhat.com Fri Nov 4 08:04:04 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 4 Nov 2011 10:04:04 +0200 Subject: [Freeipa-devel] [PATCH] #2038 modify salt creation In-Reply-To: <1320359175.7734.699.camel@willson.li.ssimo.org> References: <1320359175.7734.699.camel@willson.li.ssimo.org> Message-ID: <20111104080403.GD26667@redhat.com> On Thu, 03 Nov 2011, Simo Sorce wrote: > As stated in the bug in order to attain better interoperability with > Windows clients we need to change the way we generate the random salt. ACK. -- / Alexander Bokovoy From mkosek at redhat.com Fri Nov 4 10:14:04 2011 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 04 Nov 2011 11:14:04 +0100 Subject: [Freeipa-devel] [PATCH] #2038 modify salt creation In-Reply-To: <20111104080403.GD26667@redhat.com> References: <1320359175.7734.699.camel@willson.li.ssimo.org> <20111104080403.GD26667@redhat.com> Message-ID: <1320401646.13980.3.camel@balmora.brq.redhat.com> On Fri, 2011-11-04 at 10:04 +0200, Alexander Bokovoy wrote: > On Thu, 03 Nov 2011, Simo Sorce wrote: > > As stated in the bug in order to attain better interoperability with > > Windows clients we need to change the way we generate the random salt. > ACK. > Pushed to master. Martin From simo at redhat.com Fri Nov 4 11:41:07 2011 From: simo at redhat.com (Simo Sorce) Date: Fri, 04 Nov 2011 07:41:07 -0400 Subject: [Freeipa-devel] [PATCH] #2038 modify salt creation In-Reply-To: <1320401646.13980.3.camel@balmora.brq.redhat.com> References: <1320359175.7734.699.camel@willson.li.ssimo.org> <20111104080403.GD26667@redhat.com> <1320401646.13980.3.camel@balmora.brq.redhat.com> Message-ID: <1320406867.7734.706.camel@willson.li.ssimo.org> On Fri, 2011-11-04 at 11:14 +0100, Martin Kosek wrote: > On Fri, 2011-11-04 at 10:04 +0200, Alexander Bokovoy wrote: > > On Thu, 03 Nov 2011, Simo Sorce wrote: > > > As stated in the bug in order to attain better interoperability with > > > Windows clients we need to change the way we generate the random salt. > > ACK. > > > > Pushed to master. Should we backport this to 2.x as well ? Simo. -- Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Fri Nov 4 11:55:08 2011 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 04 Nov 2011 12:55:08 +0100 Subject: [Freeipa-devel] [PATCH] #2038 modify salt creation In-Reply-To: <1320406867.7734.706.camel@willson.li.ssimo.org> References: <1320359175.7734.699.camel@willson.li.ssimo.org> <20111104080403.GD26667@redhat.com> <1320401646.13980.3.camel@balmora.brq.redhat.com> <1320406867.7734.706.camel@willson.li.ssimo.org> Message-ID: <1320407710.13980.39.camel@balmora.brq.redhat.com> On Fri, 2011-11-04 at 07:41 -0400, Simo Sorce wrote: > On Fri, 2011-11-04 at 11:14 +0100, Martin Kosek wrote: > > On Fri, 2011-11-04 at 10:04 +0200, Alexander Bokovoy wrote: > > > On Thu, 03 Nov 2011, Simo Sorce wrote: > > > > As stated in the bug in order to attain better interoperability with > > > > Windows clients we need to change the way we generate the random salt. > > > ACK. > > > > > > > Pushed to master. > > Should we backport this to 2.x as well ? > > Simo. > Hm, looks important enough to do it. You are talking about daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_encoding.c right? It should be pretty straightforward to backport it there. Martin From simo at redhat.com Fri Nov 4 12:03:06 2011 From: simo at redhat.com (Simo Sorce) Date: Fri, 04 Nov 2011 08:03:06 -0400 Subject: [Freeipa-devel] [PATCH] #2038 modify salt creation In-Reply-To: <1320407710.13980.39.camel@balmora.brq.redhat.com> References: <1320359175.7734.699.camel@willson.li.ssimo.org> <20111104080403.GD26667@redhat.com> <1320401646.13980.3.camel@balmora.brq.redhat.com> <1320406867.7734.706.camel@willson.li.ssimo.org> <1320407710.13980.39.camel@balmora.brq.redhat.com> Message-ID: <1320408186.7734.707.camel@willson.li.ssimo.org> On Fri, 2011-11-04 at 12:55 +0100, Martin Kosek wrote: > On Fri, 2011-11-04 at 07:41 -0400, Simo Sorce wrote: > > On Fri, 2011-11-04 at 11:14 +0100, Martin Kosek wrote: > > > On Fri, 2011-11-04 at 10:04 +0200, Alexander Bokovoy wrote: > > > > On Thu, 03 Nov 2011, Simo Sorce wrote: > > > > > As stated in the bug in order to attain better interoperability with > > > > > Windows clients we need to change the way we generate the random salt. > > > > ACK. > > > > > > > > > > Pushed to master. > > > > Should we backport this to 2.x as well ? > > > > Simo. > > > > Hm, looks important enough to do it. You are talking about > > daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_encoding.c Yes > right? It should be pretty straightforward to backport it there. Yes Simo. -- Simo Sorce * Red Hat, Inc * New York From pvoborni at redhat.com Fri Nov 4 12:21:19 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 04 Nov 2011 13:21:19 +0100 Subject: [Freeipa-devel] [PATCH] 030 Extending facet's mechanism of gathering changes In-Reply-To: <4EB31AFE.5010401@redhat.com> References: <4EB28A7A.3080905@redhat.com> <4EB31AFE.5010401@redhat.com> Message-ID: <4EB3D8BF.5060204@redhat.com> On 11/03/2011 11:51 PM, Endi Sukma Dewata wrote: > On 11/3/2011 7:35 AM, Petr Vobornik wrote: >> https://fedorahosted.org/freeipa/ticket/2041 >> > Other issues: > > 1. In details.js:650 we don't use param_info anymore, it should be > metadata. Fixed > > 2. The add_field_option(command, field_name, param_info, values, join) > probably can be simplified into add_field_option(field, values). The > IPA.command_builder can store the command internally: > > var command_builder = IPA.command_builder({ > command: command > }); > > ... > > command_builder.add_field_option(field_info.field, values); > > The add_field_option() can get the name, metadata, and join from the > field object. Transformed to following usage: IPA.command_builder.add_field_option(command, field, values); I'm considering command builder more as an utility class, than proper builder. If it would gather more functionality it would be better to changed it that way. > 3. The add_record_to_command() takes a list of sections, but it might > not be necessary because (so far) it's only used to access the details > facet's own list of sections. Do you expect this to be used differently? Removed (4). But you were right. The parameter would be nice if it was in some utility class, not details facet where different usage would be unlikely. > 4. The create_fields_update_command() is essentially the same as > create_standard_update_command(). When the command_mode is 'save' is it > possible to generate an update_info from records so we can just call > create_fields_update_command()? Created save_as_update_info(only_dirty, require_value) method which should do the trick. It internally use save(record) method do get all data and the parameters are used to get only the changes. It allowed to delete add_record_to_command and create_fields_update_command methods. > > This patch I think can be ACKed after fixing #1, the rest can be dealt > with later. > > Some of the new codes are not used anywhere yet so it's a bit difficult > to review and things might change again later. I'm curious to see how > it's going to be used in HBAC/sudo rule and DNS zone that involves > multiple commands: how the commands are going to be defined, how to > associate certain fields with certain commands, how to assign > priorities, etc. Do you have any preview patch for ticket #1515? Attached preview patch for #1515. Also attaching diff patch of reviewed patch. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: wip-freeipa-pvoborni-0013-Review-changes.patch Type: text/x-patch Size: 6768 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: wip-freeipa-pvoborni-0014-Code-cleanup-of-HBAC-Sudo-rules.patch Type: text/x-patch Size: 67669 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0030-1-Extending-facet-s-mechanism-of-gathering-changes.patch Type: text/x-patch Size: 16346 bytes Desc: not available URL: From rcritten at redhat.com Fri Nov 4 13:23:22 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 04 Nov 2011 09:23:22 -0400 Subject: [Freeipa-devel] [PATCH] 157 Add --delattr option to complement --setattr/--addattr In-Reply-To: <1319714558.26374.13.camel@balmora.brq.redhat.com> References: <1319714558.26374.13.camel@balmora.brq.redhat.com> Message-ID: <4EB3E74A.1000905@redhat.com> Martin Kosek wrote: > Add a --delattr option to round out multi-valued attribute manipulation. > The new option is be available for all LDAPUpdate based commands. > > --delattr is evaluated last, it can remove any value present either > in --addattr/--setattr options or stored in LDAP. > > https://fedorahosted.org/freeipa/ticket/1929 Should --delattr raise an error if the value doesn't exist? I think it probably should. rob From simo at redhat.com Fri Nov 4 14:49:40 2011 From: simo at redhat.com (Simo Sorce) Date: Fri, 04 Nov 2011 10:49:40 -0400 Subject: [Freeipa-devel] [PATCHES] #1791 Tust Effort: Add support for generating MS-PAC Message-ID: <1320418180.7734.720.camel@willson.li.ssimo.org> The attached patches are for master and concern the effort of creating trust relationships between IPA and AD domains. With these patches if you have run ipa-adtrust-install the IPA kdc will be able to create a MS-PAC if the user has the right attributes ipaNTSecurityIdentifier on the user entry and on the primary group entry are required (or a fallback primary group). If the objects are not in place the MS-PAC generation is silently skipped and no MS-PAC will be attached to the tickets. The MS-PAC is always generated if all data is available, in future we may think of making this conditional, but that is not in the scope of this patches. In order to apply these patches you need the coverity fix patches #2036 #2037 I sent yesterday. In order to build this code you need samba 4 experimental packages with the libndr_krb5pac.so librray, header files and pkgconfig configuration files. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-support-for-generating-PAC-for-AS-requests-for-u.patch Type: text/x-patch Size: 32966 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-MS-PAC-Add-support-for-verifying-PAC-in-TGS-requests.patch Type: text/x-patch Size: 4020 bytes Desc: not available URL: From simo at redhat.com Fri Nov 4 15:43:43 2011 From: simo at redhat.com (Simo Sorce) Date: Fri, 04 Nov 2011 11:43:43 -0400 Subject: [Freeipa-devel] [PATCH] #2038 modify salt creation In-Reply-To: <1320408186.7734.707.camel@willson.li.ssimo.org> References: <1320359175.7734.699.camel@willson.li.ssimo.org> <20111104080403.GD26667@redhat.com> <1320401646.13980.3.camel@balmora.brq.redhat.com> <1320406867.7734.706.camel@willson.li.ssimo.org> <1320407710.13980.39.camel@balmora.brq.redhat.com> <1320408186.7734.707.camel@willson.li.ssimo.org> Message-ID: <1320421423.7734.721.camel@willson.li.ssimo.org> On Fri, 2011-11-04 at 08:03 -0400, Simo Sorce wrote: > On Fri, 2011-11-04 at 12:55 +0100, Martin Kosek wrote: > > On Fri, 2011-11-04 at 07:41 -0400, Simo Sorce wrote: > > > On Fri, 2011-11-04 at 11:14 +0100, Martin Kosek wrote: > > > > On Fri, 2011-11-04 at 10:04 +0200, Alexander Bokovoy wrote: > > > > > On Thu, 03 Nov 2011, Simo Sorce wrote: > > > > > > As stated in the bug in order to attain better interoperability with > > > > > > Windows clients we need to change the way we generate the random salt. > > > > > ACK. > > > > > > > > > > > > > Pushed to master. > > > > > > Should we backport this to 2.x as well ? > > > > > > Simo. > > > > > > > Hm, looks important enough to do it. You are talking about > > > > daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_encoding.c > > Yes > > > right? It should be pretty straightforward to backport it there. > > Yes Patch against ipa-2-1 attached. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: ipa-2-1-Modify-random-salt-creation-for-interoperability.patch Type: text/x-patch Size: 2674 bytes Desc: not available URL: From pvoborni at redhat.com Fri Nov 4 16:02:41 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 04 Nov 2011 17:02:41 +0100 Subject: [Freeipa-devel] [PATCH] 029 Page is cleared before it is visible In-Reply-To: <4EB30607.1080807@redhat.com> References: <4EA92AF7.8080609@redhat.com> <4EAF2365.8000005@redhat.com> <4EAFE81C.4030004@redhat.com> <4EB07331.9070606@redhat.com> <4EB146AE.9010609@redhat.com> <4EB164C1.4090308@redhat.com> <4EB16940.4040708@redhat.com> <4EB30607.1080807@redhat.com> Message-ID: <4EB40CA1.2090503@redhat.com> On 11/03/2011 10:22 PM, Endi Sukma Dewata wrote: > On 11/2/2011 11:01 AM, Petr Vobornik wrote: >>> Regardless, ACK and pushed to master. > > Found another problem, the krbtpolicy & config need to be forced to > update. See the attached patch. > ACK and pushed to master. -- Petr Vobornik From pvoborni at redhat.com Fri Nov 4 16:04:37 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 04 Nov 2011 17:04:37 +0100 Subject: [Freeipa-devel] [PATCH] 306 Moved facet code into facet.js. In-Reply-To: <4EB35D8A.80405@redhat.com> References: <4EB35D8A.80405@redhat.com> Message-ID: <4EB40D15.8070304@redhat.com> On 11/04/2011 04:35 AM, Endi Sukma Dewata wrote: > Facet-related code has been moved from entity.js into a new facet.js > because the file is getting too big. > ACK and pushed to master -- Petr Vobornik From pvoborni at redhat.com Fri Nov 4 16:10:17 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 04 Nov 2011 17:10:17 +0100 Subject: [Freeipa-devel] [PATCH] 307 Added extensible UI framework. In-Reply-To: <4EB35E02.6020700@redhat.com> References: <4EB35E02.6020700@redhat.com> Message-ID: <4EB40E69.7010904@redhat.com> On 11/04/2011 04:37 AM, Endi Sukma Dewata wrote: > The entity definitions have been converted into classes. The entity > init() method will use the builder to construct the facets and dialogs. > The UI can be customized by creating a subclass of the original entity > in extension.js and then overriding the init() method. > > Ticket #2043 There is a warning/error in browser when there is no extension.js present. This doesn't affect functionality, but I think we should try to eliminate this kinds of error. Same problem is for develop.js on production machines. This can be fixed separately. ACK and pushed to master -- Petr Vobornik From ayoung at redhat.com Fri Nov 4 17:11:20 2011 From: ayoung at redhat.com (Adam Young) Date: Fri, 04 Nov 2011 13:11:20 -0400 Subject: [Freeipa-devel] [PATCH] 307 Added extensible UI framework. In-Reply-To: <4EB40E69.7010904@redhat.com> References: <4EB35E02.6020700@redhat.com> <4EB40E69.7010904@redhat.com> Message-ID: <4EB41CB8.3010803@redhat.com> On 11/04/2011 12:10 PM, Petr Vobornik wrote: > On 11/04/2011 04:37 AM, Endi Sukma Dewata wrote: >> The entity definitions have been converted into classes. The entity >> init() method will use the builder to construct the facets and dialogs. >> The UI can be customized by creating a subclass of the original entity >> in extension.js and then overriding the init() method. >> >> Ticket #2043 > > There is a warning/error in browser when there is no extension.js > present. This doesn't affect functionality, but I think we should try > to eliminate this kinds of error. Same problem is for develop.js on > production machines. This can be fixed separately. > > ACK and pushed to master > One solution is to have stub extension.js and develop.js files with nothing in them. In the case of the extension.js file, it should be there, and served out of /etc/ipa/http. develope.jss should be done by converting of develop_blank.js to develop.js when packaging up the RPM From edewata at redhat.com Fri Nov 4 17:22:46 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Fri, 04 Nov 2011 12:22:46 -0500 Subject: [Freeipa-devel] [PATCH] 030 Extending facet's mechanism of gathering changes In-Reply-To: <4EB3D8BF.5060204@redhat.com> References: <4EB28A7A.3080905@redhat.com> <4EB31AFE.5010401@redhat.com> <4EB3D8BF.5060204@redhat.com> Message-ID: <4EB41F66.2010902@redhat.com> Rebased, ACK, and pushed to master. Some comments below. On 11/4/2011 7:21 AM, Petr Vobornik wrote: > I'm considering command builder more as an utility class, than proper > builder. If it would gather more functionality it would be better to > changed it that way. I think in general a utility class doesn't always have to be a singular object. It involves a loop and you'll be passing the same objects over multiple invocations, we might want to consider refactoring that method into a separate utility class. Also consider enhancing the class itself rather than relying on a utility class. Take a look at IPA.update_info_builder, this class is now handling different objects: update_info, field_info, and command_info. However, it's not clear which class the merge() and copy() are handling unless we look into the implementation or rename the methods to include the class name. In my opinion the code will look a lot cleaner if the methods are moved into the corresponding classes. Just something to think about. >> 4. The create_fields_update_command() is essentially the same as >> create_standard_update_command(). When the command_mode is 'save' is it >> possible to generate an update_info from records so we can just call >> create_fields_update_command()? > > Created save_as_update_info(only_dirty, require_value) method which > should do the trick. > > It internally use save(record) method do get all data and the parameters > are used to get only the changes. It allowed to delete > add_record_to_command and create_fields_update_command methods. Perhaps the save_as_update_info() later can be merged with get_update_info() too because both are essentially generating update_info for dirty fields. > Attached preview patch for #1515. Also attaching diff patch of reviewed > patch. OK, I see how the enable widget creates the update info. How would you handle the removal of users in HBAC rule when the usercategory is changed to ALL? -- Endi S. Dewata From edewata at redhat.com Fri Nov 4 17:40:50 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Fri, 04 Nov 2011 12:40:50 -0500 Subject: [Freeipa-devel] [PATCH] 307 Added extensible UI framework. In-Reply-To: <4EB41CB8.3010803@redhat.com> References: <4EB35E02.6020700@redhat.com> <4EB40E69.7010904@redhat.com> <4EB41CB8.3010803@redhat.com> Message-ID: <4EB423A2.8070804@redhat.com> On 11/4/2011 12:11 PM, Adam Young wrote: >> There is a warning/error in browser when there is no extension.js >> present. This doesn't affect functionality, but I think we should try >> to eliminate this kinds of error. Same problem is for develop.js on >> production machines. This can be fixed separately. >> > One solution is to have stub extension.js and develop.js files with > nothing in them. In the case of the extension.js file, it should be > there, and served out of /etc/ipa/http. develope.jss should be done by > converting of develop_blank.js to develop.js when packaging up the RPM I don't actually see any warnings with Firefox, but I agree we shouldn't create a broken link. Ideally when viewing the static files during development we shouldn't have a broken link either, not just in the production machine. How about this, we rename the develop.js into extension.js, but we don't include it in the RPM. Then during RPM install we will touch extension.js so we wouldn't override the existing file, and not remove it during uninstall either? Then we can remove any references to develop.js too. Is it ok to create the extension.js in /usr/share/ipa/ui? -- Endi S. Dewata From nalin at redhat.com Fri Nov 4 19:15:13 2011 From: nalin at redhat.com (Nalin Dahyabhai) Date: Fri, 4 Nov 2011 15:15:13 -0400 Subject: [Freeipa-devel] [PATCH] #2038 modify salt creation In-Reply-To: <1320359175.7734.699.camel@willson.li.ssimo.org> References: <1320359175.7734.699.camel@willson.li.ssimo.org> Message-ID: <20111104191513.GA29196@redhat.com> On Thu, Nov 03, 2011 at 06:26:15PM -0400, Simo Sorce wrote: > As stated in the bug in order to attain better interoperability with > Windows clients we need to change the way we generate the random salt. Nack. The data in a krb5_data is of type 'char', and if it's signed, the math used here doesn't produce a printable result. Might also want to increase KRB5P_SALT_SIZE. Nalin From simo at redhat.com Fri Nov 4 19:59:06 2011 From: simo at redhat.com (Simo Sorce) Date: Fri, 04 Nov 2011 15:59:06 -0400 Subject: [Freeipa-devel] [PATCH] #2038 modify salt creation In-Reply-To: <20111104191513.GA29196@redhat.com> References: <1320359175.7734.699.camel@willson.li.ssimo.org> <20111104191513.GA29196@redhat.com> Message-ID: <1320436746.7734.741.camel@willson.li.ssimo.org> On Fri, 2011-11-04 at 15:15 -0400, Nalin Dahyabhai wrote: > On Thu, Nov 03, 2011 at 06:26:15PM -0400, Simo Sorce wrote: > > As stated in the bug in order to attain better interoperability with > > Windows clients we need to change the way we generate the random salt. > > Nack. The data in a krb5_data is of type 'char', and if it's signed, > the math used here doesn't produce a printable result. Might also want > to increase KRB5P_SALT_SIZE. Ah crap, right. I initially used a safe construct: data[i] &= 0x5F Then realized that one of the possible values (5F + 20 = 7F) is unprintable, so I switched to this unsafe one. Will get a revised patch for ipa-2-1 and an amendment for master. Thanks a lot for spotting this one! Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Fri Nov 4 20:14:55 2011 From: simo at redhat.com (Simo Sorce) Date: Fri, 04 Nov 2011 16:14:55 -0400 Subject: [Freeipa-devel] [PATCH] #2038 modify salt creation In-Reply-To: <1320436746.7734.741.camel@willson.li.ssimo.org> References: <1320359175.7734.699.camel@willson.li.ssimo.org> <20111104191513.GA29196@redhat.com> <1320436746.7734.741.camel@willson.li.ssimo.org> Message-ID: <1320437695.7734.742.camel@willson.li.ssimo.org> On Fri, 2011-11-04 at 15:59 -0400, Simo Sorce wrote: > On Fri, 2011-11-04 at 15:15 -0400, Nalin Dahyabhai wrote: > > On Thu, Nov 03, 2011 at 06:26:15PM -0400, Simo Sorce wrote: > > > As stated in the bug in order to attain better interoperability with > > > Windows clients we need to change the way we generate the random salt. > > > > Nack. The data in a krb5_data is of type 'char', and if it's signed, > > the math used here doesn't produce a printable result. Might also want > > to increase KRB5P_SALT_SIZE. > > Ah crap, right. > > I initially used a safe construct: data[i] &= 0x5F > Then realized that one of the possible values (5F + 20 = 7F) is > unprintable, so I switched to this unsafe one. > > Will get a revised patch for ipa-2-1 and an amendment for master. > > Thanks a lot for spotting this one! Attached amendment patch for master and an already amended new patch for ipa-2-1. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Amend-2038-fix.patch Type: text/x-patch Size: 1171 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ipa-2-1-Modify-random-salt-creation-for-interoperability-2.patch Type: text/x-patch Size: 2940 bytes Desc: not available URL: From simo at redhat.com Fri Nov 4 20:45:02 2011 From: simo at redhat.com (Simo Sorce) Date: Fri, 04 Nov 2011 16:45:02 -0400 Subject: [Freeipa-devel] [PATCH] #2038 modify salt creation In-Reply-To: <1320437695.7734.742.camel@willson.li.ssimo.org> References: <1320359175.7734.699.camel@willson.li.ssimo.org> <20111104191513.GA29196@redhat.com> <1320436746.7734.741.camel@willson.li.ssimo.org> <1320437695.7734.742.camel@willson.li.ssimo.org> Message-ID: <1320439502.7734.744.camel@willson.li.ssimo.org> On Fri, 2011-11-04 at 16:14 -0400, Simo Sorce wrote: > On Fri, 2011-11-04 at 15:59 -0400, Simo Sorce wrote: > > On Fri, 2011-11-04 at 15:15 -0400, Nalin Dahyabhai wrote: > > > On Thu, Nov 03, 2011 at 06:26:15PM -0400, Simo Sorce wrote: > > > > As stated in the bug in order to attain better interoperability with > > > > Windows clients we need to change the way we generate the random salt. > > > > > > Nack. The data in a krb5_data is of type 'char', and if it's signed, > > > the math used here doesn't produce a printable result. Might also want > > > to increase KRB5P_SALT_SIZE. > > > > Ah crap, right. > > > > I initially used a safe construct: data[i] &= 0x5F > > Then realized that one of the possible values (5F + 20 = 7F) is > > unprintable, so I switched to this unsafe one. > > > > Will get a revised patch for ipa-2-1 and an amendment for master. > > > > Thanks a lot for spotting this one! > > Attached amendment patch for master and an already amended new patch for > ipa-2-1. After a quick review with nalin offline I decided for a different approach that properly covers the range of values we want and is more similar to the initial code. New patches attached. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Amend-2038-fix-2.patch Type: text/x-patch Size: 1295 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ipa-2-1-Modify-random-salt-creation-for-interoperability-3.patch Type: text/x-patch Size: 2752 bytes Desc: not available URL: From rcritten at redhat.com Fri Nov 4 20:53:24 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 04 Nov 2011 16:53:24 -0400 Subject: [Freeipa-devel] [PATCH] 120 Improve DNS record data validation In-Reply-To: <1319533225.14006.13.camel@balmora.brq.redhat.com> References: <1315400728.12548.12.camel@dhcp-25-52.brq.redhat.com> <1315401507.12548.14.camel@dhcp-25-52.brq.redhat.com> <1319026515.9647.13.camel@balmora.brq.redhat.com> <4E9F2742.3020109@redhat.com> <1319533225.14006.13.camel@balmora.brq.redhat.com> Message-ID: <4EB450C4.2060705@redhat.com> Martin Kosek wrote: > On Wed, 2011-10-19 at 15:38 -0400, Adam Young wrote: >> On 10/19/2011 08:15 AM, Martin Kosek wrote: >>> On Wed, 2011-09-07 at 15:18 +0200, Martin Kosek wrote: >>>> On Wed, 2011-09-07 at 15:05 +0200, Martin Kosek wrote: >>>>> This is 3.0 Core Effort Backlog patch. >>>>> >>>>> The changes to API may look scary, but it should be OK, I just added >>>>> validators and normalizers. I found a lot of RR types unsupported by >>>>> bind-dyndb-ldap. I implemented a validator telling this information to >>>>> the user. I think the message is more user-friendly than the previous >>>>> LDAP schema error. >>>>> >>>>> Enjoy the RFCs! :-) >>>>> >>>>> Martin >>>>> >>>>> --- >>>>> Implement missing validators for DNS RR types so that we can capture >>>>> at least basic user errors. Additionally, a normalizer creating >>>>> a fully-qualified domain name has been implemented for several RRs >>>>> to prevent this common user error. >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/1106 >>>>> >>>> I noticed a typo in format description for LOC record validation. A >>>> fixed patch attached. >>>> >>>> Martin >>> Rebased for current master. >>> >>> This patch is still waiting for review. As I would like to base my next >>> DNS work (structured DNS commands) on this patch I would like to have it >>> reviewed soon. >>> >>> Thanks, >>> Martin >>> >>> >>> >>> _______________________________________________ >>> Freeipa-devel mailing list >>> Freeipa-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/freeipa-devel >> >> >> I've just given it a visual review, but it looks right. Probably >> should have some unit tests to go with it for some of the more >> commonly used types. > > Good idea. A, AAAA, NS records are already being checked, I added tests > for MX and SRV records too. > > I also refactored DNS tests a little, there were many repeatedly using > hard-coded values (like default zone manager) which would be hard to fix > of anything changes. > > Martin I can't tell what your intention is with the split for cname and dname but it seems to allow just about any value. I know there are a ton of data types but is it worthwhile to have a positive and negative case for each to avoid regressions? rob From nalin at redhat.com Fri Nov 4 20:55:08 2011 From: nalin at redhat.com (Nalin Dahyabhai) Date: Fri, 4 Nov 2011 16:55:08 -0400 Subject: [Freeipa-devel] [PATCH] #2038 modify salt creation In-Reply-To: <1320439502.7734.744.camel@willson.li.ssimo.org> References: <1320359175.7734.699.camel@willson.li.ssimo.org> <20111104191513.GA29196@redhat.com> <1320436746.7734.741.camel@willson.li.ssimo.org> <1320437695.7734.742.camel@willson.li.ssimo.org> <1320439502.7734.744.camel@willson.li.ssimo.org> Message-ID: <20111104205508.GB29196@redhat.com> On Fri, Nov 04, 2011 at 04:45:02PM -0400, Simo Sorce wrote: > After a quick review with nalin offline I decided for a different > approach that properly covers the range of values we want and is more > similar to the initial code. > > New patches attached. Looks good to me. Please bump up KRB5P_SALT_SIZE, say, to 20, unless there's a good reason not to, though. Either way, ACK. Nalin From rcritten at redhat.com Fri Nov 4 21:02:43 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 04 Nov 2011 17:02:43 -0400 Subject: [Freeipa-devel] [PATCH] 152 Enable automember for upgraded servers In-Reply-To: <1318854040.26757.8.camel@balmora.brq.redhat.com> References: <1318854040.26757.8.camel@balmora.brq.redhat.com> Message-ID: <4EB452F3.1050806@redhat.com> Martin Kosek wrote: > automember functionality is depends on predefined data is in LDAP. > Since we add it for fresh installs only, automember cannot be used > for upgraded servers. Make sure that automember LDAP data is added > during upgrade too. > > https://fedorahosted.org/freeipa/ticket/1992 I think you need that automember schema as well. Can you check with the 389-ds team to see if their upgrade script automatically adds new schema or if we have to handle that ourselves? rob From nkinder at redhat.com Fri Nov 4 21:04:35 2011 From: nkinder at redhat.com (Nathan Kinder) Date: Fri, 04 Nov 2011 14:04:35 -0700 Subject: [Freeipa-devel] [PATCH] 152 Enable automember for upgraded servers In-Reply-To: <4EB452F3.1050806@redhat.com> References: <1318854040.26757.8.camel@balmora.brq.redhat.com> <4EB452F3.1050806@redhat.com> Message-ID: <4EB45363.6070303@redhat.com> On 11/04/2011 02:02 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> automember functionality is depends on predefined data is in LDAP. >> Since we add it for fresh installs only, automember cannot be used >> for upgraded servers. Make sure that automember LDAP data is added >> during upgrade too. >> >> https://fedorahosted.org/freeipa/ticket/1992 > > I think you need that automember schema as well. Can you check with > the 389-ds team to see if their upgrade script automatically adds new > schema or if we have to handle that ourselves? The new automember schema should be added by 'setup-ds.pl -u', so I don't expect you need to do anything around schema in FreeIPA. > > rob > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From rcritten at redhat.com Fri Nov 4 21:25:11 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 04 Nov 2011 17:25:11 -0400 Subject: [Freeipa-devel] [PATCH] 55 Parse comma-separated lists of values in all parameter types In-Reply-To: <4EA59620.30506@redhat.com> References: <4EA00403.9060703@redhat.com> <4EA583AB.6030906@redhat.com> <4EA5877B.2020606@redhat.com> <4EA59620.30506@redhat.com> Message-ID: <4EB45837.2020606@redhat.com> Jan Cholasta wrote: > Dne 24.10.2011 17:42, Rob Crittenden napsal(a): >> Jan Cholasta wrote: >>> Dne 20.10.2011 13:20, Jan Cholasta napsal(a): >>>> Parse comma-separated lists of values in all parameter types. This can >>>> enabled for a specific parameter by setting the "csvlist" option to >>>> True. >>>> >>>> Remove List parameter type and replace all occurences with Str with >>>> csvlist enabled. >>>> >>>> https://fedorahosted.org/freeipa/ticket/2007 >>>> >>>> This change will be useful for >>>> https://fedorahosted.org/freeipa/ticket/1487 and >>>> https://fedorahosted.org/freeipa/ticket/1847 >>>> >>>> Unit tests show no regressions. >>>> >>>> Honza >>>> >>> >>> Self-NACK - I have noticed that the batch command no longer works. >>> >>> Updated patch attached. >>> >>> Honza >> >> What is the benefit of this over the List parameter type? >> >> rob > > Mainly because the List parameter type is just a hack. This is the right > thing to do if we want to use comma-separated lists of parameters of any > type, with all the validation and other parameter type-specific features. > > For example, I've added a new parameter type for IP addresses in my > patch 46 > (http://www.redhat.com/archives/freeipa-devel/2011-September/msg00187.html) > and use it for A and AAAA DNS records. Without this patch, we can either > use List for the record parameters and lose validation in dnsrecord-find > (because it is based on crud.Search, which strips all the custom > validation rules - like _validate_ipaddr - from the command parameters, > which is one of the causes of #1627) or use IPAddress for the record > parameters and lose the ability to specify them as comma-separated list > of values. With this patch, we can have both comma-separated lists and > validation at the same time. > > Besides, the patch is not as big as it looks like, all the interesting > stuff is in ipalib/parameters.py, everything else is just > search-and-replace. Also I need it to fix #1487 and #1847 without doing > ugly hacks. > > Honza > I think this would constitute a major version change. One downside is you can no longer tell in the help with arguments take a CSV and which don't. I think the CSV-related Parameter options should all begin with csv, separator and skipspace. The batch command may eventually be made into a command, how will that affect the Any type? It otherwise seems to work in my spot-testing. rob From mkosek at redhat.com Fri Nov 4 21:26:35 2011 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 04 Nov 2011 22:26:35 +0100 Subject: [Freeipa-devel] [PATCH] 152 Enable automember for upgraded servers In-Reply-To: <4EB45363.6070303@redhat.com> References: <1318854040.26757.8.camel@balmora.brq.redhat.com> <4EB452F3.1050806@redhat.com> <4EB45363.6070303@redhat.com> Message-ID: <1320441997.2811.3.camel@priserak> On Fri, 2011-11-04 at 14:04 -0700, Nathan Kinder wrote: > On 11/04/2011 02:02 PM, Rob Crittenden wrote: > > Martin Kosek wrote: > >> automember functionality is depends on predefined data is in LDAP. > >> Since we add it for fresh installs only, automember cannot be used > >> for upgraded servers. Make sure that automember LDAP data is added > >> during upgrade too. > >> > >> https://fedorahosted.org/freeipa/ticket/1992 > > > > I think you need that automember schema as well. Can you check with > > the 389-ds team to see if their upgrade script automatically adds new > > schema or if we have to handle that ourselves? > The new automember schema should be added by 'setup-ds.pl -u', so I > don't expect you need to do anything around schema in FreeIPA. > > Nathan, when is the "setup-ds.pl -u" executed? When the dirsrv rpm is updated, just like FreeIPA runs ipa-ldap-updater in rpm update %post? Or does it have to be run manually? I am asking because the schema problem seems like the root cause that one user has here (the last post): https://bugzilla.redhat.com/show_bug.cgi?id=746589 Thanks, Martin From nkinder at redhat.com Fri Nov 4 21:35:47 2011 From: nkinder at redhat.com (Nathan Kinder) Date: Fri, 04 Nov 2011 14:35:47 -0700 Subject: [Freeipa-devel] [PATCH] 152 Enable automember for upgraded servers In-Reply-To: <1320441997.2811.3.camel@priserak> References: <1318854040.26757.8.camel@balmora.brq.redhat.com> <4EB452F3.1050806@redhat.com> <4EB45363.6070303@redhat.com> <1320441997.2811.3.camel@priserak> Message-ID: <4EB45AB3.6010305@redhat.com> On 11/04/2011 02:26 PM, Martin Kosek wrote: > On Fri, 2011-11-04 at 14:04 -0700, Nathan Kinder wrote: >> On 11/04/2011 02:02 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> automember functionality is depends on predefined data is in LDAP. >>>> Since we add it for fresh installs only, automember cannot be used >>>> for upgraded servers. Make sure that automember LDAP data is added >>>> during upgrade too. >>>> >>>> https://fedorahosted.org/freeipa/ticket/1992 >>> I think you need that automember schema as well. Can you check with >>> the 389-ds team to see if their upgrade script automatically adds new >>> schema or if we have to handle that ourselves? >> The new automember schema should be added by 'setup-ds.pl -u', so I >> don't expect you need to do anything around schema in FreeIPA. > Nathan, when is the "setup-ds.pl -u" executed? When the dirsrv rpm is > updated, just like FreeIPA runs ipa-ldap-updater in rpm update %post? Or > does it have to be run manually? It is run in the the %posttrans stage for 389-ds-base. > I am asking because the schema problem seems like the root cause that > one user has here (the last post): > > https://bugzilla.redhat.com/show_bug.cgi?id=746589 There should be a '/etc/dirsrv/slapd-/schema/10automember-plugin.ldif' file if the proper version of 389-ds-base is being used and if 'setup-ds.pl -u' successfully updated the schema. There should also be a '/etc/dirsrv/schema/10automember-plugin.ldif' file present regardless of 'setup-ds.pl -u' having run successfully. -NGK > Thanks, > Martin > From nkinder at redhat.com Fri Nov 4 21:50:17 2011 From: nkinder at redhat.com (Nathan Kinder) Date: Fri, 04 Nov 2011 14:50:17 -0700 Subject: [Freeipa-devel] [PATCH] 152 Enable automember for upgraded servers In-Reply-To: <4EB45AB3.6010305@redhat.com> References: <1318854040.26757.8.camel@balmora.brq.redhat.com> <4EB452F3.1050806@redhat.com> <4EB45363.6070303@redhat.com> <1320441997.2811.3.camel@priserak> <4EB45AB3.6010305@redhat.com> Message-ID: <4EB45E19.6060606@redhat.com> On 11/04/2011 02:35 PM, Nathan Kinder wrote: > On 11/04/2011 02:26 PM, Martin Kosek wrote: >> On Fri, 2011-11-04 at 14:04 -0700, Nathan Kinder wrote: >>> On 11/04/2011 02:02 PM, Rob Crittenden wrote: >>>> Martin Kosek wrote: >>>>> automember functionality is depends on predefined data is in LDAP. >>>>> Since we add it for fresh installs only, automember cannot be used >>>>> for upgraded servers. Make sure that automember LDAP data is added >>>>> during upgrade too. >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/1992 >>>> I think you need that automember schema as well. Can you check with >>>> the 389-ds team to see if their upgrade script automatically adds new >>>> schema or if we have to handle that ourselves? >>> The new automember schema should be added by 'setup-ds.pl -u', so I >>> don't expect you need to do anything around schema in FreeIPA. >> Nathan, when is the "setup-ds.pl -u" executed? When the dirsrv rpm is >> updated, just like FreeIPA runs ipa-ldap-updater in rpm update %post? Or >> does it have to be run manually? > It is run in the the %posttrans stage for 389-ds-base. >> I am asking because the schema problem seems like the root cause that >> one user has here (the last post): >> >> https://bugzilla.redhat.com/show_bug.cgi?id=746589 > There should be a > '/etc/dirsrv/slapd-/schema/10automember-plugin.ldif' file if > the proper version > of 389-ds-base is being used and if 'setup-ds.pl -u' successfully > updated the schema. There should also be > a '/etc/dirsrv/schema/10automember-plugin.ldif' file present > regardless of 'setup-ds.pl -u' having run > successfully. I just tested running 'setup-ds.pl -u' manually with a master build of 389-ds-base, and there is a bug that is preventing the updates from being applied. I logged the following bug for this: https://bugzilla.redhat.com/show_bug.cgi?id=751495 The fix is a one-liner, and I believe Rich is working on getting a fixed build out ASAP. > > -NGK >> Thanks, >> Martin >> > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From edewata at redhat.com Fri Nov 4 22:29:27 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Fri, 04 Nov 2011 17:29:27 -0500 Subject: [Freeipa-devel] [PATCH] 308 Added current password field. Message-ID: <4EB46747.7060509@redhat.com> The reset password dialog for user has been modified to provide a field to specify the current password when changing the user's own password. Ticket #2065 -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-edewata-0308-Added-current-password-field.patch Type: text/x-patch Size: 4996 bytes Desc: not available URL: From sbose at redhat.com Fri Nov 4 22:31:50 2011 From: sbose at redhat.com (Sumit Bose) Date: Fri, 4 Nov 2011 23:31:50 +0100 Subject: [Freeipa-devel] [PATCHES] #1791 Tust Effort: Add support for generating MS-PAC In-Reply-To: <1320418180.7734.720.camel@willson.li.ssimo.org> References: <1320418180.7734.720.camel@willson.li.ssimo.org> Message-ID: <20111104223150.GF2831@localhost.localdomain> On Fri, Nov 04, 2011 at 10:49:40AM -0400, Simo Sorce wrote: > The attached patches are for master and concern the effort of creating > trust relationships between IPA and AD domains. > > With these patches if you have run ipa-adtrust-install the IPA kdc will > be able to create a MS-PAC if the user has the right attributes > ipaNTSecurityIdentifier on the user entry and on the primary group entry > are required (or a fallback primary group). > If the objects are not in place the MS-PAC generation is silently > skipped and no MS-PAC will be attached to the tickets. > > The MS-PAC is always generated if all data is available, in future we > may think of making this conditional, but that is not in the scope of > this patches. > > In order to apply these patches you need the coverity fix patches #2036 > #2037 I sent yesterday. > > In order to build this code you need samba 4 experimental packages with > the libndr_krb5pac.so librray, header files and pkgconfig configuration > files. Please add these dependencies to the BuildRequires in the spec file. Otherwise the patch looks fine. bye, Sumit > > Simo. > > -- > Simo Sorce * Red Hat, Inc * New York From edewata at redhat.com Sat Nov 5 00:37:37 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Fri, 04 Nov 2011 19:37:37 -0500 Subject: [Freeipa-devel] [PATCH] 029 Page is cleared before it is visible In-Reply-To: <4EB40CA1.2090503@redhat.com> References: <4EA92AF7.8080609@redhat.com> <4EAF2365.8000005@redhat.com> <4EAFE81C.4030004@redhat.com> <4EB07331.9070606@redhat.com> <4EB146AE.9010609@redhat.com> <4EB164C1.4090308@redhat.com> <4EB16940.4040708@redhat.com> <4EB30607.1080807@redhat.com> <4EB40CA1.2090503@redhat.com> Message-ID: <4EB48551.3000808@redhat.com> On 11/4/2011 11:02 AM, Petr Vobornik wrote: > ACK and pushed to master. Found another problem, changing page in the association facet didn't work because pkey is still the same. See the attached patch. -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-edewata-0309-Fixed-problem-changing-page-in-association-facet.patch Type: text/x-patch Size: 2534 bytes Desc: not available URL: From mkosek at redhat.com Sat Nov 5 12:43:25 2011 From: mkosek at redhat.com (Martin Kosek) Date: Sat, 05 Nov 2011 13:43:25 +0100 Subject: [Freeipa-devel] [PATCH] 157 Add --delattr option to complement --setattr/--addattr In-Reply-To: <4EB3E74A.1000905@redhat.com> References: <1319714558.26374.13.camel@balmora.brq.redhat.com> <4EB3E74A.1000905@redhat.com> Message-ID: <1320497005.25589.5.camel@priserak> On Fri, 2011-11-04 at 09:23 -0400, Rob Crittenden wrote: > Martin Kosek wrote: > > Add a --delattr option to round out multi-valued attribute manipulation. > > The new option is be available for all LDAPUpdate based commands. > > > > --delattr is evaluated last, it can remove any value present either > > in --addattr/--setattr options or stored in LDAP. > > > > https://fedorahosted.org/freeipa/ticket/1929 > > Should --delattr raise an error if the value doesn't exist? > > I think it probably should. > > rob You are right, it would be more expected behavior. I fixed that. In the process of implementing the change I found that current --*attr processing is not good, so I refactored it completely to one function available for all baseldap.py commands. In the process I found out that we don't have any common class for all baseldap.py commands and the result is BaseLDAPCommand which can now handle attribute processing and provide it to other LDAP commands. And I also fixed one group unit test. Now all unit tests were OK. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-157-2-add-delattr-option-to-complement-setattr-addattr.patch Type: text/x-patch Size: 86755 bytes Desc: not available URL: From simo at redhat.com Sat Nov 5 23:09:59 2011 From: simo at redhat.com (Simo Sorce) Date: Sat, 05 Nov 2011 19:09:59 -0400 Subject: [Freeipa-devel] [PATCH] #2038 modify salt creation In-Reply-To: <20111104205508.GB29196@redhat.com> References: <1320359175.7734.699.camel@willson.li.ssimo.org> <20111104191513.GA29196@redhat.com> <1320436746.7734.741.camel@willson.li.ssimo.org> <1320437695.7734.742.camel@willson.li.ssimo.org> <1320439502.7734.744.camel@willson.li.ssimo.org> <20111104205508.GB29196@redhat.com> Message-ID: <1320534599.7734.761.camel@willson.li.ssimo.org> On Fri, 2011-11-04 at 16:55 -0400, Nalin Dahyabhai wrote: > On Fri, Nov 04, 2011 at 04:45:02PM -0400, Simo Sorce wrote: > > After a quick review with nalin offline I decided for a different > > approach that properly covers the range of values we want and is more > > similar to the initial code. > > > > New patches attached. > > Looks good to me. Please bump up KRB5P_SALT_SIZE, say, to 20, unless > there's a good reason not to, though. I decided to open a separate ticket to investigate salt sizes. > Either way, ACK. Pushed to master and ipa-2-1 Simo. -- Simo Sorce * Red Hat, Inc * New York From jcholast at redhat.com Mon Nov 7 10:17:07 2011 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 07 Nov 2011 11:17:07 +0100 Subject: [Freeipa-devel] [PATCH] 55 Parse comma-separated lists of values in all parameter types In-Reply-To: <4EB45837.2020606@redhat.com> References: <4EA00403.9060703@redhat.com> <4EA583AB.6030906@redhat.com> <4EA5877B.2020606@redhat.com> <4EA59620.30506@redhat.com> <4EB45837.2020606@redhat.com> Message-ID: <4EB7B023.1030004@redhat.com> Dne 4.11.2011 22:25, Rob Crittenden napsal(a): > Jan Cholasta wrote: >> Dne 24.10.2011 17:42, Rob Crittenden napsal(a): >>> Jan Cholasta wrote: >>>> Dne 20.10.2011 13:20, Jan Cholasta napsal(a): >>>>> Parse comma-separated lists of values in all parameter types. This can >>>>> enabled for a specific parameter by setting the "csvlist" option to >>>>> True. >>>>> >>>>> Remove List parameter type and replace all occurences with Str with >>>>> csvlist enabled. >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/2007 >>>>> >>>>> This change will be useful for >>>>> https://fedorahosted.org/freeipa/ticket/1487 and >>>>> https://fedorahosted.org/freeipa/ticket/1847 >>>>> >>>>> Unit tests show no regressions. >>>>> >>>>> Honza >>>>> >>>> >>>> Self-NACK - I have noticed that the batch command no longer works. >>>> >>>> Updated patch attached. >>>> >>>> Honza >>> >>> What is the benefit of this over the List parameter type? >>> >>> rob >> >> Mainly because the List parameter type is just a hack. This is the right >> thing to do if we want to use comma-separated lists of parameters of any >> type, with all the validation and other parameter type-specific features. >> >> For example, I've added a new parameter type for IP addresses in my >> patch 46 >> (http://www.redhat.com/archives/freeipa-devel/2011-September/msg00187.html) >> >> and use it for A and AAAA DNS records. Without this patch, we can either >> use List for the record parameters and lose validation in dnsrecord-find >> (because it is based on crud.Search, which strips all the custom >> validation rules - like _validate_ipaddr - from the command parameters, >> which is one of the causes of #1627) or use IPAddress for the record >> parameters and lose the ability to specify them as comma-separated list >> of values. With this patch, we can have both comma-separated lists and >> validation at the same time. >> >> Besides, the patch is not as big as it looks like, all the interesting >> stuff is in ipalib/parameters.py, everything else is just >> search-and-replace. Also I need it to fix #1487 and #1847 without doing >> ugly hacks. >> >> Honza >> > > I think this would constitute a major version change. I'm not sure about that, as the patch doesn't break API compatibility - a string containing a comma-separated list of values is used for list parameters both with and without the patch. > > One downside is you can no longer tell in the help with arguments take a > CSV and which don't. Why not? A simple look at csvlist value should provide enough information. > > I think the CSV-related Parameter options should all begin with csv, > separator and skipspace. OK. > > The batch command may eventually be made into a command, how will that > affect the Any type? Batch currently uses List for the "methods" parameter not because of its CSV capability, but because it doesn't do any type conversion and validation on the values. This allows it to use values of the form "{u'params': [args_list, kwargs_dict], u'method': method_name}". The Any parameter type exists so that this can still be done without List - it's basically a single-valued version of List (i.e. Any(csvlist=True) is equivalent to List()). IMO if batch is ever made into a command, it would have to be redesigned not to use List/Any, so that it can be used from CLI with validation of the parameter values. Any can then be easily removed. > > It otherwise seems to work in my spot-testing. > > rob Honza -- Jan Cholasta From pvoborni at redhat.com Mon Nov 7 12:48:23 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 07 Nov 2011 13:48:23 +0100 Subject: [Freeipa-devel] [PATCH] 308 Added current password field. In-Reply-To: <4EB46747.7060509@redhat.com> References: <4EB46747.7060509@redhat.com> Message-ID: <4EB7D397.7070807@redhat.com> On 11/04/2011 11:29 PM, Endi Sukma Dewata wrote: > The reset password dialog for user has been modified to provide > a field to specify the current password when changing the user's > own password. > > Ticket #2065 > ACK ipa-2-1, master -- Petr Vobornik From mkosek at redhat.com Mon Nov 7 13:21:50 2011 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 07 Nov 2011 14:21:50 +0100 Subject: [Freeipa-devel] [PATCH] 308 Added current password field. In-Reply-To: <4EB7D397.7070807@redhat.com> References: <4EB46747.7060509@redhat.com> <4EB7D397.7070807@redhat.com> Message-ID: <1320672113.29771.8.camel@balmora.brq.redhat.com> On Mon, 2011-11-07 at 13:48 +0100, Petr Vobornik wrote: > On 11/04/2011 11:29 PM, Endi Sukma Dewata wrote: > > The reset password dialog for user has been modified to provide > > a field to specify the current password when changing the user's > > own password. > > > > Ticket #2065 > > > ACK ipa-2-1, master > Pushed to master, ipa-2-1. Martin From pvoborni at redhat.com Mon Nov 7 14:11:27 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 07 Nov 2011 15:11:27 +0100 Subject: [Freeipa-devel] [PATCH] 029 Page is cleared before it is visible In-Reply-To: <4EB48551.3000808@redhat.com> References: <4EA92AF7.8080609@redhat.com> <4EAF2365.8000005@redhat.com> <4EAFE81C.4030004@redhat.com> <4EB07331.9070606@redhat.com> <4EB146AE.9010609@redhat.com> <4EB164C1.4090308@redhat.com> <4EB16940.4040708@redhat.com> <4EB30607.1080807@redhat.com> <4EB40CA1.2090503@redhat.com> <4EB48551.3000808@redhat.com> Message-ID: <4EB7E70F.3030502@redhat.com> On 11/05/2011 01:37 AM, Endi Sukma Dewata wrote: > On 11/4/2011 11:02 AM, Petr Vobornik wrote: > Found another problem, changing page in the association facet didn't > work because pkey is still the same. See the attached patch. > ACK and pushed to master -- Petr Vobornik From pvoborni at redhat.com Mon Nov 7 14:46:42 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 07 Nov 2011 15:46:42 +0100 Subject: [Freeipa-devel] [PATCH] 030 Extending facet's mechanism of gathering changes In-Reply-To: <4EB41F66.2010902@redhat.com> References: <4EB28A7A.3080905@redhat.com> <4EB31AFE.5010401@redhat.com> <4EB3D8BF.5060204@redhat.com> <4EB41F66.2010902@redhat.com> Message-ID: <4EB7EF52.1030908@redhat.com> On 11/04/2011 06:22 PM, Endi Sukma Dewata wrote: > Rebased, ACK, and pushed to master. Some comments below. > > On 11/4/2011 7:21 AM, Petr Vobornik wrote: > > Perhaps the save_as_update_info() later can be merged with > get_update_info() too because both are essentially generating > update_info for dirty fields. > Yes, If we ensure that appropriate get_update_info() method will be in each custom widget/section, which isn't now. >> Attached preview patch for #1515. Also attaching diff patch of reviewed >> patch. > > OK, I see how the enable widget creates the update info. How would you > handle the removal of users in HBAC rule when the usercategory is > changed to ALL? > This logic is part of rule_association_table_widget:rule.js:142. The radio buttons are triggering an event which disables/enables the asso. table. During get_update_info if the asso. table is disabled and it has entries in it, it creates a command which deletes the entries. -- Petr Vobornik From mkosek at redhat.com Mon Nov 7 17:42:45 2011 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 07 Nov 2011 18:42:45 +0100 Subject: [Freeipa-devel] [PATCH] 160 Hosts file not updated when IP is passed as option Message-ID: <1320687767.29771.14.camel@balmora.brq.redhat.com> When an IPA server with unresolvable hostname is being installed, a hostname record must be inserted to /etc/hosts or the installation will fail. However, it is not inserted when IP address is passed as an option (--ip-address) and not interactively. This patch fixes this so that /etc/hosts record is inserted in both cases. https://fedorahosted.org/freeipa/ticket/2074 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-160-hosts-file-not-updated-when-ip-is-passed-as-option.patch Type: text/x-patch Size: 1740 bytes Desc: not available URL: From abokovoy at redhat.com Mon Nov 7 20:42:03 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 7 Nov 2011 22:42:03 +0200 Subject: [Freeipa-devel] [PATCHES] #2036 Fix coverity bugs In-Reply-To: <1320343596.7734.675.camel@willson.li.ssimo.org> References: <1320343596.7734.675.camel@willson.li.ssimo.org> Message-ID: <20111107204203.GB16228@redhat.com> On Thu, 03 Nov 2011, Simo Sorce wrote: > Just some unchecked returns, we do not care much, but will keep Coverity > happy. ACK to all below, they seem to be straight-forward. As slapi_ch_free_string() accepts a pointer to NULL and does nothing in that case, you don't need to protect against fetched NULL in ipa_pwd_extop.c. > https://fedorahosted.org/freeipa/ticket/2036 > --- > .../ipa-enrollment/ipa_enrollment.c | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/daemons/ipa-slapi-plugins/ipa-enrollment/ipa_enrollment.c b/daemons/ipa-slapi-plugins/ipa-enrollment/ipa_enrollment.c > index 1f5ce9b477a6152e3ef7befeea1230ab75dfba70..9f884bd39233adf90b0f4eff1868885d587d351a 100644 > --- a/daemons/ipa-slapi-plugins/ipa-enrollment/ipa_enrollment.c > +++ b/daemons/ipa-slapi-plugins/ipa-enrollment/ipa_enrollment.c > @@ -451,7 +451,7 @@ ipaenrollment_init(Slapi_PBlock *pb) > if (!ret) ret = slapi_pblock_set(pb, SLAPI_PLUGIN_DESCRIPTION, (void *)&pdesc); > if (!ret) ret = slapi_pblock_set(pb, SLAPI_PLUGIN_EXT_OP_OIDLIST, ipaenrollment_oid_list); > if (!ret) ret = slapi_pblock_set(pb, SLAPI_PLUGIN_EXT_OP_NAMELIST, ipaenrollment_name_list); > - if (!ret) slapi_pblock_set(pb, SLAPI_PLUGIN_EXT_OP_FN, (void *)ipaenrollment_extop); > + if (!ret) ret = slapi_pblock_set(pb, SLAPI_PLUGIN_EXT_OP_FN, (void *)ipaenrollment_extop); > > if (ret) { > LOG("Failed to set plug-in version, function, and OID.\n"); ACK. > https://fedorahosted.org/freeipa/ticket/2036 > --- > .../ipa-pwd-extop/ipa_pwd_extop.c | 10 ++++++++-- > 1 files changed, 8 insertions(+), 2 deletions(-) > > diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c > index 95ac68e9cfc8d7024048d8a9d2793044f01dd1aa..a0f9c5e14747b5c38952db27b005874a4afe1c4d 100644 > --- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c > +++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c > @@ -504,9 +504,15 @@ free_and_return: > /* Either this is the same pointer that we allocated and set above, > * or whoever used it should have freed it and allocated a new > * value that we need to free here */ > - slapi_pblock_get(pb, SLAPI_ORIGINAL_TARGET, &dn); > + ret = slapi_pblock_get(pb, SLAPI_ORIGINAL_TARGET, &dn); > + if (ret) { > + LOG_TRACE("Failed to get SLAPI_ORIGINAL_TARGET\n"); > + } > slapi_ch_free_string(&dn); > - slapi_pblock_set(pb, SLAPI_ORIGINAL_TARGET, NULL); > + ret = slapi_pblock_set(pb, SLAPI_ORIGINAL_TARGET, NULL); > + if (ret) { > + LOG_TRACE("Failed to clear SLAPI_ORIGINAL_TARGET\n"); > + } > slapi_ch_free_string(&authmethod); > slapi_ch_free_string(&principal); ACK. > https://fedorahosted.org/freeipa/ticket/2036 > --- > .../ipa-pwd-extop/ipa_pwd_extop.c | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c > index a0f9c5e14747b5c38952db27b005874a4afe1c4d..65c5834595f89aee8502347311f247be058c3416 100644 > --- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c > +++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c > @@ -1295,7 +1295,7 @@ int ipapwd_init( Slapi_PBlock *pb ) > if (!ret) ret = slapi_pblock_set(pb, SLAPI_PLUGIN_DESCRIPTION, (void *)&ipapwd_plugin_desc); > if (!ret) ret = slapi_pblock_set(pb, SLAPI_PLUGIN_EXT_OP_OIDLIST, ipapwd_oid_list); > if (!ret) ret = slapi_pblock_set(pb, SLAPI_PLUGIN_EXT_OP_NAMELIST, ipapwd_name_list); > - if (!ret) slapi_pblock_set(pb, SLAPI_PLUGIN_EXT_OP_FN, (void *)ipapwd_extop); > + if (!ret) ret = slapi_pblock_set(pb, SLAPI_PLUGIN_EXT_OP_FN, (void *)ipapwd_extop); > if (ret) { > LOG("Failed to set plug-in version, function, and OID.\n" ); > return -1; ACK. -- / Alexander Bokovoy From abokovoy at redhat.com Mon Nov 7 22:22:01 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 8 Nov 2011 00:22:01 +0200 Subject: [Freeipa-devel] [PATCHES] #2037 Coverity issues In-Reply-To: <1320343782.7734.677.camel@willson.li.ssimo.org> References: <1320343782.7734.677.camel@willson.li.ssimo.org> Message-ID: <20111107222200.GD16228@redhat.com> On Thu, 03 Nov 2011, Simo Sorce wrote: > These are actual issues. Most are resource leaks. > And one is a bad sizeof() computation that will cause us later to > overwrite out of bound memory, so potentially a segfault. Went through all of these. ACK. > https://fedorahosted.org/freeipa/ticket/2037 > --- > daemons/ipa-kdb/ipa_kdb.c | 13 +++++++------ > 1 files changed, 7 insertions(+), 6 deletions(-) > > diff --git a/daemons/ipa-kdb/ipa_kdb.c b/daemons/ipa-kdb/ipa_kdb.c > index 6a6c2063902f8b2a76d97f3510f09333c5af168d..481b1f392766498c5d7c6333fe73bafefde87dae 100644 > --- a/daemons/ipa-kdb/ipa_kdb.c > +++ b/daemons/ipa-kdb/ipa_kdb.c > @@ -263,6 +263,13 @@ int ipadb_get_connection(struct ipadb_context *ipactx) > > done: > ldap_msgfree(res); > + > + ldap_value_free_len(vals); > + for (i = 0; i < c && cvals[i]; i++) { > + free(cvals[i]); > + } > + free(cvals); > + > if (ret) { > if (ipactx->lcontext) { > ldap_unbind_ext_s(ipactx->lcontext, NULL, NULL); > @@ -274,12 +281,6 @@ done: > return EIO; > } > > - ldap_value_free_len(vals); > - for (i = 0; i < c; i++) { > - free(cvals[i]); > - } > - free(cvals); > - > return 0; > } ACK. > https://fedorahosted.org/freeipa/ticket/2037 > --- > daemons/ipa-kdb/ipa_kdb_passwords.c | 1 + > 1 files changed, 1 insertions(+), 0 deletions(-) > > diff --git a/daemons/ipa-kdb/ipa_kdb_passwords.c b/daemons/ipa-kdb/ipa_kdb_passwords.c > index 93e9e206081af412a472ab0c7624611a628a15b7..0bb7fa72496789241e27ecc852a1c6ede7f8e40a 100644 > --- a/daemons/ipa-kdb/ipa_kdb_passwords.c > +++ b/daemons/ipa-kdb/ipa_kdb_passwords.c > @@ -203,6 +203,7 @@ krb5_error_code ipadb_change_pwd(krb5_context context, > ret = asprintf(&ied->pw_policy_dn, > "cn=global_policy,%s", ipactx->realm_base); > if (ret == -1) { > + free(ied); > return ENOMEM; > } > db_entry->e_data = (krb5_octet *)ied; ACK. > https://fedorahosted.org/freeipa/ticket/2037 > --- > util/ipa_pwd.c | 4 +++- > 1 files changed, 3 insertions(+), 1 deletions(-) > > diff --git a/util/ipa_pwd.c b/util/ipa_pwd.c > index c41617533ec25e8e656e9bb7a69d5b8b5dd8b5f7..fda6cb34ef24059362207325db61aedb62d7b665 100644 > --- a/util/ipa_pwd.c > +++ b/util/ipa_pwd.c > @@ -560,7 +560,7 @@ int ipapwd_generate_new_history(char *password, > unsigned char *hash = NULL; > unsigned int hash_len; > char *new_element; > - char **ordered; > + char **ordered = NULL; > int c, i, n; > int len; > int ret; > @@ -626,9 +626,11 @@ int ipapwd_generate_new_history(char *password, > > *new_pwd_history = ordered; > *new_pwd_hlen = n; > + ordered = NULL; > ret = IPAPWD_POLICY_OK; > > done: > + free(ordered); > free(hash); > return ret; > } ACK > https://fedorahosted.org/freeipa/ticket/2037 > --- > daemons/ipa-kdb/ipa_kdb_principals.c | 7 +++++++ > 1 files changed, 7 insertions(+), 0 deletions(-) > > diff --git a/daemons/ipa-kdb/ipa_kdb_principals.c b/daemons/ipa-kdb/ipa_kdb_principals.c > index fdd834f355fd9e056058fa205b217e9e1f142e51..117eea86952ee4662930b80ba5e54c75aa587faf 100644 > --- a/daemons/ipa-kdb/ipa_kdb_principals.c > +++ b/daemons/ipa-kdb/ipa_kdb_principals.c > @@ -1571,6 +1571,7 @@ static krb5_error_code ipadb_entry_to_mods(krb5_context kcontext, > char **new_history; > int nh_len; > int ret; > + int i; > > ied = (struct ipadb_e_data *)entry->e_data; > if (ied->magic != IPA_E_DATA_MAGIC) { > @@ -1619,6 +1620,12 @@ static krb5_error_code ipadb_entry_to_mods(krb5_context kcontext, > > kerr = ipadb_get_ldap_mod_str_list(imods, "passwordHistory", > new_history, nh_len, mod_op); > + > + for (i = 0; i < nh_len; i++) { > + free(new_history[i]); > + } > + free(new_history); > + > if (kerr) { > goto done; > } ACK. > https://fedorahosted.org/freeipa/ticket/2037 > --- > daemons/ipa-kdb/ipa_kdb_principals.c | 1 + > 1 files changed, 1 insertions(+), 0 deletions(-) > > diff --git a/daemons/ipa-kdb/ipa_kdb_principals.c b/daemons/ipa-kdb/ipa_kdb_principals.c > index 117eea86952ee4662930b80ba5e54c75aa587faf..bb1356a01a27a639c15439187ffb8d3537c1fcec 100644 > --- a/daemons/ipa-kdb/ipa_kdb_principals.c > +++ b/daemons/ipa-kdb/ipa_kdb_principals.c > @@ -334,6 +334,7 @@ done: > free(keys[i].key_data_contents[0]); > free(keys[i].key_data_contents[1]); > } > + free(keys); > *result = NULL; > *num = 0; > } ACK. > https://fedorahosted.org/freeipa/ticket/2037 > --- > daemons/ipa-kdb/ipa_kdb_principals.c | 1 + > 1 files changed, 1 insertions(+), 0 deletions(-) > > diff --git a/daemons/ipa-kdb/ipa_kdb_principals.c b/daemons/ipa-kdb/ipa_kdb_principals.c > index bb1356a01a27a639c15439187ffb8d3537c1fcec..818ef033f5a085c0f60e479021f0964d81487704 100644 > --- a/daemons/ipa-kdb/ipa_kdb_principals.c > +++ b/daemons/ipa-kdb/ipa_kdb_principals.c > @@ -128,6 +128,7 @@ static int ipadb_ldap_attr_to_tl_data(LDAP *lcontext, LDAPMessage *le, > > done: > if (ret) { > + free(next); > if (*result) { > prev = *result; > while (prev) { ACK. > https://fedorahosted.org/freeipa/ticket/2037 > --- > daemons/ipa-kdb/ipa_kdb_principals.c | 4 ++-- > 1 files changed, 2 insertions(+), 2 deletions(-) > > diff --git a/daemons/ipa-kdb/ipa_kdb_principals.c b/daemons/ipa-kdb/ipa_kdb_principals.c > index 818ef033f5a085c0f60e479021f0964d81487704..33ed7b0e15ff1fa29150d1c2695c5a3b0c4c5f03 100644 > --- a/daemons/ipa-kdb/ipa_kdb_principals.c > +++ b/daemons/ipa-kdb/ipa_kdb_principals.c > @@ -554,6 +554,8 @@ static krb5_error_code ipadb_parse_ldap_entry(krb5_context kcontext, > } > ied->magic = IPA_E_DATA_MAGIC; > > + entry->e_data = (krb5_octet *)ied; > + > /* mark this as an ipa_user if it has the posixaccount objectclass */ > ret = ipadb_ldap_attr_has_value(lcontext, lentry, > "objectClass", "posixAccount"); > @@ -610,8 +612,6 @@ static krb5_error_code ipadb_parse_ldap_entry(krb5_context kcontext, > ied->last_pwd_change = restime; > } > > - entry->e_data = (krb5_octet *)ied; > - > kerr = 0; > > done: ACK. > https://fedorahosted.org/freeipa/ticket/2037 > --- > daemons/ipa-kdb/ipa_kdb_pwdpolicy.c | 5 ++++- > 1 files changed, 4 insertions(+), 1 deletions(-) > > diff --git a/daemons/ipa-kdb/ipa_kdb_pwdpolicy.c b/daemons/ipa-kdb/ipa_kdb_pwdpolicy.c > index 3dc4c218891e3ab2735eac1dcc721173657827f7..d439feb907eebda70b513ac9ca70f3e259ad3909 100644 > --- a/daemons/ipa-kdb/ipa_kdb_pwdpolicy.c > +++ b/daemons/ipa-kdb/ipa_kdb_pwdpolicy.c > @@ -52,7 +52,7 @@ krb5_error_code ipadb_get_pwd_policy(krb5_context kcontext, char *name, > krb5_error_code kerr; > LDAPMessage *res = NULL; > LDAPMessage *lentry; > - osa_policy_ent_t pentry; > + osa_policy_ent_t pentry = NULL; > uint32_t result; > int ret; > > @@ -150,6 +150,9 @@ krb5_error_code ipadb_get_pwd_policy(krb5_context kcontext, char *name, > *policy = pentry; > > done: > + if (kerr) { > + free(pentry); > + } > free(esc_name); > free(src_filter); > ldap_msgfree(res); ACK. > https://fedorahosted.org/freeipa/ticket/2037 > --- > daemons/ipa-kdb/ipa_kdb_pwdpolicy.c | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/daemons/ipa-kdb/ipa_kdb_pwdpolicy.c b/daemons/ipa-kdb/ipa_kdb_pwdpolicy.c > index d439feb907eebda70b513ac9ca70f3e259ad3909..46a0513307c859ff2cfef7ad58442edb1b9cc78d 100644 > --- a/daemons/ipa-kdb/ipa_kdb_pwdpolicy.c > +++ b/daemons/ipa-kdb/ipa_kdb_pwdpolicy.c > @@ -85,7 +85,7 @@ krb5_error_code ipadb_get_pwd_policy(krb5_context kcontext, char *name, > goto done; > } > > - pentry = calloc(1, sizeof(osa_policy_ent_t)); > + pentry = calloc(1, sizeof(osa_policy_ent_rec)); > if (!pentry) { > kerr = ENOMEM; > goto done; How this one has even worked? :) ACK. -- / Alexander Bokovoy From simo at redhat.com Mon Nov 7 22:30:41 2011 From: simo at redhat.com (Simo Sorce) Date: Mon, 07 Nov 2011 17:30:41 -0500 Subject: [Freeipa-devel] [PATCHES] #2036 Fix coverity bugs In-Reply-To: <20111107204203.GB16228@redhat.com> References: <1320343596.7734.675.camel@willson.li.ssimo.org> <20111107204203.GB16228@redhat.com> Message-ID: <1320705041.7734.792.camel@willson.li.ssimo.org> On Mon, 2011-11-07 at 22:42 +0200, Alexander Bokovoy wrote: > ACK to all below, they seem to be straight-forward. Thanks, pushed to master. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Mon Nov 7 22:31:05 2011 From: simo at redhat.com (Simo Sorce) Date: Mon, 07 Nov 2011 17:31:05 -0500 Subject: [Freeipa-devel] [PATCHES] #2037 Coverity issues In-Reply-To: <20111107222200.GD16228@redhat.com> References: <1320343782.7734.677.camel@willson.li.ssimo.org> <20111107222200.GD16228@redhat.com> Message-ID: <1320705065.7734.793.camel@willson.li.ssimo.org> On Tue, 2011-11-08 at 00:22 +0200, Alexander Bokovoy wrote: > Went through all of these. ACK. > Thanks again, pushed to master. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Mon Nov 7 22:31:45 2011 From: simo at redhat.com (Simo Sorce) Date: Mon, 07 Nov 2011 17:31:45 -0500 Subject: [Freeipa-devel] [PATCHES] #1791 Tust Effort: Add support for generating MS-PAC In-Reply-To: <20111104223150.GF2831@localhost.localdomain> References: <1320418180.7734.720.camel@willson.li.ssimo.org> <20111104223150.GF2831@localhost.localdomain> Message-ID: <1320705105.7734.794.camel@willson.li.ssimo.org> On Fri, 2011-11-04 at 23:31 +0100, Sumit Bose wrote: > On Fri, Nov 04, 2011 at 10:49:40AM -0400, Simo Sorce wrote: > > The attached patches are for master and concern the effort of creating > > trust relationships between IPA and AD domains. > > > > With these patches if you have run ipa-adtrust-install the IPA kdc will > > be able to create a MS-PAC if the user has the right attributes > > ipaNTSecurityIdentifier on the user entry and on the primary group entry > > are required (or a fallback primary group). > > If the objects are not in place the MS-PAC generation is silently > > skipped and no MS-PAC will be attached to the tickets. > > > > The MS-PAC is always generated if all data is available, in future we > > may think of making this conditional, but that is not in the scope of > > this patches. > > > > In order to apply these patches you need the coverity fix patches #2036 > > #2037 I sent yesterday. > > > > In order to build this code you need samba 4 experimental packages with > > the libndr_krb5pac.so librray, header files and pkgconfig configuration > > files. > > Please add these dependencies to the BuildRequires in the spec file. > Otherwise the patch looks fine. Added "BuildRequires: samba-4.0-devel", tested and pushed to master. Simo. -- Simo Sorce * Red Hat, Inc * New York From ayoung at redhat.com Tue Nov 8 02:24:38 2011 From: ayoung at redhat.com (Adam Young) Date: Mon, 07 Nov 2011 21:24:38 -0500 Subject: [Freeipa-devel] LDAPS for the IPA LDAP server? Message-ID: <4EB892E6.4070302@redhat.com> I noticed that the PKI Directory server has a secure port set but the IPA DS instance does not: PKI nsslapd-secureport: 7390 Why doesn IPA set up ldaps on port 636? From jcholast at redhat.com Tue Nov 8 09:19:08 2011 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 08 Nov 2011 10:19:08 +0100 Subject: [Freeipa-devel] LDAPS for the IPA LDAP server? In-Reply-To: <4EB892E6.4070302@redhat.com> References: <4EB892E6.4070302@redhat.com> Message-ID: <4EB8F40C.7090304@redhat.com> Dne 8.11.2011 03:24, Adam Young napsal(a): > I noticed that the PKI Directory server has a secure port set but the > IPA DS instance does not: > > PKI > nsslapd-secureport: 7390 > > Why doesn IPA set up ldaps on port 636? > I guess secure connections are set up using STARTTLS. Honza -- Jan Cholasta From sgallagh at redhat.com Tue Nov 8 13:18:44 2011 From: sgallagh at redhat.com (Stephen Gallagher) Date: Tue, 08 Nov 2011 08:18:44 -0500 Subject: [Freeipa-devel] LDAPS for the IPA LDAP server? In-Reply-To: <4EB892E6.4070302@redhat.com> References: <4EB892E6.4070302@redhat.com> Message-ID: <1320758324.2255.3.camel@sgallagh520.ipa.sgallagh.bos.redhat.com> On Mon, 2011-11-07 at 21:24 -0500, Adam Young wrote: > I noticed that the PKI Directory server has a secure port set but the > IPA DS instance does not: > > PKI > nsslapd-secureport: 7390 > > Why doesn IPA set up ldaps on port 636? I think you're confused. FreeIPA does indeed set up to listen on both 636 (LDAPS) and 389 (LDAP/TLS) by default. Take a look at 'netstat -lptn' as root. If you cannot connect to the LDAPS port, it may be due to a firewall issue or a certificate issue (make sure you have the FreeIPA CA cert loaded in /etc/openldap/cacerts and have called cacertdir_rehash on that directory) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From rcritten at redhat.com Tue Nov 8 13:43:25 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 08 Nov 2011 08:43:25 -0500 Subject: [Freeipa-devel] LDAPS for the IPA LDAP server? In-Reply-To: <1320758324.2255.3.camel@sgallagh520.ipa.sgallagh.bos.redhat.com> References: <4EB892E6.4070302@redhat.com> <1320758324.2255.3.camel@sgallagh520.ipa.sgallagh.bos.redhat.com> Message-ID: <4EB931FD.9020103@redhat.com> Stephen Gallagher wrote: > On Mon, 2011-11-07 at 21:24 -0500, Adam Young wrote: >> I noticed that the PKI Directory server has a secure port set but the >> IPA DS instance does not: >> >> PKI >> nsslapd-secureport: 7390 >> >> Why doesn IPA set up ldaps on port 636? > > > I think you're confused. FreeIPA does indeed set up to listen on both > 636 (LDAPS) and 389 (LDAP/TLS) by default. > > Take a look at 'netstat -lptn' as root. > > If you cannot connect to the LDAPS port, it may be due to a firewall > issue or a certificate issue (make sure you have the FreeIPA CA cert > loaded in /etc/openldap/cacerts and have called cacertdir_rehash on that > directory) Adam, are you looking in dse.ldif? I'm guessing that the default settings aren't written. It does appear in ldap: $ ldapsearch -LL -x -D 'cn=directory manager' -W -s base -b cn=config nsslapd-secureport Enter LDAP Password: version: 1 dn: cn=config nsslapd-secureport: 636 It isn't set in dse.ldif: # grep -c nsslapd-secureport /etc/dirsrv/slapd-EXAMPLE-COM/dse.ldif 0 rob From mkosek at redhat.com Tue Nov 8 15:14:12 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 08 Nov 2011 16:14:12 +0100 Subject: [Freeipa-devel] [PATCH] 161 Make ipa-server-install clean after itself Message-ID: <1320765254.2629.9.camel@balmora.brq.redhat.com> How to test: 1) ipa-server-install -p secret123 -a secret123 --hostname ipa.example.com 2) Continue in interactive wizard until IP address is requested (as ipa.example.com cannot be resolved) 3) When it is entered and ipa-server-install gives this output: # ipa-server-install -p kokos123 -a kokos123 --hostname ipa.example.com --setup-dns ... Please confirm the domain name [example.com]: Unable to resolve IP address for host name Please provide the IP address to be used for this host name: 10.16.78.93 Adding [10.16.78.93 ipa.example.com] to your /etc/hosts file <<<<< ... hit CTRL+C to quit from the installation. 4) Try running ipa-server-install again. It will fail as /etc/hosts has been changed - this patch fixes it. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-161-make-ipa-server-install-clean-after-itself.patch Type: text/x-patch Size: 5318 bytes Desc: not available URL: From ayoung at redhat.com Tue Nov 8 15:32:26 2011 From: ayoung at redhat.com (Adam Young) Date: Tue, 08 Nov 2011 10:32:26 -0500 Subject: [Freeipa-devel] LDAPS for the IPA LDAP server? In-Reply-To: <4EB931FD.9020103@redhat.com> References: <4EB892E6.4070302@redhat.com> <1320758324.2255.3.camel@sgallagh520.ipa.sgallagh.bos.redhat.com> <4EB931FD.9020103@redhat.com> Message-ID: <4EB94B8A.3080200@redhat.com> On 11/08/2011 08:43 AM, Rob Crittenden wrote: > Stephen Gallagher wrote: >> On Mon, 2011-11-07 at 21:24 -0500, Adam Young wrote: >>> I noticed that the PKI Directory server has a secure port set but the >>> IPA DS instance does not: >>> >>> PKI >>> nsslapd-secureport: 7390 >>> >>> Why doesn IPA set up ldaps on port 636? >> >> >> I think you're confused. FreeIPA does indeed set up to listen on both >> 636 (LDAPS) and 389 (LDAP/TLS) by default. >> >> Take a look at 'netstat -lptn' as root. >> >> If you cannot connect to the LDAPS port, it may be due to a firewall >> issue or a certificate issue (make sure you have the FreeIPA CA cert >> loaded in /etc/openldap/cacerts and have called cacertdir_rehash on that >> directory) > > Adam, are you looking in dse.ldif? I'm guessing that the default > settings aren't written. It does appear in ldap: Yes, I was. Thanks. > > $ ldapsearch -LL -x -D 'cn=directory manager' -W -s base -b cn=config > nsslapd-secureport > Enter LDAP Password: > version: 1 > > dn: cn=config > nsslapd-secureport: 636 > > It isn't set in dse.ldif: > > # grep -c nsslapd-secureport /etc/dirsrv/slapd-EXAMPLE-COM/dse.ldif > 0 > > rob > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From mkosek at redhat.com Tue Nov 8 17:07:06 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 08 Nov 2011 18:07:06 +0100 Subject: [Freeipa-devel] [PATCH] 162 Fix coverity issues in client CLI tools Message-ID: <1320772029.32755.1.camel@balmora.brq.redhat.com> This patch fixes 2 coverity issues: * ipa-client/config.c: CID 11090: Resource leak * ipa-client/ipa-getkeytab.c: CID 11018: Unchecked return value https://fedorahosted.org/freeipa/ticket/2035 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-162-fix-coverity-issues-in-client-cli-tools.patch Type: text/x-patch Size: 2875 bytes Desc: not available URL: From jzeleny at redhat.com Tue Nov 8 17:38:08 2011 From: jzeleny at redhat.com (Jan =?iso-8859-1?q?Zelen=FD?=) Date: Tue, 8 Nov 2011 18:38:08 +0100 Subject: [Freeipa-devel] OpenSSH integration - known_hosts Message-ID: <201111081838.13363.jzeleny@redhat.com> Hello everyone, there is a new effort in IPA and SSSD teams and that is SSH key integration in both parts of SSSD-IPA infrastructure. We've put together some basic plans and now we would like to know your opinion. Note that this is just shortened version to make it easier to read. It doesn't contain every bit of information about the design. For full version see https://fedorahosted.org/freeipa/wiki/SSH-FreeIPA-Integration Problems: ========= * the known_hosts file becomes outdated as machines get new host keys (e.g. re- installed systems in virtualized environment) * the user accepts any host key of the remote host without validating its authenticity Solution: ========= Instead of checking stale known_hosts file, provide a dynamic mechanism to lookup and deliver the public ssh key of the remote host to the client and use it for validation of the remote host identity. The dynamic mechanism would imply that no action is needed from the user because the source of the retrieved key is trusted. Limitations: ============ It is out of scope of this work to solve the problem in general. We propose a solution for following use case: Client host is a managed host meaning that it has SSSD installed and it is joined an IPA domain. It also has OpenSSH patched to interact with SSSD to get the information about the remote host Other UNIX machines or Windows machines as SSH clients are out of the scope of the current project. For the client hosts that can not be managed but can access IPA via the standard LDAP tools we will provide documentation on how to construct the content of the known_hosts file by querying LDAP server and saving the results. The remote host can be a managed (joined IPA domain via SSSD) or an unmanaged host. IPA server needs to provide a way to create entries for any managed and unmanaged hosts and store public keys for those hosts in that entries. What would change in IPA: ========================= * external host would have entries with the possibility of storing their public keys * new mechanism to work with keys through UI and CLI * host key fingerprints would be stored in SSHFP DNS records for each host joined in IPA domain What would change on the client: ================================ * SSSD would fetch and cache host public keys from IPA * joining to IPA domain would upload host public key * ssh client would communicate with SSSD, probably through ssh-agent, to check if the remote host is known It is still a question whether the solution is sufficient enough to address the needs and pains of the real deployments or other technologies outside the proposed should be used later (or instead). -- Thank you Jan Zeleny Red Hat Software Engineer Brno, Czech Republic -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part. URL: From jzeleny at redhat.com Tue Nov 8 17:39:27 2011 From: jzeleny at redhat.com (Jan =?iso-8859-1?q?Zelen=FD?=) Date: Tue, 8 Nov 2011 18:39:27 +0100 Subject: [Freeipa-devel] OpenSSH integration - authorized_keys Message-ID: <201111081839.27385.jzeleny@redhat.com> Hello everyone, this is a follow-up on the email on OpenSSH integration - known_host. It describes another scenario we want to address in the process of integrating OpenSSH to SSSD-IPA infrastructure - user public keys and their central management. As in the previous email, we would also like to know your opinion. Note that this is just shortened version to make it easier to read. It doesn't contain every bit of information about the design. For full version see https://fedorahosted.org/freeipa/wiki/SSH-FreeIPA-Integration Problems: ========= * how to distribute keys for new users / regenerated keys through the domain. The authorized_keys is probably not an option, we also want to cover use case when home directories are remote and not mounted on the server. * user may want to log on to the remote server using different account. We need to determine if he is allowed to impersonate that account Solution: ========= Similarly to openssh-lpk, the solution is to centrally manage and store user public keys in the IPA server and deliver them to the host for validation when user accesses that host. In the central server provide a way to define which account can do impersonation of which other accounts. Optionally add a way to represent special service accounts that are not full user accounts but can be logged as via ssh (stretch goal). What would change on IPA: ========================= * user entry will have additional multi-valued attribute for storing public keys. Unlike in openssh-lpk, this attribute will store what keys the user has, not who can impersonate him. * user entry would also have a multi-valued attribute containg DNs of users he can impersonate * new mechanisms to work with account public keys and impersonation via UI and CLI * HBAC rules would be extended to cover impersonation * provide an LDAP control to get a list of ssh keys that correspond to accounts that can impersonate a particular account in one operation. On the client side: =================== * SSSD would fetch (and cache?) user public keys from IPA * new SSSD client would fetch user public keys from SSSD * use SSH agent feature to get user public key from an output of the SSSD client -- Thank you Jan Zeleny Red Hat Software Engineer Brno, Czech Republic -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part. URL: From ayoung at redhat.com Tue Nov 8 18:10:51 2011 From: ayoung at redhat.com (Adam Young) Date: Tue, 08 Nov 2011 13:10:51 -0500 Subject: [Freeipa-devel] Tomcat Realms and Directory Server Message-ID: <4EB970AB.9080807@redhat.com> One issue I have been looking at recently is how to integrate PKI and IPA at the auth level while keeping a clean separation. We can extract the authentication from the servlet code, so it is purely a matter of configuring the Tomcat instance Realm. I wrote up a Proof of concept for just doing pure LDAP using simple bind, which is not a bad starting point. http://adam.younglogic.com/2011/11/tomcat-simple-ipa/ We want to continue this approach, but use a more secure authentication method. We won't be using basic auth, and we won't be using simple bind. There are two forms of authentication we want to support: Client Certificates and Kerberos. Certificates will work as they do now, and Kerberos will be for passing through user credentials from IPA, through HTTP to CS. In both cases, the data that backs it will be stored in the DS instance. Tomcat has a class classed a CombinedRealm: http://tomcat.apache.org/tomcat-6.0-doc/realm-howto.html#CombinedRealm That might support stacking Certificate and Kerberos auth on top of each other. The Realm will then delegate to LDAP for extracting the Roles. http://tomcat.apache.org/tomcat-6.0-doc/realm-howto.html#JAASRealm Kerberos is typically done using a JAAS Realm. I have to admit I don't really like the fact that we have to modify the JVM startup to do so, it is not really that big of a deal. I was also not a fan of setting the Realm up as a single service ticket until Simo informed me that the Browser NEGOTIATE mechanism assumes that the Service ticket is going to be Named HTTP. This means that for Proxied implementations, to include IPA, we will have to share the Service Principal Identity with the Apache HTTPD server. http://tomcat.apache.org/tomcat-6.0-doc/realm-howto.html#JAASRealm However, once you start digging in, you will find that the solutions are suboptimal. It turns out that the Negotiate auth-method has only very recently been supported, and that is only on Tomcat7. The best resource I have found on the options for a custom realm is here: http://wiki.wsmoak.net/cgi-bin/wiki.pl?TomcatKerberos and the most likely option http://wiki.wsmoak.net/cgi-bin/wiki.pl?TomcatKerberosLoginModule We really want a mix of the KRB5Login Module and the JNDIRealm. That seems to be what is described here: http://tomcat.apache.org/tomcat-7.0-doc/windows-auth-howto.html We should target Tomcat 7 for Dogtag:future. Fedora 16 ships with Tomcat-7. I suspect that the CombinedRealm approach will not support falling back from one auth-method to another: I've been looking and have not see it specified that you can put multiple auth-method entries inside a login-config in the web.xml. Ideally, we would attempt a Certificate based authentication first, and then fall back to Negotiate. However, we can say that a given deployment is going to be either Kerberos or Client Certificate, and swap out the configuration at the Tomcat level. I don't like that nearly as much. The document here: http://wiki.apache.org/tomcat/SSLWithFORMFallback talks about how to do Client Cert with a fallback to Form based authentication. We'd want to do Client Cert with a fallback to Krb5. This is using what is called a Valve. In Tomcat 6 and 7 valves have been deprecated in favor of Filters. The general approach is the same. I'd like to keep the idea of the Realm as the primary approach. If we do have to build a custom Realm, and I suspect that we will, we might want to spin it off into its own package, or submit it for Inclusion in Tomcat 7 upstream. It seems that the PKI approach has been to Bind as Directory Manager. What I would like to target moving forward is that the Bind is done as the user making the web request. For managed operations that require a higher level of authentication, we use the concept of queues like we do now. The threads that manage those queue will use a Principal with a higher level of authorization: not "Directory Manger", but perhaps "CA Manager" which is a user we create that manages the CA subtree in the Directory server. For a Dogtag deployment without IPA, the CA Manger would have write privileged on the Identity subtree. For integrated deployments, IPA would have its own principal "IPA Manager" that would not have read or write capabilities in the CA Subtree. DRM, TKS and other subsytems would in turn also get their own Manager users, and they would only have permissions to manage their own trees: we will need to clear up which gets read and write permissions on which other subtrees. Directory Manager would be limited to performing operations that effect the entire DS instance: Setup and Replication. The Directory Manager, CA Manager, IPA manager users should be binding with a certificate or a keytab, not with a cleartext password. From ohamada at redhat.com Tue Nov 8 19:41:44 2011 From: ohamada at redhat.com (Ondrej Hamada) Date: Tue, 08 Nov 2011 20:41:44 +0100 Subject: [Freeipa-devel] [PATCH] the 'Keytab:' field in "ipa user-show" output is misleading Message-ID: <4EB985F8.6060107@redhat.com> https://fedorahosted.org/freeipa/ticket/1961 The 'Keytab' filed in output of all 'user-*' commands was changed to 'Kerberos keys available'. In order to do this change for 'user-*' commands only, the flag 'has_keytab' had to be removed from common output parametrs in ipalib/baseldap.py. This change also affected the host.py and service.py, where the 'has_keytab' flag was added to their local output params. Both host.py and service.py holds the old field caption - 'Keytab' - because of compatibility with older clients. -- Regards, Ondrej Hamada FreeIPA team jabber:ohama at jabbim.cz IRC: ohamada -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-PATCH-the-Keytab-field-in-ipa-user-show-output-is-mi.patch Type: text/x-patch Size: 4671 bytes Desc: not available URL: From rcritten at redhat.com Tue Nov 8 19:49:06 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 08 Nov 2011 14:49:06 -0500 Subject: [Freeipa-devel] [PATCH] 901 remove has_upg() and has_managed_entries() Message-ID: <4EB987B2.1040105@redhat.com> These functions are leftovers from when the managed entries plugin was being developed and not widely available. They are no longer needed. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-901-managed.patch Type: text/x-patch Size: 5231 bytes Desc: not available URL: From danieljamesscott at gmail.com Tue Nov 8 19:56:45 2011 From: danieljamesscott at gmail.com (Dan Scott) Date: Tue, 8 Nov 2011 14:56:45 -0500 Subject: [Freeipa-devel] [Freeipa-users] OpenSSH integration - known_hosts In-Reply-To: <201111081838.13363.jzeleny@redhat.com> References: <201111081838.13363.jzeleny@redhat.com> Message-ID: Hi, This is a great feature. It feels like I'm always re-installing VMs and having to remove old SSH keys and re-accept new ones. One feature I'd like is to have this working cross-realm. We have 2 IPA realms here and it would be great if I could configure SSSD to check the local realm if I'm SSHing to a local PC and to check the other IPA server(s) if my SSH target is part of the other realm. Even better if it could do this without explicit configuration. Do you think it would be possible to do this securely? Dan On Tue, Nov 8, 2011 at 12:38, Jan Zelen? wrote: > Hello everyone, > there is a new effort in IPA and SSSD teams and that is SSH key integration in > both parts of SSSD-IPA infrastructure. We've put together some basic plans and > now we would like to know your opinion. > > Note that this is just shortened version to make it easier to read. It doesn't > contain every bit of information about the design. For full version see > https://fedorahosted.org/freeipa/wiki/SSH-FreeIPA-Integration > > Problems: > ========= > * the known_hosts file becomes outdated as machines get new host keys (e.g. re- > installed systems in virtualized environment) > * the user accepts any host key of the remote host without validating its > authenticity > > > Solution: > ========= > Instead of checking stale known_hosts file, provide a dynamic mechanism to > lookup and deliver the public ssh key of the remote host to the client and use > it for validation of the remote host identity. The dynamic mechanism would > imply that no action is needed from the user because the source of the > retrieved key is trusted. > > > Limitations: > ============ > It is out of scope of this work to solve the problem in general. We propose a > solution for following use case: > > Client host is a managed host meaning that it has SSSD installed and it is > joined an IPA domain. It also has OpenSSH patched to interact with SSSD to get > the information about the remote host > > Other UNIX machines or Windows machines as SSH clients are out of the scope of > the current project. For the client hosts that can not be managed but can > access IPA via the standard LDAP tools we will provide documentation on how to > construct the content of the known_hosts file by querying LDAP server and > saving the results. > > The remote host can be a managed (joined IPA domain via SSSD) or an unmanaged > host. IPA server needs to provide a way to create entries for any managed and > unmanaged hosts and store public keys for those hosts in that entries. > > What would change in IPA: > ========================= > * external host would have entries with the possibility of storing their > public keys > * new mechanism to work with keys through UI and CLI > * host key fingerprints would be stored in SSHFP DNS records for each host > joined in IPA domain > > What would change on the client: > ================================ > * SSSD would fetch and cache host public keys from IPA > * joining to IPA domain would upload host public key > * ssh client would communicate with SSSD, probably through ssh-agent, to check > if the remote host is known > > It is still a question whether the solution is sufficient enough to address the > needs and pains of the real deployments or other technologies outside the > proposed should be used later (or instead). > > -- > Thank you > Jan Zeleny > > Red Hat Software Engineer > Brno, Czech Republic > > _______________________________________________ > Freeipa-users mailing list > Freeipa-users at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-users > From rcritten at redhat.com Tue Nov 8 22:05:46 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 08 Nov 2011 17:05:46 -0500 Subject: [Freeipa-devel] [PATCH] 902 Don't allow empty default object classes Message-ID: <4EB9A7BA.2000906@redhat.com> Don't allow one to set a blank list of default objectclasses in cn=ipaconfig. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-902-config.patch Type: text/x-patch Size: 1133 bytes Desc: not available URL: From dpal at redhat.com Tue Nov 8 22:57:20 2011 From: dpal at redhat.com (Dmitri Pal) Date: Tue, 08 Nov 2011 17:57:20 -0500 Subject: [Freeipa-devel] [Freeipa-users] OpenSSH integration - known_hosts In-Reply-To: References: <201111081838.13363.jzeleny@redhat.com> Message-ID: <4EB9B3D0.9080904@redhat.com> On 11/08/2011 02:56 PM, Dan Scott wrote: > Hi, > > This is a great feature. It feels like I'm always re-installing VMs > and having to remove old SSH keys and re-accept new ones. > > One feature I'd like is to have this working cross-realm. We have 2 > IPA realms here and it would be great if I could configure SSSD to > check the local realm if I'm SSHing to a local PC and to check the > other IPA server(s) if my SSH target is part of the other realm. Even > better if it could do this without explicit configuration. > > Do you think it would be possible to do this securely? When we start to support Cross Realm Kerberos Trusts for IPA to IPA I think this would be doable but then I do not think the ssh host keys will be used (needed). Simo, am I correct? > Dan > > On Tue, Nov 8, 2011 at 12:38, Jan Zelen? wrote: >> Hello everyone, >> there is a new effort in IPA and SSSD teams and that is SSH key integration in >> both parts of SSSD-IPA infrastructure. We've put together some basic plans and >> now we would like to know your opinion. >> >> Note that this is just shortened version to make it easier to read. It doesn't >> contain every bit of information about the design. For full version see >> https://fedorahosted.org/freeipa/wiki/SSH-FreeIPA-Integration >> >> Problems: >> ========= >> * the known_hosts file becomes outdated as machines get new host keys (e.g. re- >> installed systems in virtualized environment) >> * the user accepts any host key of the remote host without validating its >> authenticity >> >> >> Solution: >> ========= >> Instead of checking stale known_hosts file, provide a dynamic mechanism to >> lookup and deliver the public ssh key of the remote host to the client and use >> it for validation of the remote host identity. The dynamic mechanism would >> imply that no action is needed from the user because the source of the >> retrieved key is trusted. >> >> >> Limitations: >> ============ >> It is out of scope of this work to solve the problem in general. We propose a >> solution for following use case: >> >> Client host is a managed host meaning that it has SSSD installed and it is >> joined an IPA domain. It also has OpenSSH patched to interact with SSSD to get >> the information about the remote host >> >> Other UNIX machines or Windows machines as SSH clients are out of the scope of >> the current project. For the client hosts that can not be managed but can >> access IPA via the standard LDAP tools we will provide documentation on how to >> construct the content of the known_hosts file by querying LDAP server and >> saving the results. >> >> The remote host can be a managed (joined IPA domain via SSSD) or an unmanaged >> host. IPA server needs to provide a way to create entries for any managed and >> unmanaged hosts and store public keys for those hosts in that entries. >> >> What would change in IPA: >> ========================= >> * external host would have entries with the possibility of storing their >> public keys >> * new mechanism to work with keys through UI and CLI >> * host key fingerprints would be stored in SSHFP DNS records for each host >> joined in IPA domain >> >> What would change on the client: >> ================================ >> * SSSD would fetch and cache host public keys from IPA >> * joining to IPA domain would upload host public key >> * ssh client would communicate with SSSD, probably through ssh-agent, to check >> if the remote host is known >> >> It is still a question whether the solution is sufficient enough to address the >> needs and pains of the real deployments or other technologies outside the >> proposed should be used later (or instead). >> >> -- >> Thank you >> Jan Zeleny >> >> Red Hat Software Engineer >> Brno, Czech Republic >> >> _______________________________________________ >> Freeipa-users mailing list >> Freeipa-users at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-users >> > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Tue Nov 8 23:33:41 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 08 Nov 2011 18:33:41 -0500 Subject: [Freeipa-devel] [PATCH] 903 normalize paths in certmonger Message-ID: <4EB9BC55.1010706@redhat.com> There are times we need to hunt through the certmonger request files trying (such as trying to stop tracking a cert). One criteria is the cert database and they need to match exactly. We weren't normalizing this so something as simple as a trailing slash would cause a match to fail. Normalize both values to address this. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-903-certmonger.patch Type: text/x-patch Size: 3751 bytes Desc: not available URL: From simo at redhat.com Tue Nov 8 23:35:10 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 08 Nov 2011 18:35:10 -0500 Subject: [Freeipa-devel] [Freeipa-users] OpenSSH integration - known_hosts In-Reply-To: <4EB9B3D0.9080904@redhat.com> References: <201111081838.13363.jzeleny@redhat.com> <4EB9B3D0.9080904@redhat.com> Message-ID: <1320795310.7734.853.camel@willson.li.ssimo.org> On Tue, 2011-11-08 at 17:57 -0500, Dmitri Pal wrote: > On 11/08/2011 02:56 PM, Dan Scott wrote: > > Hi, > > > > This is a great feature. It feels like I'm always re-installing VMs > > and having to remove old SSH keys and re-accept new ones. > > > > One feature I'd like is to have this working cross-realm. We have 2 > > IPA realms here and it would be great if I could configure SSSD to > > check the local realm if I'm SSHing to a local PC and to check the > > other IPA server(s) if my SSH target is part of the other realm. Even > > better if it could do this without explicit configuration. > > > > Do you think it would be possible to do this securely? > > When we start to support Cross Realm Kerberos Trusts for IPA to IPA I > think this would be doable but then I do not think the ssh host keys > will be used (needed). Simo, am I correct? We do not have the GSSAPI key exchange patches in OpenSSH. With those the ssh host key is not necessary when using GSSAPI auth, even in the same realm. But when you want to use ssh host keys, across realm kerberos trust is not going to help. In order to validate keys from different realms I guess we could use DNSSEC where the signatures of one realm are trusted by the other. Then by storing ssh host keys as DNS fields a different domain could still trust those keys. This works only for enrolled hosts though, I guess. Or at least only for hosts in DNS domains that are controlled by IPA. For hosts in other DNS domains we cannot distribute keys through DNS. If that is necessary then we would have to define some sort of protocol to allow fetching of keys from one domain to the other. We could use a mechanism similar to what we will need to implement for sid2name resolution for windows domain trusts. Where the IPA server becomes a proxy to request host keys from other domains. Bottom line, we can come up with something but it is not scoped yet. And needs some more thinking so that we put in place something that scales well. Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Tue Nov 8 23:46:19 2011 From: dpal at redhat.com (Dmitri Pal) Date: Tue, 08 Nov 2011 18:46:19 -0500 Subject: [Freeipa-devel] [Freeipa-users] OpenSSH integration - known_hosts In-Reply-To: <1320795310.7734.853.camel@willson.li.ssimo.org> References: <201111081838.13363.jzeleny@redhat.com> <4EB9B3D0.9080904@redhat.com> <1320795310.7734.853.camel@willson.li.ssimo.org> Message-ID: <4EB9BF4B.3050604@redhat.com> On 11/08/2011 06:35 PM, Simo Sorce wrote: > On Tue, 2011-11-08 at 17:57 -0500, Dmitri Pal wrote: >> On 11/08/2011 02:56 PM, Dan Scott wrote: >>> Hi, >>> >>> This is a great feature. It feels like I'm always re-installing VMs >>> and having to remove old SSH keys and re-accept new ones. >>> >>> One feature I'd like is to have this working cross-realm. We have 2 >>> IPA realms here and it would be great if I could configure SSSD to >>> check the local realm if I'm SSHing to a local PC and to check the >>> other IPA server(s) if my SSH target is part of the other realm. Even >>> better if it could do this without explicit configuration. >>> >>> Do you think it would be possible to do this securely? >> When we start to support Cross Realm Kerberos Trusts for IPA to IPA I >> think this would be doable but then I do not think the ssh host keys >> will be used (needed). Simo, am I correct? > We do not have the GSSAPI key exchange patches in OpenSSH. With those > the ssh host key is not necessary when using GSSAPI auth, even in the > same realm. > > But when you want to use ssh host keys, across realm kerberos trust is > not going to help. > > In order to validate keys from different realms I guess we could use > DNSSEC where the signatures of one realm are trusted by the other. > Then by storing ssh host keys as DNS fields a different domain could > still trust those keys. This works only for enrolled hosts though, I > guess. Or at least only for hosts in DNS domains that are controlled by > IPA. For hosts in other DNS domains we cannot distribute keys through > DNS. > If that is necessary then we would have to define some sort of protocol > to allow fetching of keys from one domain to the other. > We could use a mechanism similar to what we will need to implement for > sid2name resolution for windows domain trusts. Where the IPA server > becomes a proxy to request host keys from other domains. > > Bottom line, we can come up with something but it is not scoped yet. And > needs some more thinking so that we put in place something that scales > well. > > Simo. > Ok: https://fedorahosted.org/freeipa/ticket/2081 -- Thank you, Dmitri Pal Sr. Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From abokovoy at redhat.com Wed Nov 9 07:23:30 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 9 Nov 2011 09:23:30 +0200 Subject: [Freeipa-devel] [PATCH] 902 Don't allow empty default object classes In-Reply-To: <4EB9A7BA.2000906@redhat.com> References: <4EB9A7BA.2000906@redhat.com> Message-ID: <20111109072330.GA31855@redhat.com> On Tue, 08 Nov 2011, Rob Crittenden wrote: > Don't allow one to set a blank list of default objectclasses in > cn=ipaconfig. > ACK -- / Alexander Bokovoy From mkosek at redhat.com Wed Nov 9 07:51:20 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 09 Nov 2011 08:51:20 +0100 Subject: [Freeipa-devel] [PATCH] 902 Don't allow empty default object classes In-Reply-To: <20111109072330.GA31855@redhat.com> References: <4EB9A7BA.2000906@redhat.com> <20111109072330.GA31855@redhat.com> Message-ID: <1320825082.11381.2.camel@balmora.brq.redhat.com> On Wed, 2011-11-09 at 09:23 +0200, Alexander Bokovoy wrote: > On Tue, 08 Nov 2011, Rob Crittenden wrote: > > > Don't allow one to set a blank list of default objectclasses in > > cn=ipaconfig. > > > ACK > Pushed to master, ipa-2-1. Martin From jcholast at redhat.com Wed Nov 9 10:24:17 2011 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 09 Nov 2011 11:24:17 +0100 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <20111103114352.GB26667@redhat.com> References: <4EAE8416.4060401@redhat.com> <20111031121904.GA9412@redhat.com> <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <4EB152A7.5010303@redhat.com> <20111102151114.GD14373@redhat.com> <4EB15E73.8070402@redhat.com> <20111102163720.GA23092@redhat.com> <20111102163848.GB23092@redhat.com> <4EB2614E.8090307@redhat.com> <20111103114352.GB26667@redhat.com> Message-ID: <4EBA54D1.4040700@redhat.com> Dne 3.11.2011 12:43, Alexander Bokovoy napsal(a): > On Thu, 03 Nov 2011, Jan Cholasta wrote: >> The problem with the API class is that it creates all the plugin >> instances every time. Both mine and your patch don't change that, >> they deal only with finalization, which is the easier part of the >> lazy initialization problem. Additionally, in your patch finalize() >> is called only on Command subclasses, so non-Command plugins are not >> ReadOnly-locked like they should (currently this is done only when >> production mode is off). > I ended up with explicitly finalizing Object, Property, and Backend > objects. This, by the way, proved that major slowdown is brought by > the Command and Method instances as I've got statistically negligible > variations of execution time, within less than 1%. > >> We could change API so that the plugins are initialized on-demand. >> That way we could probably get rid off finalize() on plugins >> altogether and move the initialization code into __init__() (because >> all the required information would be available on-demand) and the >> locking into API. The flaw of this approach is that it would require >> a number of fundamental changes, namely changing the way API class >> handles plugin initialization and moving the initialization of >> "static" Plugin properties (name, module, fullname, bases, label, >> ...) from instance level to class level. (Nonetheless, I think API >> would't suffer from a facelift - currently it is messy, hard to read >> code, with bits nobody ever used or uses anymore, some of the >> docstrings and comments aren't correct, etc.) > A review of API class is a good idea. However, I think it is currently > enough what is in proposed patch as it gets you 2-3x speed increase > already. I've already started experimenting with the API class, hopefully it will result in something useful :) > > One important feature I'd like to still keep in FreeIPA is ability to > extend any object and action during plugin import phase regardless of > the python source file it comes from. So far our split between 'user > methods' and 'dnsrecord methods' is mostly utilitarian as they are > implemented in their own source code files. However, nobody prevents > you from implementing all needed modifications to all classes in the > single source file and I expect this to be fairly typical to > site-specific cases. > > This is something I'd like to keep as it has great value for all > end-users and it requires loading all Python files in ipalib/plugins > (and in ipaserver/plugins for server side). I'm not suggesting we should skip importing the modules. The plugin initialization process consists of these 3 steps: 1. import plugin modules 2. create plugin instances 3. finalize plugin instances What I'm suggesting is that we do steps 2 and 3 on-demand. We can't do step 1 on-demand, because we have no way of knowing what plugins are available without importing the modules. (Both your and my patch does only step 3 on-demand, but I think we can do better than that.) > >> Anyway, if we decide to go with your solution, please make sure >> *all* the plugins that are used are finalized (because of the >> locking) and add a new api.env attribute which controls the lazy >> finalization, so that the behavior can be overriden (actually I have >> already done that - see attached patch - just use >> "api.env.plugins_on_demand" instead of "api.env.context == 'cli'"). > Done. > + if not self.__dict__['_Plugin__finalized']: + self.finalize() This doesn't seem right, coding style-wise. If you want to access the property from outside the Plugin class, why did you name-mangle it in the first place? def finalize(self): """ """ + if not self.__finalized: + self.__finalized = True + else: + return if not is_production_mode(self): lock(self) Finalize is supposed to be called just once, so IMO it would be better not to do the check and let it throw the exception. + for i in ('Object', 'Property', 'Backend'): + if i in self: + namespaces.append(self[i]) AFAIK the framework is supposed to be generally usable for other projects in the future. With that in mind, I think we can't afford to hard-code things like this. Anyway, I've made a patch that uses data descriptors for the trap objects (which is IMHO more elegant than what you do). I've also managed to add on-demand finalization to Object and Attribute subclasses by moving the set-up code from set_api() to finalize() (it doesn't add much performance, though). See attachment. The numbers from my VM: master abbra jcholast $ time ipa real 0m1.288s 0m0.619s 0m0.601s user 0m1.103s 0m0.478s 0m0.451s sys 0m0.161s 0m0.126s 0m0.133s $ time ipa user-find real 0m1.897s 0m1.253s 0m1.235s user 0m1.119s 0m0.486s 0m0.488s sys 0m0.160s 0m0.160s 0m0.136s $ time ipa help real 0m1.299s 0m0.629s 0m0.600s user 0m1.094s 0m0.477s 0m0.446s sys 0m0.183s 0m0.136s 0m0.140s Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-57.1-lazy-finalize.patch Type: text/x-patch Size: 13780 bytes Desc: not available URL: From abokovoy at redhat.com Wed Nov 9 10:59:11 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 9 Nov 2011 12:59:11 +0200 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <4EBA54D1.4040700@redhat.com> References: <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <4EB152A7.5010303@redhat.com> <20111102151114.GD14373@redhat.com> <4EB15E73.8070402@redhat.com> <20111102163720.GA23092@redhat.com> <20111102163848.GB23092@redhat.com> <4EB2614E.8090307@redhat.com> <20111103114352.GB26667@redhat.com> <4EBA54D1.4040700@redhat.com> Message-ID: <20111109105904.GA5529@redhat.com> On Wed, 09 Nov 2011, Jan Cholasta wrote: > >>would't suffer from a facelift - currently it is messy, hard to read > >>code, with bits nobody ever used or uses anymore, some of the > >>docstrings and comments aren't correct, etc.) > >A review of API class is a good idea. However, I think it is currently > >enough what is in proposed patch as it gets you 2-3x speed increase > >already. > > I've already started experimenting with the API class, hopefully it > will result in something useful :) Good. > >This is something I'd like to keep as it has great value for all > >end-users and it requires loading all Python files in ipalib/plugins > >(and in ipaserver/plugins for server side). > > I'm not suggesting we should skip importing the modules. The plugin > initialization process consists of these 3 steps: > > 1. import plugin modules > 2. create plugin instances > 3. finalize plugin instances > > What I'm suggesting is that we do steps 2 and 3 on-demand. We can't > do step 1 on-demand, because we have no way of knowing what plugins > are available without importing the modules. (Both your and my patch > does only step 3 on-demand, but I think we can do better than that.) Agreed. > >>Anyway, if we decide to go with your solution, please make sure > >>*all* the plugins that are used are finalized (because of the > >>locking) and add a new api.env attribute which controls the lazy > >>finalization, so that the behavior can be overriden (actually I have > >>already done that - see attached patch - just use > >>"api.env.plugins_on_demand" instead of "api.env.context == 'cli'"). > >Done. > > > > + if not self.__dict__['_Plugin__finalized']: > + self.finalize() > > This doesn't seem right, coding style-wise. If you want to access > the property from outside the Plugin class, why did you name-mangle > it in the first place? It is supposed to be internal detail of a Plugin. I agree it stinks a bit here. :) > def finalize(self): > """ > """ > + if not self.__finalized: > + self.__finalized = True > + else: > + return > if not is_production_mode(self): > lock(self) > > Finalize is supposed to be called just once, so IMO it would be > better not to do the check and let it throw the exception. On contrary, I think sequential finalize() calls should do nothing. This is at least what I expect from a finalized object -- no-op. > + for i in ('Object', 'Property', 'Backend'): > + if i in self: > + namespaces.append(self[i]) > > AFAIK the framework is supposed to be generally usable for other > projects in the future. With that in mind, I think we can't afford > to hard-code things like this. That's true. As you managed to get around without hardcoding, we can drop this part. > Anyway, I've made a patch that uses data descriptors for the trap > objects (which is IMHO more elegant than what you do). I've also > managed to add on-demand finalization to Object and Attribute > subclasses by moving the set-up code from set_api() to finalize() > (it doesn't add much performance, though). See attachment. I read through the code and I like it! Did you get the whole test suite running with it? There are some parts of tests that expect non-finalized objects to have None properties while they are now not None. If so, preliminary ACK for your patch from reading it if you would make sure on_finalize() allows multiple calls and would make a better commit message. ;) I'll try to arrange testing myself today/tomorrow. > The numbers from my VM: > > master abbra jcholast > $ time ipa > real 0m1.288s 0m0.619s 0m0.601s > user 0m1.103s 0m0.478s 0m0.451s > sys 0m0.161s 0m0.126s 0m0.133s > > $ time ipa user-find > real 0m1.897s 0m1.253s 0m1.235s > user 0m1.119s 0m0.486s 0m0.488s > sys 0m0.160s 0m0.160s 0m0.136s > > $ time ipa help > real 0m1.299s 0m0.629s 0m0.600s > user 0m1.094s 0m0.477s 0m0.446s > sys 0m0.183s 0m0.136s 0m0.140s Looks good (your VM is supposedly at the same farm as mine so numbers are comparable). There is anyway great variation in execution times in VMs so 600ms vs 629ms is roughly the same. Thanks a lot! I think it is great advance over the original approach. -- / Alexander Bokovoy From mkosek at redhat.com Wed Nov 9 13:42:01 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 09 Nov 2011 14:42:01 +0100 Subject: [Freeipa-devel] [PATCH] 163+164 Fix DNS zone --allow-dynupdate option behavior Message-ID: <1320846124.11381.10.camel@balmora.brq.redhat.com> Patch 164 fixes dnszone-mod --allow-dynupdate behavior which disables the zone dynamic updates value whenever a dnszone-mod command is run and --allow-dynupdate options is not set. I introduced a Param.encode() function (patch 163) to our framework to help encoding Python native values to LDAP representation more effectively. True/False to LDAP's "TRUE"/"FALSE" in this case. Encoding functions are executed in a server context only. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-163-allow-custom-server-backend-encoding.patch Type: text/x-patch Size: 4768 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-164-fix-dns-zone-allow-dynupdate-option-behavior.patch Type: text/x-patch Size: 8982 bytes Desc: not available URL: From simo at redhat.com Wed Nov 9 13:55:28 2011 From: simo at redhat.com (Simo Sorce) Date: Wed, 09 Nov 2011 08:55:28 -0500 Subject: [Freeipa-devel] [Freeipa-users] OpenSSH integration - known_hosts In-Reply-To: References: <201111081838.13363.jzeleny@redhat.com> <4EB9B3D0.9080904@redhat.com> <1320795310.7734.853.camel@willson.li.ssimo.org> Message-ID: <1320846928.7734.859.camel@willson.li.ssimo.org> On Tue, 2011-11-08 at 20:45 -0500, Dan Scott wrote: > Hi, > > On Tue, Nov 8, 2011 at 18:35, Simo Sorce wrote: > > On Tue, 2011-11-08 at 17:57 -0500, Dmitri Pal wrote: > >> On 11/08/2011 02:56 PM, Dan Scott wrote: > >> > Hi, > >> > > >> > This is a great feature. It feels like I'm always re-installing VMs > >> > and having to remove old SSH keys and re-accept new ones. > >> > > >> > One feature I'd like is to have this working cross-realm. We have 2 > >> > IPA realms here and it would be great if I could configure SSSD to > >> > check the local realm if I'm SSHing to a local PC and to check the > >> > other IPA server(s) if my SSH target is part of the other realm. Even > >> > better if it could do this without explicit configuration. > >> > > >> > Do you think it would be possible to do this securely? > >> > >> When we start to support Cross Realm Kerberos Trusts for IPA to IPA I > >> think this would be doable but then I do not think the ssh host keys > >> will be used (needed). Simo, am I correct? > > > > We do not have the GSSAPI key exchange patches in OpenSSH. With those > > the ssh host key is not necessary when using GSSAPI auth, even in the > > same realm. > > > > But when you want to use ssh host keys, across realm kerberos trust is > > not going to help. > > I don't quite understand this. What trust is required, other than the > cross-realm authentication of kerberos tickets? Surely each realm > would manage its own host keys. All I'm looking for is an > authenticated cross-realm key lookup so that my client can pre-cache > entries in the known_hosts file. Wouldn't this just be an LDAP lookup? Well in 2-way trusts you could do that. But in general you do not want to have any client in realm1 to contact any server in realm2. They might be geographically very far and use high latency/low bandwidth links. So, for a first implementation, we could do what you say, but I rather think it through and see how to cache/transfer information making clients go through their IPA server rather than trying to connect directly to a remote one. Simo. -- Simo Sorce * Red Hat, Inc * New York From abokovoy at redhat.com Wed Nov 9 14:14:02 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 9 Nov 2011 16:14:02 +0200 Subject: [Freeipa-devel] [PATCH] 163+164 Fix DNS zone --allow-dynupdate option behavior In-Reply-To: <1320846124.11381.10.camel@balmora.brq.redhat.com> References: <1320846124.11381.10.camel@balmora.brq.redhat.com> Message-ID: <20111109141401.GA8188@redhat.com> On Wed, 09 Nov 2011, Martin Kosek wrote: > Patch 164 fixes dnszone-mod --allow-dynupdate behavior which disables > the zone dynamic updates value whenever a dnszone-mod command is run and > --allow-dynupdate options is not set. As discussed on IRC, please rename the CLI option name to --dynamic-update. Otherwise, ACK. Please also file a ticket for 3.0 for gathering all renames of CLI options where we feel they are unclear or unfriendly. > I introduced a Param.encode() function (patch 163) to our framework to > help encoding Python native values to LDAP representation more > effectively. True/False to LDAP's "TRUE"/"FALSE" in this case. Encoding > functions are executed in a server context only. Ack. -- / Alexander Bokovoy From danieljamesscott at gmail.com Wed Nov 9 01:45:29 2011 From: danieljamesscott at gmail.com (Dan Scott) Date: Tue, 8 Nov 2011 20:45:29 -0500 Subject: [Freeipa-devel] [Freeipa-users] OpenSSH integration - known_hosts In-Reply-To: <1320795310.7734.853.camel@willson.li.ssimo.org> References: <201111081838.13363.jzeleny@redhat.com> <4EB9B3D0.9080904@redhat.com> <1320795310.7734.853.camel@willson.li.ssimo.org> Message-ID: Hi, On Tue, Nov 8, 2011 at 18:35, Simo Sorce wrote: > On Tue, 2011-11-08 at 17:57 -0500, Dmitri Pal wrote: >> On 11/08/2011 02:56 PM, Dan Scott wrote: >> > Hi, >> > >> > This is a great feature. It feels like I'm always re-installing VMs >> > and having to remove old SSH keys and re-accept new ones. >> > >> > One feature I'd like is to have this working cross-realm. We have 2 >> > IPA realms here and it would be great if I could configure SSSD to >> > check the local realm if I'm SSHing to a local PC and to check the >> > other IPA server(s) if my SSH target is part of the other realm. Even >> > better if it could do this without explicit configuration. >> > >> > Do you think it would be possible to do this securely? >> >> When we start to support Cross Realm Kerberos Trusts for IPA to IPA I >> think this would be doable but then I do not think the ssh host keys >> will be used (needed). Simo, am I correct? > > We do not have the GSSAPI key exchange patches in OpenSSH. With those > the ssh host key is not necessary when using GSSAPI auth, even in the > same realm. > > But when you want to use ssh host keys, across realm kerberos trust is > not going to help. I don't quite understand this. What trust is required, other than the cross-realm authentication of kerberos tickets? Surely each realm would manage its own host keys. All I'm looking for is an authenticated cross-realm key lookup so that my client can pre-cache entries in the known_hosts file. Wouldn't this just be an LDAP lookup? Dan From jcholast at redhat.com Wed Nov 9 14:38:30 2011 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 09 Nov 2011 15:38:30 +0100 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <20111109105904.GA5529@redhat.com> References: <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <4EB152A7.5010303@redhat.com> <20111102151114.GD14373@redhat.com> <4EB15E73.8070402@redhat.com> <20111102163720.GA23092@redhat.com> <20111102163848.GB23092@redhat.com> <4EB2614E.8090307@redhat.com> <20111103114352.GB26667@redhat.com> <4EBA54D1.4040700@redhat.com> <20111109105904.GA5529@redhat.com> Message-ID: <4EBA9066.4070600@redhat.com> Dne 9.11.2011 11:59, Alexander Bokovoy napsal(a): > On Wed, 09 Nov 2011, Jan Cholasta wrote: >>>> would't suffer from a facelift - currently it is messy, hard to read >>>> code, with bits nobody ever used or uses anymore, some of the >>>> docstrings and comments aren't correct, etc.) >>> A review of API class is a good idea. However, I think it is currently >>> enough what is in proposed patch as it gets you 2-3x speed increase >>> already. >> >> I've already started experimenting with the API class, hopefully it >> will result in something useful :) > Good. > >>> This is something I'd like to keep as it has great value for all >>> end-users and it requires loading all Python files in ipalib/plugins >>> (and in ipaserver/plugins for server side). >> >> I'm not suggesting we should skip importing the modules. The plugin >> initialization process consists of these 3 steps: >> >> 1. import plugin modules >> 2. create plugin instances >> 3. finalize plugin instances >> >> What I'm suggesting is that we do steps 2 and 3 on-demand. We can't >> do step 1 on-demand, because we have no way of knowing what plugins >> are available without importing the modules. (Both your and my patch >> does only step 3 on-demand, but I think we can do better than that.) > Agreed. > >>>> Anyway, if we decide to go with your solution, please make sure >>>> *all* the plugins that are used are finalized (because of the >>>> locking) and add a new api.env attribute which controls the lazy >>>> finalization, so that the behavior can be overriden (actually I have >>>> already done that - see attached patch - just use >>>> "api.env.plugins_on_demand" instead of "api.env.context == 'cli'"). >>> Done. >>> >> >> + if not self.__dict__['_Plugin__finalized']: >> + self.finalize() >> >> This doesn't seem right, coding style-wise. If you want to access >> the property from outside the Plugin class, why did you name-mangle >> it in the first place? > It is supposed to be internal detail of a Plugin. I agree it stinks a > bit here. :) > > >> def finalize(self): >> """ >> """ >> + if not self.__finalized: >> + self.__finalized = True >> + else: >> + return >> if not is_production_mode(self): >> lock(self) >> >> Finalize is supposed to be called just once, so IMO it would be >> better not to do the check and let it throw the exception. > On contrary, I think sequential finalize() calls should do nothing. > This is at least what I expect from a finalized object -- no-op. In my patch, finalize() throws an exception if called more than once, but ensure_finalized() does nothing for plugins that are already finalized. So there are both kinds of behavior available. > >> + for i in ('Object', 'Property', 'Backend'): >> + if i in self: >> + namespaces.append(self[i]) >> >> AFAIK the framework is supposed to be generally usable for other >> projects in the future. With that in mind, I think we can't afford >> to hard-code things like this. > That's true. As you managed to get around without hardcoding, we can > drop this part. > >> Anyway, I've made a patch that uses data descriptors for the trap >> objects (which is IMHO more elegant than what you do). I've also >> managed to add on-demand finalization to Object and Attribute >> subclasses by moving the set-up code from set_api() to finalize() >> (it doesn't add much performance, though). See attachment. > I read through the code and I like it! Did you get the whole test > suite running with it? There are some parts of tests that expect > non-finalized objects to have None properties while they are now not > None. I always run the test suite ;-) All the unit tests succeed (they do when run with my patch 54 applied, which fixes failures of some unrelated unit tests). > > If so, preliminary ACK for your patch from reading it if you would > make sure on_finalize() allows multiple calls and would make a better > commit message. ;) on_finalize() shouldn't be called directly (perhaps I should rename it to _on_finalize to emphasize that...?) I'll work on the commit message. As usual, it is the hardest part of the patch for me. > > I'll try to arrange testing myself today/tomorrow. > >> The numbers from my VM: >> >> master abbra jcholast >> $ time ipa >> real 0m1.288s 0m0.619s 0m0.601s >> user 0m1.103s 0m0.478s 0m0.451s >> sys 0m0.161s 0m0.126s 0m0.133s >> >> $ time ipa user-find >> real 0m1.897s 0m1.253s 0m1.235s >> user 0m1.119s 0m0.486s 0m0.488s >> sys 0m0.160s 0m0.160s 0m0.136s >> >> $ time ipa help >> real 0m1.299s 0m0.629s 0m0.600s >> user 0m1.094s 0m0.477s 0m0.446s >> sys 0m0.183s 0m0.136s 0m0.140s > Looks good (your VM is supposedly at the same farm as mine so numbers > are comparable). There is anyway great variation in execution times in > VMs so 600ms vs 629ms is roughly the same. Yep. > > Thanks a lot! I think it is great advance over the original approach. Thanks for the kind words :-) Honza -- Jan Cholasta From mkosek at redhat.com Wed Nov 9 14:38:46 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 09 Nov 2011 15:38:46 +0100 Subject: [Freeipa-devel] [PATCH] 163+164 Fix DNS zone --allow-dynupdate option behavior In-Reply-To: <20111109141401.GA8188@redhat.com> References: <1320846124.11381.10.camel@balmora.brq.redhat.com> <20111109141401.GA8188@redhat.com> Message-ID: <1320849528.11381.13.camel@balmora.brq.redhat.com> On Wed, 2011-11-09 at 16:14 +0200, Alexander Bokovoy wrote: > On Wed, 09 Nov 2011, Martin Kosek wrote: > > Patch 164 fixes dnszone-mod --allow-dynupdate behavior which disables > > the zone dynamic updates value whenever a dnszone-mod command is run and > > --allow-dynupdate options is not set. > As discussed on IRC, please rename the CLI option name to > --dynamic-update. > > Otherwise, ACK. Renamed and pushed to master. Endi, Petr, I created a ticket to implement this new behavior in WebUI too: https://fedorahosted.org/freeipa/ticket/2084 > > Please also file a ticket for 3.0 for gathering all renames of CLI > options where we feel they are unclear or unfriendly. https://fedorahosted.org/freeipa/ticket/2083 > > > I introduced a Param.encode() function (patch 163) to our framework to > > help encoding Python native values to LDAP representation more > > effectively. True/False to LDAP's "TRUE"/"FALSE" in this case. Encoding > > functions are executed in a server context only. > Ack. > Pushed to master. Martin From mkosek at redhat.com Wed Nov 9 16:51:15 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 09 Nov 2011 17:51:15 +0100 Subject: [Freeipa-devel] [PATCH] 120 Improve DNS record data validation In-Reply-To: <4EB450C4.2060705@redhat.com> References: <1315400728.12548.12.camel@dhcp-25-52.brq.redhat.com> <1315401507.12548.14.camel@dhcp-25-52.brq.redhat.com> <1319026515.9647.13.camel@balmora.brq.redhat.com> <4E9F2742.3020109@redhat.com> <1319533225.14006.13.camel@balmora.brq.redhat.com> <4EB450C4.2060705@redhat.com> Message-ID: <1320857477.11381.26.camel@balmora.brq.redhat.com> On Fri, 2011-11-04 at 16:53 -0400, Rob Crittenden wrote: > Martin Kosek wrote: > > On Wed, 2011-10-19 at 15:38 -0400, Adam Young wrote: > >> On 10/19/2011 08:15 AM, Martin Kosek wrote: > >>> On Wed, 2011-09-07 at 15:18 +0200, Martin Kosek wrote: > >>>> On Wed, 2011-09-07 at 15:05 +0200, Martin Kosek wrote: > >>>>> This is 3.0 Core Effort Backlog patch. > >>>>> > >>>>> The changes to API may look scary, but it should be OK, I just added > >>>>> validators and normalizers. I found a lot of RR types unsupported by > >>>>> bind-dyndb-ldap. I implemented a validator telling this information to > >>>>> the user. I think the message is more user-friendly than the previous > >>>>> LDAP schema error. > >>>>> > >>>>> Enjoy the RFCs! :-) > >>>>> > >>>>> Martin > >>>>> > >>>>> --- > >>>>> Implement missing validators for DNS RR types so that we can capture > >>>>> at least basic user errors. Additionally, a normalizer creating > >>>>> a fully-qualified domain name has been implemented for several RRs > >>>>> to prevent this common user error. > >>>>> > >>>>> https://fedorahosted.org/freeipa/ticket/1106 > >>>>> > >>>> I noticed a typo in format description for LOC record validation. A > >>>> fixed patch attached. > >>>> > >>>> Martin > >>> Rebased for current master. > >>> > >>> This patch is still waiting for review. As I would like to base my next > >>> DNS work (structured DNS commands) on this patch I would like to have it > >>> reviewed soon. > >>> > >>> Thanks, > >>> Martin > >>> > >>> > >>> > >>> _______________________________________________ > >>> Freeipa-devel mailing list > >>> Freeipa-devel at redhat.com > >>> https://www.redhat.com/mailman/listinfo/freeipa-devel > >> > >> > >> I've just given it a visual review, but it looks right. Probably > >> should have some unit tests to go with it for some of the more > >> commonly used types. > > > > Good idea. A, AAAA, NS records are already being checked, I added tests > > for MX and SRV records too. > > > > I also refactored DNS tests a little, there were many repeatedly using > > hard-coded values (like default zone manager) which would be hard to fix > > of anything changes. > > > > Martin > > I can't tell what your intention is with the split for cname and dname > but it seems to allow just about any value. > > I know there are a ton of data types but is it worthwhile to have a > positive and negative case for each to avoid regressions? > > rob I just wanted to make sure that user enters one value and not more. This effectively tests if there is no whitespace in the value, but you are right that this is a silly way of testing it. I rather implemented a hostname validator to common ipalib module util.py following the RFCs for hostname. It is intended to replace various hostname validators we use across IPA code. But I will leave this step for future. For now, I just replaced the dumb validator with split() for CNAME, DNAME and PTR validators to this new function which actually validates the hostname value. As you advised, I implemented some more unit tests with positive, negative cases. Now, we should have most of the common types covered (A, AAAA, NS, MX, SRV, LOC, CNAME, KX, PTR). I did not implement tests for DNSSEC RR types as I expect some changes there in the future. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-120-5-improve-dns-record-data-validation.patch Type: text/x-patch Size: 91294 bytes Desc: not available URL: From simo at redhat.com Wed Nov 9 17:43:33 2011 From: simo at redhat.com (Simo Sorce) Date: Wed, 09 Nov 2011 12:43:33 -0500 Subject: [Freeipa-devel] [Freeipa-users] OpenSSH integration - known_hosts In-Reply-To: References: <201111081838.13363.jzeleny@redhat.com> <4EB9B3D0.9080904@redhat.com> <1320795310.7734.853.camel@willson.li.ssimo.org> <1320846928.7734.859.camel@willson.li.ssimo.org> Message-ID: <1320860613.7734.868.camel@willson.li.ssimo.org> CCing freeipa-devel so that others can benefit from replies too. On Wed, 2011-11-09 at 09:49 -0500, Dan Scott wrote: > > Well in 2-way trusts you could do that. But in general you do not want > > to have any client in realm1 to contact any server in realm2. They might > > be geographically very far and use high latency/low bandwidth links. > > I must be missing something here. Why does it need to be a 2-way > trust? Because otherwise you may not have access to the keys at all unless anonymous binds are allowed (this is the default indeed). > My local IPA server doesn't have to trust the remote IPA server > at all, does it (other than encryption/certificates to prevent a MITM > - although the existing system doesn't help with that either, for the > initial host key transfer)? Even then, wouldn't an unauthenticated > LDAP lookup be OK for transferring the host keys? No, this is the other aspect. With GSSAPI auth you have mutual authentication. This means you can trust the remote server. If you just let clients do anonymous binds and fetch data from a server w/o authentication then it is easy for an attacker to MITM the LDAP connection and give you public keys for a ssh server that will also MITM you. I know that in many environments people will consider this an unlikely threat and not care about it, so maybe an RFC to allow SSSD to do this with un-authenticated connections will be allowed. > My client would be trying to contact a system in the other realm > anyway, so I don't really see the latency/bandwidth problems. It depends on the architecture of your specific setup. I was giving generic reasons why. > Surely > whatever I'm doing over SSH will require more bandwidth than the key > transfer. Maybe, maybe not, you may have a host in your local lan that belongs to a remote realm and only the KDC/LDAP server is remote to you. > Instead of my client contacting the remote SSH host > directly. My local SSSD would contact the other realm's IPA server to > get the key of the host I'm connecting to. Even then, this would be a > 'one-time' key transfer although it would need to be re-checked > occasionally. Yes, hence we may allow it. > > So, for a first implementation, we could do what you say, but I rather > > think it through and see how to cache/transfer information making > > clients go through their IPA server rather than trying to connect > > directly to a remote one. > > Sure, makes sense. Getting it working in one realm is the priority. There is also another reason why going through a 'proxy' service is useful. If you have different IPA versions (or even different DC technology like IPA vs AD) on the 2 sides of the trust we would be able to build a 'translation' service on the IPA server so that clients do not need to learn how to speak to other Identity Domains directly. Simo. -- Simo Sorce * Red Hat, Inc * New York From edewata at redhat.com Wed Nov 9 20:28:13 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 09 Nov 2011 14:28:13 -0600 Subject: [Freeipa-devel] [PATCH] 310 Updated sample data. Message-ID: <4EBAE25D.8010003@redhat.com> New sample data files have been added for search facet paging. Unused files have been removed. The names used in the files have been updated for consistency. Ticket #981 -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-edewata-0310-Updated-sample-data.patch Type: text/x-patch Size: 411155 bytes Desc: not available URL: From edewata at redhat.com Wed Nov 9 20:28:30 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 09 Nov 2011 14:28:30 -0600 Subject: [Freeipa-devel] [PATCH] 311 Added paging on search facet. Message-ID: <4EBAE26E.9010302@redhat.com> The search facet has been modified to support paging on most entities using the --pkey-only option to get the primary keys and a batch command to get the complete records. Paging on DNS records is not supported because a record may appear as multiple rows. The following entities do not have --pkey-only option: Automount Key, Self-Service Permissions, Delegation. The search and association facet have been refactored to reuse the common code from the table facet base class. Ticket #981 -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-edewata-0311-Added-paging-on-search-facet.patch Type: text/x-patch Size: 32739 bytes Desc: not available URL: From kybaker at redhat.com Wed Nov 9 22:08:31 2011 From: kybaker at redhat.com (Kyle Baker) Date: Wed, 09 Nov 2011 17:08:31 -0500 (EST) Subject: [Freeipa-devel] Screens For HBAC Testing (Ticket #388) In-Reply-To: <3344a423-406e-4a5f-9557-27eaf35037c9@zmail05.collab.prod.int.phx2.redhat.com> Message-ID: <849298c7-3090-46ae-a699-420a76d4ec32@zmail05.collab.prod.int.phx2.redhat.com> Endi and I worked to create the model for the new HBAC Testing section. The screens are attached. Kyle Baker Visual Designer Desk (978) 392 3116 UX Team IRC kylebaker -------------- next part -------------- A non-text attachment was scrubbed... Name: HBAC Test UI Model.zip Type: application/zip Size: 478156 bytes Desc: not available URL: From dpal at redhat.com Wed Nov 9 22:32:31 2011 From: dpal at redhat.com (Dmitri Pal) Date: Wed, 09 Nov 2011 17:32:31 -0500 Subject: [Freeipa-devel] Screens For HBAC Testing (Ticket #388) In-Reply-To: <849298c7-3090-46ae-a699-420a76d4ec32@zmail05.collab.prod.int.phx2.redhat.com> References: <849298c7-3090-46ae-a699-420a76d4ec32@zmail05.collab.prod.int.phx2.redhat.com> Message-ID: <4EBAFF7F.7010007@redhat.com> On 11/09/2011 05:08 PM, Kyle Baker wrote: > Endi and I worked to create the model for the new HBAC Testing section. The screens are attached. > > Kyle Baker > Visual Designer Desk (978) 392 3116 > UX Team IRC kylebaker > Since "from" host is unreliable, one of the latest decisions in SSSD was to ignore "from" host part of the rule by default (causes a lot of performance issues too) and have a config parameter to explicitly not ignore it. I think the UI should reflect in some way that "From" should not be generally used and not an "equal" citizen of the HBAC rule. We probably should update the existing UI too to discourage people from using it and also document it in man pages for HBAC and in the docs. -- Thank you, Dmitri Pal Sr. Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From mkosek at redhat.com Thu Nov 10 09:30:42 2011 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 10 Nov 2011 10:30:42 +0100 Subject: [Freeipa-devel] [PATCH] the 'Keytab:' field in "ipa user-show" output is misleading In-Reply-To: <4EB985F8.6060107@redhat.com> References: <4EB985F8.6060107@redhat.com> Message-ID: <1320917445.31133.19.camel@balmora.brq.redhat.com> On Tue, 2011-11-08 at 20:41 +0100, Ondrej Hamada wrote: > https://fedorahosted.org/freeipa/ticket/1961 > > The 'Keytab' filed in output of all 'user-*' commands was changed to > 'Kerberos keys available'. In order to do this change for 'user-*' > commands only, the flag 'has_keytab' had to be removed from common > output parametrs in ipalib/baseldap.py. This change also affected the > host.py and service.py, where the 'has_keytab' flag was added to their > local output params. Both host.py and service.py holds the old field > caption - 'Keytab' - because of compatibility with older clients. > Ondra, thanks for the patch. It looks OK, everything behaves as expected. I am still concerned about your patch formatting: 1) Patch naming does not follow FreeIPA conventions. You can check others patch file names - mine, Rob's or Alexander's for example. The patch file name should be freeipa-ohamada-2-.patch. The patch number should also be in your mail subject - it helps when searching mails etc. 2) Patch title is wrong - you don't need to include [PATCH] in git commit's title. This then makes it here twice. 3) Patch description is insufficient. I miss link to ticket and some description. You only added it to the mail. When I am traversing FreeIPA git logs, I must be able to quickly read what this patch does. You would have seen all these conventions I wrote you about if you had read some patches in freeipa-devel or had read some in FreeIPA git log. Martin From abokovoy at redhat.com Thu Nov 10 10:19:45 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 10 Nov 2011 12:19:45 +0200 Subject: [Freeipa-devel] [PATCH] the 'Keytab:' field in "ipa user-show" output is misleading In-Reply-To: <1320917445.31133.19.camel@balmora.brq.redhat.com> References: <4EB985F8.6060107@redhat.com> <1320917445.31133.19.camel@balmora.brq.redhat.com> Message-ID: <20111110101944.GA1762@redhat.com> On Thu, 10 Nov 2011, Martin Kosek wrote: > On Tue, 2011-11-08 at 20:41 +0100, Ondrej Hamada wrote: > > https://fedorahosted.org/freeipa/ticket/1961 > > > > The 'Keytab' filed in output of all 'user-*' commands was changed to > > 'Kerberos keys available'. In order to do this change for 'user-*' > > commands only, the flag 'has_keytab' had to be removed from common > > output parametrs in ipalib/baseldap.py. This change also affected the > > host.py and service.py, where the 'has_keytab' flag was added to their > > local output params. Both host.py and service.py holds the old field > > caption - 'Keytab' - because of compatibility with older clients. > > > > Ondra, thanks for the patch. It looks OK, everything behaves as > expected. > > I am still concerned about your patch formatting: > 1) Patch naming does not follow FreeIPA conventions. You can check > others patch file names - mine, Rob's or Alexander's for example. The > patch file name should be freeipa-ohamada-2-.patch. The > patch number should also be in your mail subject - it helps when > searching mails etc. > > 2) Patch title is wrong - you don't need to include [PATCH] in git > commit's title. This then makes it here twice. > > 3) Patch description is insufficient. I miss link to ticket and some > description. You only added it to the mail. When I am traversing FreeIPA > git logs, I must be able to quickly read what this patch does. > > You would have seen all these conventions I wrote you about if you had > read some patches in freeipa-devel or had read some in FreeIPA git log. We have most of this knowledge documented at https://fedorahosted.org/freeipa/wiki/PatchFormat -- / Alexander Bokovoy From mkosek at redhat.com Thu Nov 10 11:03:58 2011 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 10 Nov 2011 12:03:58 +0100 Subject: [Freeipa-devel] [PATCH] the 'Keytab:' field in "ipa user-show" output is misleading In-Reply-To: <20111110101944.GA1762@redhat.com> References: <4EB985F8.6060107@redhat.com> <1320917445.31133.19.camel@balmora.brq.redhat.com> <20111110101944.GA1762@redhat.com> Message-ID: <1320923040.31133.21.camel@balmora.brq.redhat.com> On Thu, 2011-11-10 at 12:19 +0200, Alexander Bokovoy wrote: > On Thu, 10 Nov 2011, Martin Kosek wrote: > > On Tue, 2011-11-08 at 20:41 +0100, Ondrej Hamada wrote: > > > https://fedorahosted.org/freeipa/ticket/1961 > > > > > > The 'Keytab' filed in output of all 'user-*' commands was changed to > > > 'Kerberos keys available'. In order to do this change for 'user-*' > > > commands only, the flag 'has_keytab' had to be removed from common > > > output parametrs in ipalib/baseldap.py. This change also affected the > > > host.py and service.py, where the 'has_keytab' flag was added to their > > > local output params. Both host.py and service.py holds the old field > > > caption - 'Keytab' - because of compatibility with older clients. > > > > > > > Ondra, thanks for the patch. It looks OK, everything behaves as > > expected. > > > > I am still concerned about your patch formatting: > > 1) Patch naming does not follow FreeIPA conventions. You can check > > others patch file names - mine, Rob's or Alexander's for example. The > > patch file name should be freeipa-ohamada-2-.patch. The > > patch number should also be in your mail subject - it helps when > > searching mails etc. > > > > 2) Patch title is wrong - you don't need to include [PATCH] in git > > commit's title. This then makes it here twice. > > > > 3) Patch description is insufficient. I miss link to ticket and some > > description. You only added it to the mail. When I am traversing FreeIPA > > git logs, I must be able to quickly read what this patch does. > > > > You would have seen all these conventions I wrote you about if you had > > read some patches in freeipa-devel or had read some in FreeIPA git log. > We have most of this knowledge documented at > https://fedorahosted.org/freeipa/wiki/PatchFormat > Thanks for the link. I updated the wiki page and fixed the formatting and some spelling errors as it was not very easy to read the previous version. Martin From mkosek at redhat.com Thu Nov 10 11:13:14 2011 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 10 Nov 2011 12:13:14 +0100 Subject: [Freeipa-devel] [PATCH] 165 Polish ipa config help Message-ID: <1320923596.31133.22.camel@balmora.brq.redhat.com> Remove config-mod options help from config module help to keep the option doc on one place and without unnecessary redundancy. The new format is more consistent with the rest of the plugins. Also fix several inconsistencies in the labels/doc, including: - using abbreviation when not needed - using '.' at the end of options help https://fedorahosted.org/freeipa/ticket/1906 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-165-polish-ipa-config-help.patch Type: text/x-patch Size: 11066 bytes Desc: not available URL: From ohamada at redhat.com Thu Nov 10 13:01:06 2011 From: ohamada at redhat.com (Ondrej Hamada) Date: Thu, 10 Nov 2011 14:01:06 +0100 Subject: [Freeipa-devel] [PATCH] the 'Keytab:' field in "ipa user-show" output is misleading In-Reply-To: <1320917445.31133.19.camel@balmora.brq.redhat.com> References: <4EB985F8.6060107@redhat.com> <1320917445.31133.19.camel@balmora.brq.redhat.com> Message-ID: <4EBBCB12.8060604@redhat.com> On 11/10/2011 10:30 AM, Martin Kosek wrote: > On Tue, 2011-11-08 at 20:41 +0100, Ondrej Hamada wrote: >> https://fedorahosted.org/freeipa/ticket/1961 >> >> The 'Keytab' filed in output of all 'user-*' commands was changed to >> 'Kerberos keys available'. In order to do this change for 'user-*' >> commands only, the flag 'has_keytab' had to be removed from common >> output parametrs in ipalib/baseldap.py. This change also affected the >> host.py and service.py, where the 'has_keytab' flag was added to their >> local output params. Both host.py and service.py holds the old field >> caption - 'Keytab' - because of compatibility with older clients. >> > Ondra, thanks for the patch. It looks OK, everything behaves as > expected. > > I am still concerned about your patch formatting: > 1) Patch naming does not follow FreeIPA conventions. You can check > others patch file names - mine, Rob's or Alexander's for example. The > patch file name should be freeipa-ohamada-2-.patch. The > patch number should also be in your mail subject - it helps when > searching mails etc. > > 2) Patch title is wrong - you don't need to include [PATCH] in git > commit's title. This then makes it here twice. > > 3) Patch description is insufficient. I miss link to ticket and some > description. You only added it to the mail. When I am traversing FreeIPA > git logs, I must be able to quickly read what this patch does. > > You would have seen all these conventions I wrote you about if you had > read some patches in freeipa-devel or had read some in FreeIPA git log. > > Martin > > Shame on me. Sorry for that. Corrected patch attached. -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ohamada-2-2-Misleading-Keytab-field.patch Type: text/x-patch Size: 5152 bytes Desc: not available URL: From mkosek at redhat.com Thu Nov 10 13:14:07 2011 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 10 Nov 2011 14:14:07 +0100 Subject: [Freeipa-devel] [PATCH] the 'Keytab:' field in "ipa user-show" output is misleading In-Reply-To: <4EBBCB12.8060604@redhat.com> References: <4EB985F8.6060107@redhat.com> <1320917445.31133.19.camel@balmora.brq.redhat.com> <4EBBCB12.8060604@redhat.com> Message-ID: <1320930849.31133.23.camel@balmora.brq.redhat.com> On Thu, 2011-11-10 at 14:01 +0100, Ondrej Hamada wrote: > On 11/10/2011 10:30 AM, Martin Kosek wrote: > > On Tue, 2011-11-08 at 20:41 +0100, Ondrej Hamada wrote: > >> https://fedorahosted.org/freeipa/ticket/1961 > >> > >> The 'Keytab' filed in output of all 'user-*' commands was changed to > >> 'Kerberos keys available'. In order to do this change for 'user-*' > >> commands only, the flag 'has_keytab' had to be removed from common > >> output parametrs in ipalib/baseldap.py. This change also affected the > >> host.py and service.py, where the 'has_keytab' flag was added to their > >> local output params. Both host.py and service.py holds the old field > >> caption - 'Keytab' - because of compatibility with older clients. > >> > > Ondra, thanks for the patch. It looks OK, everything behaves as > > expected. > > > > I am still concerned about your patch formatting: > > 1) Patch naming does not follow FreeIPA conventions. You can check > > others patch file names - mine, Rob's or Alexander's for example. The > > patch file name should be freeipa-ohamada-2-.patch. The > > patch number should also be in your mail subject - it helps when > > searching mails etc. > > > > 2) Patch title is wrong - you don't need to include [PATCH] in git > > commit's title. This then makes it here twice. > > > > 3) Patch description is insufficient. I miss link to ticket and some > > description. You only added it to the mail. When I am traversing FreeIPA > > git logs, I must be able to quickly read what this patch does. > > > > You would have seen all these conventions I wrote you about if you had > > read some patches in freeipa-devel or had read some in FreeIPA git log. > > > > Martin > > > > > Shame on me. Sorry for that. > > Corrected patch attached. > ACK. Pushed to master. You can celebrate your first upstream patch :-) Martin From mkosek at redhat.com Thu Nov 10 14:10:48 2011 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 10 Nov 2011 15:10:48 +0100 Subject: [Freeipa-devel] [PATCH] 901 remove has_upg() and has_managed_entries() In-Reply-To: <4EB987B2.1040105@redhat.com> References: <4EB987B2.1040105@redhat.com> Message-ID: <1320934251.31133.27.camel@balmora.brq.redhat.com> On Tue, 2011-11-08 at 14:49 -0500, Rob Crittenden wrote: > These functions are leftovers from when the managed entries plugin was > being developed and not widely available. They are no longer needed. > > rob NACK. has_upg() does not check if the managed entry plugin present in dirsrv but rather if the UPG rule is enabled. If the UPG rule is not enabled, user gidNumber is set to ipausers' GID. has_upg() behavior was lately being fixed in: https://fedorahosted.org/freeipa/ticket/1964 Martin From edewata at redhat.com Thu Nov 10 17:06:54 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 10 Nov 2011 11:06:54 -0600 Subject: [Freeipa-devel] Screens For HBAC Testing (Ticket #388) In-Reply-To: <4EBAFF7F.7010007@redhat.com> References: <849298c7-3090-46ae-a699-420a76d4ec32@zmail05.collab.prod.int.phx2.redhat.com> <4EBAFF7F.7010007@redhat.com> Message-ID: <4EBC04AE.7010005@redhat.com> On 11/9/2011 4:32 PM, Dmitri Pal wrote: > Since "from" host is unreliable, one of the latest decisions in SSSD was > to ignore "from" host part of the rule by default (causes a lot of > performance issues too) and have a config parameter to explicitly not > ignore it. I think the UI should reflect in some way that "From" should > not be generally used and not an "equal" citizen of the HBAC rule. We > probably should update the existing UI too to discourage people from > using it and also document it in man pages for HBAC and in the docs. In HBAC test we can add a note saying the source host is optional. In HBAC rule the default source host category is 'all', which has the same effect. Should we display a warning when the category is changed? The CLI will be changed to accept empty source host: https://fedorahosted.org/freeipa/ticket/2085 -- Endi S. Dewata From kybaker at redhat.com Thu Nov 10 20:00:19 2011 From: kybaker at redhat.com (Kyle Baker) Date: Thu, 10 Nov 2011 15:00:19 -0500 (EST) Subject: [Freeipa-devel] Ticket #1976 - Tab color groups In-Reply-To: <7b235877-7161-4162-930c-1376b9248084@zmail05.collab.prod.int.phx2.redhat.com> Message-ID: <34d9364e-9d93-4c3b-bb5c-bcb4c8601531@zmail05.collab.prod.int.phx2.redhat.com> Attached a mockup which shows each tab in a color set. This offers a quick visual reference as to the tab groupings. Also I have shown settings to the right as this is inconsistent amongst the different sections of the tool. Setting should be displayed to the right as users are more likely to focus on the content. Kyle Baker Visual Designer Desk (978) 392 3116 UX Team IRC kylebaker -------------- next part -------------- A non-text attachment was scrubbed... Name: IPA_tab-color-mockup.png Type: image/png Size: 73277 bytes Desc: not available URL: From rcritten at redhat.com Thu Nov 10 20:00:53 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 10 Nov 2011 15:00:53 -0500 Subject: [Freeipa-devel] [PATCH] 901 remove has_upg() and has_managed_entries() In-Reply-To: <1320934251.31133.27.camel@balmora.brq.redhat.com> References: <4EB987B2.1040105@redhat.com> <1320934251.31133.27.camel@balmora.brq.redhat.com> Message-ID: <4EBC2D75.30807@redhat.com> Martin Kosek wrote: > On Tue, 2011-11-08 at 14:49 -0500, Rob Crittenden wrote: >> These functions are leftovers from when the managed entries plugin was >> being developed and not widely available. They are no longer needed. >> >> rob > > NACK. > > has_upg() does not check if the managed entry plugin present in dirsrv > but rather if the UPG rule is enabled. If the UPG rule is not enabled, > user gidNumber is set to ipausers' GID. > > has_upg() behavior was lately being fixed in: > https://fedorahosted.org/freeipa/ticket/1964 > > Martin > This is one case I should have listened to the little voice in my head. Revised patch. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-901-2-managed.patch Type: text/x-patch Size: 3269 bytes Desc: not available URL: From ayoung at redhat.com Thu Nov 10 20:30:25 2011 From: ayoung at redhat.com (Adam Young) Date: Thu, 10 Nov 2011 15:30:25 -0500 Subject: [Freeipa-devel] Ticket #1976 - Tab color groups In-Reply-To: <34d9364e-9d93-4c3b-bb5c-bcb4c8601531@zmail05.collab.prod.int.phx2.redhat.com> References: <34d9364e-9d93-4c3b-bb5c-bcb4c8601531@zmail05.collab.prod.int.phx2.redhat.com> Message-ID: <4EBC3461.80401@redhat.com> I like. On 11/10/2011 03:00 PM, Kyle Baker wrote: > Attached a mockup which shows each tab in a color set. This offers a quick visual reference as to the tab groupings. Also I have shown settings to the right as this is inconsistent amongst the different sections of the tool. Setting should be displayed to the right as users are more likely to focus on the content. > > Kyle Baker > Visual Designer Desk (978) 392 3116 > UX Team IRC kylebaker > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From edewata at redhat.com Thu Nov 10 21:23:24 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 10 Nov 2011 15:23:24 -0600 Subject: [Freeipa-devel] Ticket #1976 - Tab color groups In-Reply-To: <34d9364e-9d93-4c3b-bb5c-bcb4c8601531@zmail05.collab.prod.int.phx2.redhat.com> References: <34d9364e-9d93-4c3b-bb5c-bcb4c8601531@zmail05.collab.prod.int.phx2.redhat.com> Message-ID: <4EBC40CC.6060903@redhat.com> On 11/10/2011 2:00 PM, Kyle Baker wrote: > Attached a mockup which shows each tab in a color set. This offers a > quick visual reference as to the tab groupings. The colored tab looks good. What are the color settings? Suppose we have more than 3 tab groups what colors should we use? > Also I have shown settings to the right as this is inconsistent > amongst the different sections of the tool. Setting should be > displayed to the right as users are more likely to focus on the > content. It might not be the same for all entities. For groups the members are probably more interesting, but for ACI permissions users are probably more interested to see the settings rather than the privileges in which it belongs to. Should we have a 'summary' tab that displays the most important info, then we provide links to go to the tab to edit the info. For example: DNS Zone: example.com | Summary | | Settings | | Records | ---------------------------------------- Settings [edit] Name: example.com Status: Active Records [edit] host1 A 192.168.1.1 host2 A 192.168.1.2 -- Endi S. Dewata From dpal at redhat.com Thu Nov 10 23:41:04 2011 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 10 Nov 2011 18:41:04 -0500 Subject: [Freeipa-devel] Screens For HBAC Testing (Ticket #388) In-Reply-To: <4EBC04AE.7010005@redhat.com> References: <849298c7-3090-46ae-a699-420a76d4ec32@zmail05.collab.prod.int.phx2.redhat.com> <4EBAFF7F.7010007@redhat.com> <4EBC04AE.7010005@redhat.com> Message-ID: <4EBC6110.4030600@redhat.com> On 11/10/2011 12:06 PM, Endi Sukma Dewata wrote: > On 11/9/2011 4:32 PM, Dmitri Pal wrote: >> Since "from" host is unreliable, one of the latest decisions in SSSD was >> to ignore "from" host part of the rule by default (causes a lot of >> performance issues too) and have a config parameter to explicitly not >> ignore it. I think the UI should reflect in some way that "From" should >> not be generally used and not an "equal" citizen of the HBAC rule. We >> probably should update the existing UI too to discourage people from >> using it and also document it in man pages for HBAC and in the docs. > > In HBAC test we can add a note saying the source host is optional. In > HBAC rule the default source host category is 'all', which has the > same effect. Should we display a warning when the category is changed? Probably. Some reasonable indication should be used. > > The CLI will be changed to accept empty source host: > https://fedorahosted.org/freeipa/ticket/2085 > -- Thank you, Dmitri Pal Sr. Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jdennis at redhat.com Fri Nov 11 00:04:43 2011 From: jdennis at redhat.com (John Dennis) Date: Thu, 10 Nov 2011 19:04:43 -0500 Subject: [Freeipa-devel] [PATCH] new logging manager module Message-ID: <4EBC669B.4020101@redhat.com> O.K. the subject is a bit deceptive, there is no patch, this is really a review request and I'm going to handle this a bit differently. Rather than attaching a patch I'm including a link to the module's HTML documentation. You'll be able to read the documentation and follow links which may be a better introduction for a review. Don't worry, the actual source is there too! Every page has a clickable link "Source Code". Here is the doc (and source): http://jdennis.fedorapeople.org/log_manager_doc/ By the way, this is the proposed fix for ticket 2022 (https://fedorahosted.org/freeipa/ticket/2022). Which grew out of my frustration with not being able to turn on debug logging when running the unit tests. This should solve that problem, consolidate a lot of existing duplicated code spread out across our code base and make using logging easier and more productive, especially for developers. Just one minor caveat. The module needs to be located in ipapython because it will be used by all our code. At the moment there is a IPALogManger class which is subclassed from the generic LogManager class. This is mostly for reading the logging config from our Env configuration and enforcing some IPA standards. But Env is located in ipalib.config so we may want to move the IPALogManager class out of ipapython and into ipalib (not entirely sure about that yet). Comments and questions welcome of course. P.S. there is a fair amount of doc with the rationale for why things were done this way, plenty of examples and even a FAQ. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From ayoung at redhat.com Fri Nov 11 02:03:13 2011 From: ayoung at redhat.com (Adam Young) Date: Thu, 10 Nov 2011 21:03:13 -0500 Subject: [Freeipa-devel] Ticket #1976 - Tab color groups In-Reply-To: <4EBC40CC.6060903@redhat.com> References: <34d9364e-9d93-4c3b-bb5c-bcb4c8601531@zmail05.collab.prod.int.phx2.redhat.com> <4EBC40CC.6060903@redhat.com> Message-ID: <4EBC8261.8090501@redhat.com> On 11/10/2011 04:23 PM, Endi Sukma Dewata wrote: > On 11/10/2011 2:00 PM, Kyle Baker wrote: >> Attached a mockup which shows each tab in a color set. This offers a >> quick visual reference as to the tab groupings. > > The colored tab looks good. What are the color settings? Suppose we > have more than 3 tab groups what colors should we use? > >> Also I have shown settings to the right as this is inconsistent >> amongst the different sections of the tool. Setting should be >> displayed to the right as users are more likely to focus on the >> content. > > It might not be the same for all entities. For groups the members are > probably more interesting, but for ACI permissions users are probably > more interested to see the settings rather than the privileges in > which it belongs to. Please leave the order as is unless we get real feed back from the field that says that this approach is messing people up. > > Should we have a 'summary' tab that displays the most important info, > then we provide links to go to the tab to edit the info. For example: No. It is a wasted click > > DNS Zone: example.com > > | Summary | | Settings | | Records | > ---------------------------------------- > > Settings [edit] > Name: example.com > Status: Active > > Records [edit] > host1 A 192.168.1.1 > host2 A 192.168.1.2 > From mkosek at redhat.com Fri Nov 11 07:45:46 2011 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 11 Nov 2011 08:45:46 +0100 Subject: [Freeipa-devel] [PATCH] 901 remove has_upg() and has_managed_entries() In-Reply-To: <4EBC2D75.30807@redhat.com> References: <4EB987B2.1040105@redhat.com> <1320934251.31133.27.camel@balmora.brq.redhat.com> <4EBC2D75.30807@redhat.com> Message-ID: <1320997546.4215.4.camel@balmora.brq.redhat.com> On Thu, 2011-11-10 at 15:00 -0500, Rob Crittenden wrote: > Martin Kosek wrote: > > On Tue, 2011-11-08 at 14:49 -0500, Rob Crittenden wrote: > >> These functions are leftovers from when the managed entries plugin was > >> being developed and not widely available. They are no longer needed. > >> > >> rob > > > > NACK. > > > > has_upg() does not check if the managed entry plugin present in dirsrv > > but rather if the UPG rule is enabled. If the UPG rule is not enabled, > > user gidNumber is set to ipausers' GID. > > > > has_upg() behavior was lately being fixed in: > > https://fedorahosted.org/freeipa/ticket/1964 > > > > Martin > > > > This is one case I should have listened to the little voice in my head. > > Revised patch. > > rob Now we're talking :-) ACK. Pushed to master. Martin From pvoborni at redhat.com Fri Nov 11 10:02:58 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 11 Nov 2011 11:02:58 +0100 Subject: [Freeipa-devel] [PATCH] 310 Updated sample data. In-Reply-To: <4EBAE25D.8010003@redhat.com> References: <4EBAE25D.8010003@redhat.com> Message-ID: <4EBCF2D2.9070107@redhat.com> On 11/09/2011 09:28 PM, Endi Sukma Dewata wrote: > New sample data files have been added for search facet paging. Unused > files have been removed. The names used in the files have been updated > for consistency. > > Ticket #981 > ACK -- Petr Vobornik From pvoborni at redhat.com Fri Nov 11 10:07:11 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 11 Nov 2011 11:07:11 +0100 Subject: [Freeipa-devel] [PATCH] 311 Added paging on search facet. In-Reply-To: <4EBAE26E.9010302@redhat.com> References: <4EBAE26E.9010302@redhat.com> Message-ID: <4EBCF3CF.1080807@redhat.com> On 11/09/2011 09:28 PM, Endi Sukma Dewata wrote: > The search facet has been modified to support paging on most entities > using the --pkey-only option to get the primary keys and a batch command > to get the complete records. > > Paging on DNS records is not supported because a record may appear as > multiple rows. The following entities do not have --pkey-only option: > Automount Key, Self-Service Permissions, Delegation. > > The search and association facet have been refactored to reuse the > common code from the table facet base class. > > Ticket #981 > On facets where paging is not enabled is still shown pager (because page_length property is set). This can be misleading. Otherwise it is OK. -- Petr Vobornik From pvoborni at redhat.com Fri Nov 11 12:23:31 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 11 Nov 2011 13:23:31 +0100 Subject: [Freeipa-devel] Ticket #1976 - Tab color groups In-Reply-To: <4EBC40CC.6060903@redhat.com> References: <34d9364e-9d93-4c3b-bb5c-bcb4c8601531@zmail05.collab.prod.int.phx2.redhat.com> <4EBC40CC.6060903@redhat.com> Message-ID: <4EBD13C3.7020803@redhat.com> On 11/10/2011 10:23 PM, Endi Sukma Dewata wrote: > On 11/10/2011 2:00 PM, Kyle Baker wrote: >> Attached a mockup which shows each tab in a color set. This offers a >> quick visual reference as to the tab groupings. > > The colored tab looks good. What are the color settings? Suppose we have > more than 3 tab groups what colors should we use? +1 > >> Also I have shown settings to the right as this is inconsistent >> amongst the different sections of the tool. Setting should be >> displayed to the right as users are more likely to focus on the >> content. > It is consistent. Facet groups are always ordered in following way: 'member', 'settings', 'memberof', 'managedby'. If some facet group doesn't have any facet, it isn't shown. Changing the order can lead to the problem Endi pointed out: > It might not be the same for all entities. For groups the members are > probably more interesting, but for ACI permissions users are probably > more interested to see the settings rather than the privileges in which > it belongs to. The problem might be changed into two separate questions: * How to order the facet groups? * Which facet show as default when displaying record? Right now it is the first facet of first facet group (source of the problem). > > Should we have a 'summary' tab that displays the most important info, > then we provide links to go to the tab to edit the info. For example: > > DNS Zone: example.com > > | Summary | | Settings | | Records | > ---------------------------------------- > > Settings [edit] > Name: example.com > Status: Active > > Records [edit] > host1 A 192.168.1.1 > host2 A 192.168.1.2 > May be useful for some users. -- Petr Vobornik From ohamada at redhat.com Fri Nov 11 13:55:22 2011 From: ohamada at redhat.com (Ondrej Hamada) Date: Fri, 11 Nov 2011 14:55:22 +0100 Subject: [Freeipa-devel] [PATCH] ipa-client-install with --no-sssd option should check for nss_ldap Message-ID: <4EBD294A.20200@redhat.com> https://fedorahosted.org/freeipa/ticket/2063 In order to check presence of nss_ldap when installing client with '--no-sssd' option there was added code into ipa-client-install. Check is base on existence of nss_ldap configuration files. This configuration could be in 'etc/ldap.conf', '/etc/nss_ldap.conf' or '/etc/libnss_ldap.conf'. Presence of any of these files is considered as success otherwise failure. -- Regards, Ondrej Hamada FreeIPA team jabber:ohama at jabbim.cz IRC: ohamada -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ohamada-3-Client-install-checks-for-nss_ldap.patch Type: text/x-patch Size: 1789 bytes Desc: not available URL: From rcritten at redhat.com Fri Nov 11 14:18:31 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 11 Nov 2011 09:18:31 -0500 Subject: [Freeipa-devel] [PATCH] ipa-client-install with --no-sssd option should check for nss_ldap In-Reply-To: <4EBD294A.20200@redhat.com> References: <4EBD294A.20200@redhat.com> Message-ID: <4EBD2EB7.20203@redhat.com> Ondrej Hamada wrote: > https://fedorahosted.org/freeipa/ticket/2063 > > In order to check presence of nss_ldap when installing client with > '--no-sssd' option there was added code into ipa-client-install. Check > is base on existence of nss_ldap configuration files. This configuration > could be in 'etc/ldap.conf', '/etc/nss_ldap.conf' or > '/etc/libnss_ldap.conf'. Presence of any of these files is considered as > success otherwise failure. I think we should check for nslcd.conf as well and report that neither nss-ldap nor nss-pam-ldapd are installed. rob From mkosek at redhat.com Fri Nov 11 14:22:16 2011 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 11 Nov 2011 15:22:16 +0100 Subject: [Freeipa-devel] [PATCH] 166+167 Zonemgr improvements Message-ID: <1321021336.4215.15.camel@balmora.brq.redhat.com> There are 2 patches improving our zone zonemgr: 1) ipa-server-install --zonemgr option validation and normalization + the same thing in ipa dnszone-add/mod --admin-email. They now allow and correctly process '.' in a local-part of the zonemgr e-mail (it is encoded as '\.'. How to test: ipa-server-install -p secret123 -a secret123 --setup-dns --zonemgr=foo.bar at example.com OR if e-mail is passed in SOA format: ipa-server-install -p secret123 -a secret123 --setup-dns --zonemgr='foo\.bar.example.com' In both cases, the zonemgr e-mail will be set to correct format: 'foo \.bar.example.com'. 2) Our default zonemgr is changed to follow RFC 2142 recommendation - hostmaster@ hostmaster is an alias to root anyway (see /etc/aliases). Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-166-improve-zonemgr-validator-and-normalizer.patch Type: text/x-patch Size: 12601 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-167-default-dns-zone-manager.patch Type: text/x-patch Size: 1710 bytes Desc: not available URL: From abokovoy at redhat.com Fri Nov 11 14:25:22 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 11 Nov 2011 16:25:22 +0200 Subject: [Freeipa-devel] [PATCH] ipa-client-install with --no-sssd option should check for nss_ldap In-Reply-To: <4EBD2EB7.20203@redhat.com> References: <4EBD294A.20200@redhat.com> <4EBD2EB7.20203@redhat.com> Message-ID: <20111111142521.GE1762@redhat.com> On Fri, 11 Nov 2011, Rob Crittenden wrote: > Ondrej Hamada wrote: > >https://fedorahosted.org/freeipa/ticket/2063 > > > >In order to check presence of nss_ldap when installing client with > >'--no-sssd' option there was added code into ipa-client-install. Check > >is base on existence of nss_ldap configuration files. This configuration > >could be in 'etc/ldap.conf', '/etc/nss_ldap.conf' or > >'/etc/libnss_ldap.conf'. Presence of any of these files is considered as > >success otherwise failure. > > I think we should check for nslcd.conf as well and report that > neither nss-ldap nor nss-pam-ldapd are installed. We have already code in configure_ldap_config() and configure_nslcd_conf that checks all these different files and after configuration reports what was configured. I would rather did a commonalization of detection instead of duplicating the code. We can re-use result of detecting what exists later in configure_{ldap,nslcd}_config(). -- / Alexander Bokovoy From rcritten at redhat.com Fri Nov 11 15:10:18 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 11 Nov 2011 10:10:18 -0500 Subject: [Freeipa-devel] [PATCH] 904 two more buildrequires Message-ID: <4EBD3ADA.10006@redhat.com> We need two more buildrequires, libtalloc-devel and libtevent-devel. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-904-requires.patch Type: text/x-patch Size: 763 bytes Desc: not available URL: From rcritten at redhat.com Fri Nov 11 15:22:08 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 11 Nov 2011 10:22:08 -0500 Subject: [Freeipa-devel] [PATCH] 166+167 Zonemgr improvements In-Reply-To: <1321021336.4215.15.camel@balmora.brq.redhat.com> References: <1321021336.4215.15.camel@balmora.brq.redhat.com> Message-ID: <4EBD3DA0.6050908@redhat.com> Martin Kosek wrote: > There are 2 patches improving our zone zonemgr: > > 1) ipa-server-install --zonemgr option validation and normalization + > the same thing in ipa dnszone-add/mod --admin-email. They now allow and > correctly process '.' in a local-part of the zonemgr e-mail (it is > encoded as '\.'. > > How to test: > > ipa-server-install -p secret123 -a secret123 --setup-dns --zonemgr=foo.bar at example.com > OR if e-mail is passed in SOA format: > ipa-server-install -p secret123 -a secret123 --setup-dns --zonemgr='foo\.bar.example.com' > > In both cases, the zonemgr e-mail will be set to correct format: 'foo > \.bar.example.com'. > > 2) Our default zonemgr is changed to follow RFC 2142 recommendation - > hostmaster@ > > hostmaster is an alias to root anyway (see /etc/aliases). > > Martin NACK 166, this illegal address is allowed: foo\.bar\.baz\.com ACK 167 when 166 is ready. rob From abokovoy at redhat.com Fri Nov 11 15:28:10 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 11 Nov 2011 17:28:10 +0200 Subject: [Freeipa-devel] [PATCH] 904 two more buildrequires In-Reply-To: <4EBD3ADA.10006@redhat.com> References: <4EBD3ADA.10006@redhat.com> Message-ID: <20111111152810.GG1762@redhat.com> On Fri, 11 Nov 2011, Rob Crittenden wrote: > We need two more buildrequires, libtalloc-devel and libtevent-devel. ACK but do we need to specify versioning? Do we depend on specific features? I remember there was issue with upgrade of libtalloc/libtevent in Fedora that blocked using samba4. Simo, do you remember details? -- / Alexander Bokovoy From jfenal at gmail.com Fri Nov 11 15:26:15 2011 From: jfenal at gmail.com (=?UTF-8?B?SsOpcsO0bWUgRmVuYWw=?=) Date: Fri, 11 Nov 2011 16:26:15 +0100 Subject: [Freeipa-devel] [PATCH] fix copy/paste Message-ID: Hi all, A trivial patch found during French translation : diff --git a/ipalib/plugins/aci.py b/ipalib/plugins/aci.py index fc5582d..429ae6e 100644 --- a/ipalib/plugins/aci.py +++ b/ipalib/plugins/aci.py @@ -543,7 +543,7 @@ class aci_del(crud.Delete): """ Execute the aci-delete operation. - :param aciname: The name of the ACI being added. + :param aciname: The name of the ACI being deleted. :param kw: unused """ assert 'aciname' not in kw Regards, J. -- J?r?me Fenal - jfenal AT gmail.com - http://fenal.org/ Paris.pm - http://paris.mongueurs.net/ From simo at redhat.com Fri Nov 11 15:30:27 2011 From: simo at redhat.com (Simo Sorce) Date: Fri, 11 Nov 2011 10:30:27 -0500 Subject: [Freeipa-devel] [PATCH] 904 two more buildrequires In-Reply-To: <4EBD3ADA.10006@redhat.com> References: <4EBD3ADA.10006@redhat.com> Message-ID: <1321025427.20140.37.camel@willson.li.ssimo.org> On Fri, 2011-11-11 at 10:10 -0500, Rob Crittenden wrote: > We need two more buildrequires, libtalloc-devel and libtevent-devel. They should be dragged in from samba-4.0-devel If it doesn;t do so it is a samba-4.0 packaging bug. We should not explicitly add the requires to freeipa.spec I think. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Fri Nov 11 15:31:26 2011 From: simo at redhat.com (Simo Sorce) Date: Fri, 11 Nov 2011 10:31:26 -0500 Subject: [Freeipa-devel] [PATCH] 904 two more buildrequires In-Reply-To: <20111111152810.GG1762@redhat.com> References: <4EBD3ADA.10006@redhat.com> <20111111152810.GG1762@redhat.com> Message-ID: <1321025486.20140.38.camel@willson.li.ssimo.org> On Fri, 2011-11-11 at 17:28 +0200, Alexander Bokovoy wrote: > On Fri, 11 Nov 2011, Rob Crittenden wrote: > > We need two more buildrequires, libtalloc-devel and libtevent-devel. > ACK but do we need to specify versioning? Do we depend on specific > features? I remember there was issue with upgrade of > libtalloc/libtevent in Fedora that blocked using samba4. > > Simo, do you remember details? Steve is working on that. Ah Rob I think it is a NACK, due to my other comment. Simo. -- Simo Sorce * Red Hat, Inc * New York From abokovoy at redhat.com Fri Nov 11 15:33:58 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 11 Nov 2011 17:33:58 +0200 Subject: [Freeipa-devel] [PATCH] fix copy/paste In-Reply-To: References: Message-ID: <20111111153358.GH1762@redhat.com> On Fri, 11 Nov 2011, J?r?me Fenal wrote: > A trivial patch found during French translation : > > diff --git a/ipalib/plugins/aci.py b/ipalib/plugins/aci.py > index fc5582d..429ae6e 100644 > --- a/ipalib/plugins/aci.py > +++ b/ipalib/plugins/aci.py > @@ -543,7 +543,7 @@ class aci_del(crud.Delete): > """ > Execute the aci-delete operation. > > - :param aciname: The name of the ACI being added. > + :param aciname: The name of the ACI being deleted. > :param kw: unused > """ > assert 'aciname' not in kw ACK. -- / Alexander Bokovoy From rcritten at redhat.com Fri Nov 11 15:34:14 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 11 Nov 2011 10:34:14 -0500 Subject: [Freeipa-devel] [PATCH] 904 two more buildrequires In-Reply-To: <1321025486.20140.38.camel@willson.li.ssimo.org> References: <4EBD3ADA.10006@redhat.com> <20111111152810.GG1762@redhat.com> <1321025486.20140.38.camel@willson.li.ssimo.org> Message-ID: <4EBD4076.9050308@redhat.com> Simo Sorce wrote: > On Fri, 2011-11-11 at 17:28 +0200, Alexander Bokovoy wrote: >> On Fri, 11 Nov 2011, Rob Crittenden wrote: >>> We need two more buildrequires, libtalloc-devel and libtevent-devel. >> ACK but do we need to specify versioning? Do we depend on specific >> features? I remember there was issue with upgrade of >> libtalloc/libtevent in Fedora that blocked using samba4. >> >> Simo, do you remember details? > > Steve is working on that. > Ah Rob I think it is a NACK, due to my other comment. > > Simo. > Ok, no problem, I thought this might be the case which is why I didn't push it as a one-liner. rob From edewata at redhat.com Fri Nov 11 15:57:29 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Fri, 11 Nov 2011 09:57:29 -0600 Subject: [Freeipa-devel] [PATCH] 310 Updated sample data. In-Reply-To: <4EBCF2D2.9070107@redhat.com> References: <4EBAE25D.8010003@redhat.com> <4EBCF2D2.9070107@redhat.com> Message-ID: <4EBD45E9.9090109@redhat.com> On 11/11/2011 4:02 AM, Petr Vobornik wrote: > ACK Pushed to master. -- Endi S. Dewata From edewata at redhat.com Fri Nov 11 15:58:04 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Fri, 11 Nov 2011 09:58:04 -0600 Subject: [Freeipa-devel] [PATCH] 311 Added paging on search facet. In-Reply-To: <4EBCF3CF.1080807@redhat.com> References: <4EBAE26E.9010302@redhat.com> <4EBCF3CF.1080807@redhat.com> Message-ID: <4EBD460C.8040305@redhat.com> On 11/11/2011 4:07 AM, Petr Vobornik wrote: > On facets where paging is not enabled is still shown pager (because > page_length property is set). This can be misleading. As discussed, this is highly subjective. Ultimately this will be decided by UXD. > Otherwise it is OK. I pushed it to master, we can change this easily later if needed. -- Endi S. Dewata From ohamada at redhat.com Fri Nov 11 16:35:13 2011 From: ohamada at redhat.com (Ondrej Hamada) Date: Fri, 11 Nov 2011 17:35:13 +0100 Subject: [Freeipa-devel] [PATCH] ipa-client-install with --no-sssd option should check for nss_ldap In-Reply-To: <20111111142521.GE1762@redhat.com> References: <4EBD294A.20200@redhat.com> <4EBD2EB7.20203@redhat.com> <20111111142521.GE1762@redhat.com> Message-ID: <4EBD4EC1.9070601@redhat.com> On 11/11/2011 03:25 PM, Alexander Bokovoy wrote: > On Fri, 11 Nov 2011, Rob Crittenden wrote: >> Ondrej Hamada wrote: >>> https://fedorahosted.org/freeipa/ticket/2063 >>> >>> In order to check presence of nss_ldap when installing client with >>> '--no-sssd' option there was added code into ipa-client-install. Check >>> is base on existence of nss_ldap configuration files. This configuration >>> could be in 'etc/ldap.conf', '/etc/nss_ldap.conf' or >>> '/etc/libnss_ldap.conf'. Presence of any of these files is considered as >>> success otherwise failure. >> I think we should check for nslcd.conf as well and report that >> neither nss-ldap nor nss-pam-ldapd are installed. > We have already code in configure_ldap_config() and > configure_nslcd_conf that checks all these different files and after > configuration reports what was configured. > > I would rather did a commonalization of detection instead of > duplicating the code. We can re-use result of detecting what exists > later in configure_{ldap,nslcd}_config(). > I'll do it, but I have question: configure_ldap_config() also checks whether file 'pam_ldap.conf' exists. Is installed pam_ldap package without nss_ldap enough to allow ipa-client installation with --no-sssd option? -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada From rcritten at redhat.com Fri Nov 11 16:57:17 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 11 Nov 2011 11:57:17 -0500 Subject: [Freeipa-devel] [PATCH] 120 Improve DNS record data validation In-Reply-To: <1320857477.11381.26.camel@balmora.brq.redhat.com> References: <1315400728.12548.12.camel@dhcp-25-52.brq.redhat.com> <1315401507.12548.14.camel@dhcp-25-52.brq.redhat.com> <1319026515.9647.13.camel@balmora.brq.redhat.com> <4E9F2742.3020109@redhat.com> <1319533225.14006.13.camel@balmora.brq.redhat.com> <4EB450C4.2060705@redhat.com> <1320857477.11381.26.camel@balmora.brq.redhat.com> Message-ID: <4EBD53ED.9070605@redhat.com> Martin Kosek wrote: > On Fri, 2011-11-04 at 16:53 -0400, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On Wed, 2011-10-19 at 15:38 -0400, Adam Young wrote: >>>> On 10/19/2011 08:15 AM, Martin Kosek wrote: >>>>> On Wed, 2011-09-07 at 15:18 +0200, Martin Kosek wrote: >>>>>> On Wed, 2011-09-07 at 15:05 +0200, Martin Kosek wrote: >>>>>>> This is 3.0 Core Effort Backlog patch. >>>>>>> >>>>>>> The changes to API may look scary, but it should be OK, I just added >>>>>>> validators and normalizers. I found a lot of RR types unsupported by >>>>>>> bind-dyndb-ldap. I implemented a validator telling this information to >>>>>>> the user. I think the message is more user-friendly than the previous >>>>>>> LDAP schema error. >>>>>>> >>>>>>> Enjoy the RFCs! :-) >>>>>>> >>>>>>> Martin >>>>>>> >>>>>>> --- >>>>>>> Implement missing validators for DNS RR types so that we can capture >>>>>>> at least basic user errors. Additionally, a normalizer creating >>>>>>> a fully-qualified domain name has been implemented for several RRs >>>>>>> to prevent this common user error. >>>>>>> >>>>>>> https://fedorahosted.org/freeipa/ticket/1106 >>>>>>> >>>>>> I noticed a typo in format description for LOC record validation. A >>>>>> fixed patch attached. >>>>>> >>>>>> Martin >>>>> Rebased for current master. >>>>> >>>>> This patch is still waiting for review. As I would like to base my next >>>>> DNS work (structured DNS commands) on this patch I would like to have it >>>>> reviewed soon. >>>>> >>>>> Thanks, >>>>> Martin >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Freeipa-devel mailing list >>>>> Freeipa-devel at redhat.com >>>>> https://www.redhat.com/mailman/listinfo/freeipa-devel >>>> >>>> >>>> I've just given it a visual review, but it looks right. Probably >>>> should have some unit tests to go with it for some of the more >>>> commonly used types. >>> >>> Good idea. A, AAAA, NS records are already being checked, I added tests >>> for MX and SRV records too. >>> >>> I also refactored DNS tests a little, there were many repeatedly using >>> hard-coded values (like default zone manager) which would be hard to fix >>> of anything changes. >>> >>> Martin >> >> I can't tell what your intention is with the split for cname and dname >> but it seems to allow just about any value. >> >> I know there are a ton of data types but is it worthwhile to have a >> positive and negative case for each to avoid regressions? >> >> rob > > I just wanted to make sure that user enters one value and not more. This > effectively tests if there is no whitespace in the value, but you are > right that this is a silly way of testing it. > > I rather implemented a hostname validator to common ipalib module > util.py following the RFCs for hostname. It is intended to replace > various hostname validators we use across IPA code. But I will leave > this step for future. For now, I just replaced the dumb validator with > split() for CNAME, DNAME and PTR validators to this new function which > actually validates the hostname value. > > As you advised, I implemented some more unit tests with positive, > negative cases. Now, we should have most of the common types covered (A, > AAAA, NS, MX, SRV, LOC, CNAME, KX, PTR). I did not implement tests for > DNSSEC RR types as I expect some changes there in the future. > > Martin > ACK, pushed to master. rob From abokovoy at redhat.com Fri Nov 11 17:31:51 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 11 Nov 2011 19:31:51 +0200 Subject: [Freeipa-devel] [PATCH] ipa-client-install with --no-sssd option should check for nss_ldap In-Reply-To: <4EBD4EC1.9070601@redhat.com> References: <4EBD294A.20200@redhat.com> <4EBD2EB7.20203@redhat.com> <20111111142521.GE1762@redhat.com> <4EBD4EC1.9070601@redhat.com> Message-ID: <20111111173151.GI1762@redhat.com> On Fri, 11 Nov 2011, Ondrej Hamada wrote: > >>I think we should check for nslcd.conf as well and report that > >>neither nss-ldap nor nss-pam-ldapd are installed. > >We have already code in configure_ldap_config() and > >configure_nslcd_conf that checks all these different files and after > >configuration reports what was configured. > > > >I would rather did a commonalization of detection instead of > >duplicating the code. We can re-use result of detecting what exists > >later in configure_{ldap,nslcd}_config(). > > > I'll do it, but I have question: > configure_ldap_config() also checks whether file 'pam_ldap.conf' > exists. Is installed pam_ldap package without nss_ldap enough to > allow ipa-client installation with --no-sssd option? If you have kerberos setup, then authentication could be done via kerberos and NSS module would give you users and groups with nss_ldap. So pam_ldap + nss_ldap is one of possible configurations, but pam_krb5 + nss_ldap is also possible to use, without pam_ldap. -- / Alexander Bokovoy From dpal at redhat.com Fri Nov 11 18:00:29 2011 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 11 Nov 2011 13:00:29 -0500 Subject: [Freeipa-devel] [PATCH] ipa-client-install with --no-sssd option should check for nss_ldap In-Reply-To: <4EBD4EC1.9070601@redhat.com> References: <4EBD294A.20200@redhat.com> <4EBD2EB7.20203@redhat.com> <20111111142521.GE1762@redhat.com> <4EBD4EC1.9070601@redhat.com> Message-ID: <4EBD62BD.3090700@redhat.com> On 11/11/2011 11:35 AM, Ondrej Hamada wrote: > On 11/11/2011 03:25 PM, Alexander Bokovoy wrote: >> On Fri, 11 Nov 2011, Rob Crittenden wrote: >>> Ondrej Hamada wrote: >>>> https://fedorahosted.org/freeipa/ticket/2063 >>>> >>>> In order to check presence of nss_ldap when installing client with >>>> '--no-sssd' option there was added code into ipa-client-install. Check >>>> is base on existence of nss_ldap configuration files. This >>>> configuration >>>> could be in 'etc/ldap.conf', '/etc/nss_ldap.conf' or >>>> '/etc/libnss_ldap.conf'. Presence of any of these files is >>>> considered as >>>> success otherwise failure. >>> I think we should check for nslcd.conf as well and report that >>> neither nss-ldap nor nss-pam-ldapd are installed. >> We have already code in configure_ldap_config() and >> configure_nslcd_conf that checks all these different files and after >> configuration reports what was configured. >> >> I would rather did a commonalization of detection instead of >> duplicating the code. We can re-use result of detecting what exists >> later in configure_{ldap,nslcd}_config(). >> > I'll do it, but I have question: > configure_ldap_config() also checks whether file 'pam_ldap.conf' > exists. Is installed pam_ldap package without nss_ldap enough to allow > ipa-client installation with --no-sssd option? > pam-ldap by itself is not enough. There should be something for nss. But there are different combinations of packages depending upon RHEL version. -- Thank you, Dmitri Pal Sr. Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Fri Nov 11 18:03:02 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 11 Nov 2011 13:03:02 -0500 Subject: [Freeipa-devel] [PATCH] fix copy/paste In-Reply-To: <20111111153358.GH1762@redhat.com> References: <20111111153358.GH1762@redhat.com> Message-ID: <4EBD6356.904@redhat.com> Alexander Bokovoy wrote: > On Fri, 11 Nov 2011, J?r?me Fenal wrote: >> A trivial patch found during French translation : >> >> diff --git a/ipalib/plugins/aci.py b/ipalib/plugins/aci.py >> index fc5582d..429ae6e 100644 >> --- a/ipalib/plugins/aci.py >> +++ b/ipalib/plugins/aci.py >> @@ -543,7 +543,7 @@ class aci_del(crud.Delete): >> """ >> Execute the aci-delete operation. >> >> - :param aciname: The name of the ACI being added. >> + :param aciname: The name of the ACI being deleted. >> :param kw: unused >> """ >> assert 'aciname' not in kw > ACK. > pushed to master From rcritten at redhat.com Fri Nov 11 18:06:44 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 11 Nov 2011 13:06:44 -0500 Subject: [Freeipa-devel] [PATCH] the 'Keytab:' field in "ipa user-show" output is misleading In-Reply-To: <1320930849.31133.23.camel@balmora.brq.redhat.com> References: <4EB985F8.6060107@redhat.com> <1320917445.31133.19.camel@balmora.brq.redhat.com> <4EBBCB12.8060604@redhat.com> <1320930849.31133.23.camel@balmora.brq.redhat.com> Message-ID: <4EBD6434.9070100@redhat.com> Martin Kosek wrote: > On Thu, 2011-11-10 at 14:01 +0100, Ondrej Hamada wrote: >> On 11/10/2011 10:30 AM, Martin Kosek wrote: >>> On Tue, 2011-11-08 at 20:41 +0100, Ondrej Hamada wrote: >>>> https://fedorahosted.org/freeipa/ticket/1961 >>>> >>>> The 'Keytab' filed in output of all 'user-*' commands was changed to >>>> 'Kerberos keys available'. In order to do this change for 'user-*' >>>> commands only, the flag 'has_keytab' had to be removed from common >>>> output parametrs in ipalib/baseldap.py. This change also affected the >>>> host.py and service.py, where the 'has_keytab' flag was added to their >>>> local output params. Both host.py and service.py holds the old field >>>> caption - 'Keytab' - because of compatibility with older clients. >>>> >>> Ondra, thanks for the patch. It looks OK, everything behaves as >>> expected. >>> >>> I am still concerned about your patch formatting: >>> 1) Patch naming does not follow FreeIPA conventions. You can check >>> others patch file names - mine, Rob's or Alexander's for example. The >>> patch file name should be freeipa-ohamada-2-.patch. The >>> patch number should also be in your mail subject - it helps when >>> searching mails etc. >>> >>> 2) Patch title is wrong - you don't need to include [PATCH] in git >>> commit's title. This then makes it here twice. >>> >>> 3) Patch description is insufficient. I miss link to ticket and some >>> description. You only added it to the mail. When I am traversing FreeIPA >>> git logs, I must be able to quickly read what this patch does. >>> >>> You would have seen all these conventions I wrote you about if you had >>> read some patches in freeipa-devel or had read some in FreeIPA git log. >>> >>> Martin >>> >>> >> Shame on me. Sorry for that. >> >> Corrected patch attached. >> > > ACK. Pushed to master. > > You can celebrate your first upstream patch :-) > > Martin To make it all official I added Ondrej to Contributors.txt in master. rob From rcritten at redhat.com Fri Nov 11 18:29:21 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 11 Nov 2011 13:29:21 -0500 Subject: [Freeipa-devel] [PATCH] 165 Polish ipa config help In-Reply-To: <1320923596.31133.22.camel@balmora.brq.redhat.com> References: <1320923596.31133.22.camel@balmora.brq.redhat.com> Message-ID: <4EBD6981.60106@redhat.com> Martin Kosek wrote: > Remove config-mod options help from config module help to keep the > option doc on one place and without unnecessary redundancy. The new > format is more consistent with the rest of the plugins. Also fix > several inconsistencies in the labels/doc, including: > - using abbreviation when not needed > - using '.' at the end of options help > > https://fedorahosted.org/freeipa/ticket/1906 ack, pushed to master From rcritten at redhat.com Fri Nov 11 18:41:58 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 11 Nov 2011 13:41:58 -0500 Subject: [Freeipa-devel] [PATCH] 162 Fix coverity issues in client CLI tools In-Reply-To: <1320772029.32755.1.camel@balmora.brq.redhat.com> References: <1320772029.32755.1.camel@balmora.brq.redhat.com> Message-ID: <4EBD6C76.9010600@redhat.com> Martin Kosek wrote: > This patch fixes 2 coverity issues: > * ipa-client/config.c: CID 11090: Resource leak > * ipa-client/ipa-getkeytab.c: CID 11018: Unchecked return value > > https://fedorahosted.org/freeipa/ticket/2035 You don't need to test a variable before you free it, so you can just call free(data). Since we are capturing the kerberos error should it be displayed as well in ipa-getkeytab? rob From rcritten at redhat.com Fri Nov 11 19:05:08 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 11 Nov 2011 14:05:08 -0500 Subject: [Freeipa-devel] [PATCH] 160 Hosts file not updated when IP is passed as option In-Reply-To: <1320687767.29771.14.camel@balmora.brq.redhat.com> References: <1320687767.29771.14.camel@balmora.brq.redhat.com> Message-ID: <4EBD71E4.8020708@redhat.com> Martin Kosek wrote: > When an IPA server with unresolvable hostname is being installed, > a hostname record must be inserted to /etc/hosts or the > installation will fail. However, it is not inserted when IP > address is passed as an option (--ip-address) and not > interactively. This patch fixes this so that /etc/hosts > record is inserted in both cases. > > https://fedorahosted.org/freeipa/ticket/2074 ack, pushed to master From rcritten at redhat.com Fri Nov 11 19:07:37 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 11 Nov 2011 14:07:37 -0500 Subject: [Freeipa-devel] [PATCH] 55 Parse comma-separated lists of values in all parameter types In-Reply-To: <4EB7B023.1030004@redhat.com> References: <4EA00403.9060703@redhat.com> <4EA583AB.6030906@redhat.com> <4EA5877B.2020606@redhat.com> <4EA59620.30506@redhat.com> <4EB45837.2020606@redhat.com> <4EB7B023.1030004@redhat.com> Message-ID: <4EBD7279.8030106@redhat.com> Jan Cholasta wrote: > Dne 4.11.2011 22:25, Rob Crittenden napsal(a): >> Jan Cholasta wrote: >>> Dne 24.10.2011 17:42, Rob Crittenden napsal(a): >>>> Jan Cholasta wrote: >>>>> Dne 20.10.2011 13:20, Jan Cholasta napsal(a): >>>>>> Parse comma-separated lists of values in all parameter types. This >>>>>> can >>>>>> enabled for a specific parameter by setting the "csvlist" option to >>>>>> True. >>>>>> >>>>>> Remove List parameter type and replace all occurences with Str with >>>>>> csvlist enabled. >>>>>> >>>>>> https://fedorahosted.org/freeipa/ticket/2007 >>>>>> >>>>>> This change will be useful for >>>>>> https://fedorahosted.org/freeipa/ticket/1487 and >>>>>> https://fedorahosted.org/freeipa/ticket/1847 >>>>>> >>>>>> Unit tests show no regressions. >>>>>> >>>>>> Honza >>>>>> >>>>> >>>>> Self-NACK - I have noticed that the batch command no longer works. >>>>> >>>>> Updated patch attached. >>>>> >>>>> Honza >>>> >>>> What is the benefit of this over the List parameter type? >>>> >>>> rob >>> >>> Mainly because the List parameter type is just a hack. This is the right >>> thing to do if we want to use comma-separated lists of parameters of any >>> type, with all the validation and other parameter type-specific >>> features. >>> >>> For example, I've added a new parameter type for IP addresses in my >>> patch 46 >>> (http://www.redhat.com/archives/freeipa-devel/2011-September/msg00187.html) >>> >>> >>> and use it for A and AAAA DNS records. Without this patch, we can either >>> use List for the record parameters and lose validation in dnsrecord-find >>> (because it is based on crud.Search, which strips all the custom >>> validation rules - like _validate_ipaddr - from the command parameters, >>> which is one of the causes of #1627) or use IPAddress for the record >>> parameters and lose the ability to specify them as comma-separated list >>> of values. With this patch, we can have both comma-separated lists and >>> validation at the same time. >>> >>> Besides, the patch is not as big as it looks like, all the interesting >>> stuff is in ipalib/parameters.py, everything else is just >>> search-and-replace. Also I need it to fix #1487 and #1847 without doing >>> ugly hacks. >>> >>> Honza >>> >> >> I think this would constitute a major version change. > > I'm not sure about that, as the patch doesn't break API compatibility - > a string containing a comma-separated list of values is used for list > parameters both with and without the patch. > >> >> One downside is you can no longer tell in the help with arguments take a >> CSV and which don't. > > Why not? A simple look at csvlist value should provide enough information. > >> >> I think the CSV-related Parameter options should all begin with csv, >> separator and skipspace. > > OK. > >> >> The batch command may eventually be made into a command, how will that >> affect the Any type? > > Batch currently uses List for the "methods" parameter not because of its > CSV capability, but because it doesn't do any type conversion and > validation on the values. This allows it to use values of the form > "{u'params': [args_list, kwargs_dict], u'method': method_name}". The Any > parameter type exists so that this can still be done without List - it's > basically a single-valued version of List (i.e. Any(csvlist=True) is > equivalent to List()). > > IMO if batch is ever made into a command, it would have to be redesigned > not to use List/Any, so that it can be used from CLI with validation of > the parameter values. Any can then be easily removed. > >> >> It otherwise seems to work in my spot-testing. >> >> rob > > Honza > Given that we want to maintain backward API compatibility can you make sure older clients will work with this? My gut tells me it will since this is really a marshaling issue but I don't want to assume. thanks rob From edewata at redhat.com Fri Nov 11 19:59:15 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Fri, 11 Nov 2011 13:59:15 -0600 Subject: [Freeipa-devel] Ticket #1976 - Tab color groups In-Reply-To: <4EBD13C3.7020803@redhat.com> References: <34d9364e-9d93-4c3b-bb5c-bcb4c8601531@zmail05.collab.prod.int.phx2.redhat.com> <4EBC40CC.6060903@redhat.com> <4EBD13C3.7020803@redhat.com> Message-ID: <4EBD7E93.2020002@redhat.com> On 11/11/2011 6:23 AM, Petr Vobornik wrote: > The problem might be changed into two separate questions: > * How to order the facet groups? The order can be set in the entity definition using facet_groups(). It's been used in ACI, DNS, and automount because the default ordering doesn't produce the desired results. > * Which facet show as default when displaying record? Right now it is > the first facet of first facet group (source of the problem). I think it would be unnatural not to show the first facet in the first group, so the default facet needs to be in the first group. The problem is the default facet could be different for every entity, so assigning the same order for everything might not work. On the other hand, changing the order for every entity will make it inconsistent too. >> Should we have a 'summary' tab that displays the most important info, >> then we provide links to go to the tab to edit the info. For example: > May be useful for some users. The summary page is a new facet that will act as the default facet (which is placed first), so we will not need to worry about the order of the other facets. The assumption is most people will open an entry to view some important data only. Not everything is important and they might not need to update anything either. So with a summary page for majority of people things can be done in less mouse clicks. -- Endi S. Dewata From jfenal at gmail.com Sun Nov 13 15:27:06 2011 From: jfenal at gmail.com (=?UTF-8?B?SsOpcsO0bWUgRmVuYWw=?=) Date: Sun, 13 Nov 2011 16:27:06 +0100 Subject: [Freeipa-devel] FreeIPA documentation source on Transifex ? Message-ID: Hi all, Are there plans to push the documentation source (docbook, I guess) on Transifex ? That would help to initiate a translation effort (French being my target here). Regards, J. -- J?r?me Fenal - jfenal at gmail.com - jfenal at redhat.com From mkosek at redhat.com Mon Nov 14 09:30:22 2011 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 14 Nov 2011 10:30:22 +0100 Subject: [Freeipa-devel] [PATCH] 165 Polish ipa config help In-Reply-To: <4EBD6981.60106@redhat.com> References: <1320923596.31133.22.camel@balmora.brq.redhat.com> <4EBD6981.60106@redhat.com> Message-ID: <1321263022.4805.5.camel@balmora.brq.redhat.com> On Fri, 2011-11-11 at 13:29 -0500, Rob Crittenden wrote: > Martin Kosek wrote: > > Remove config-mod options help from config module help to keep the > > option doc on one place and without unnecessary redundancy. The new > > format is more consistent with the rest of the plugins. Also fix > > several inconsistencies in the labels/doc, including: > > - using abbreviation when not needed > > - using '.' at the end of options help > > > > https://fedorahosted.org/freeipa/ticket/1906 > > ack, pushed to master > Unfortunately, one option in the patch was changed after API.txt was generated. Fixed and pushed under one-liner rule. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-168-fix-api.txt.patch Type: text/x-patch Size: 2496 bytes Desc: not available URL: From abokovoy at redhat.com Mon Nov 14 09:32:44 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 14 Nov 2011 11:32:44 +0200 Subject: [Freeipa-devel] [PATCH] 0032 Validate sudo RunAsUser/RunAsGroup arguments Message-ID: <20111114093243.GA312@redhat.com> Hi, FreeIPA SUDO rules use --usercat/--groupcat to specify that rule applies to all users or groups. Thus, sudorule-add-runasuser and sudorule-add-runasgroup accept specific groups and users and do not accept ALL reserved word. The patch validates user and group passed to these commands and reports appropriate errors when these are ALL or all arguments are empty. Ticket #1496 https://fedorahosted.org/freeipa/ticket/1496 One thing I'm not sure about is blocking all variants of the reserved word 'ALL'. The patch blocks them all due to the fact that most likely any of 'all', 'All', 'ALL', 'aLL', and so on are mistyping but there are might be valid cases when group or user is called 'all'. -- / Alexander Bokovoy -------------- next part -------------- >From 726dee0d53736f7ec42569e6f65e112f663a7fb8 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Mon, 14 Nov 2011 11:23:19 +0200 Subject: [PATCH] Validate sudo RunAsUser/RunAsGroup arguments FreeIPA SUDO rules use --usercat/--groupcat to specify that rule applies to all users or groups. Thus, sudorule-add-runasuser and sudorule-add-runasgroup accept specific groups and users and do not accept ALL reserved word. The patch validates user and group passed to these commands and reports appropriate errors when these are ALL or all arguments are empty. Ticket #1496 https://fedorahosted.org/freeipa/ticket/1496 --- ipalib/plugins/sudorule.py | 52 ++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 52 insertions(+), 0 deletions(-) diff --git a/ipalib/plugins/sudorule.py b/ipalib/plugins/sudorule.py index 93ca03f0170d922b91eff45ec2f42871336973f1..b5a3b4352f6e749a02c175efd698341667e3b610 100644 --- a/ipalib/plugins/sudorule.py +++ b/ipalib/plugins/sudorule.py @@ -479,6 +479,35 @@ class sudorule_add_runasuser(LDAPAddMember): member_attributes = ['ipasudorunas'] member_count_out = ('%i object added.', '%i objects added.') + def pre_callback(self, ldap, dn, entry_attrs, attrs_list, *keys, **options): + def check_validity(runas): + v = unicode(runas[0].value) + if v.upper() == u'ALL': + return False + return True + + if 'ipasudorunas' in entry_attrs: + # Map group and user options. If option value is missing, runas_* will + # be set to False. The ordering is group, user and values are + # DNs already so we'll use DN class for extracting actual CN. + (runas_group, runas_user) = map(lambda x: len(x)>0 and x[0], entry_attrs['ipasudorunas'].values()) + if runas_group: + group = DN(runas_group) + if not check_validity(group): + raise errors.ValidationError(name='runas-group', + error=unicode(_("RunAsUser does not accept '%s' as a group name")) % (group[0].value)) + if runas_user: + user = DN(runas_user) + if not check_validity(user): + raise errors.ValidationError(name='runas-user', + error=unicode(_("RunAsUser does not accept '%s' as a user name")) % (user[0].value)) + + if runas_group == False and runas_user == False: + raise errors.ValidationError(name='runas-empty', + error=unicode(_("RunAsUser does not accept empty group and user names"))) + + return dn + def post_callback(self, ldap, completed, failed, dn, entry_attrs, *keys, **options): completed_external = 0 # Sift through the user failures. We assume that these are all @@ -547,6 +576,29 @@ class sudorule_add_runasgroup(LDAPAddMember): member_attributes = ['ipasudorunasgroup'] member_count_out = ('%i object added.', '%i objects added.') + def pre_callback(self, ldap, dn, entry_attrs, attrs_list, *keys, **options): + def check_validity(runas): + v = unicode(runas[0].value) + if v.upper() == u'ALL': + return False + return True + + if 'ipasudorunasgroup' in entry_attrs: + # Extract group DN. If it is empty, runas_group will be set to False. + # We'll use DN class to extract actual CN we are interested in. + runas_group = entry_attrs['ipasudorunasgroup']['group'] + runas_group = len(runas_group)>0 and runas_group[0] + if runas_group: + group = DN(runas_group) + if not check_validity(group): + raise errors.ValidationError(name='runas-group', + error=unicode(_("RunAsGroup does not accept '%s' as a group name")) % (group[0].value)) + else: + raise errors.ValidationError(name='runas-empty', + error=unicode(_("RunAsGroup does not accept empty group name"))) + + return dn + def post_callback(self, ldap, completed, failed, dn, entry_attrs, *keys, **options): completed_external = 0 # Sift through the group failures. We assume that these are all -- 1.7.7.2 From mkosek at redhat.com Mon Nov 14 09:37:09 2011 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 14 Nov 2011 10:37:09 +0100 Subject: [Freeipa-devel] [PATCH] 166+167 Zonemgr improvements In-Reply-To: <4EBD3DA0.6050908@redhat.com> References: <1321021336.4215.15.camel@balmora.brq.redhat.com> <4EBD3DA0.6050908@redhat.com> Message-ID: <1321263429.4805.7.camel@balmora.brq.redhat.com> On Fri, 2011-11-11 at 10:22 -0500, Rob Crittenden wrote: > Martin Kosek wrote: > > There are 2 patches improving our zone zonemgr: > > > > 1) ipa-server-install --zonemgr option validation and normalization + > > the same thing in ipa dnszone-add/mod --admin-email. They now allow and > > correctly process '.' in a local-part of the zonemgr e-mail (it is > > encoded as '\.'. > > > > How to test: > > > > ipa-server-install -p secret123 -a secret123 --setup-dns --zonemgr=foo.bar at example.com > > OR if e-mail is passed in SOA format: > > ipa-server-install -p secret123 -a secret123 --setup-dns --zonemgr='foo\.bar.example.com' > > > > In both cases, the zonemgr e-mail will be set to correct format: 'foo > > \.bar.example.com'. > > > > 2) Our default zonemgr is changed to follow RFC 2142 recommendation - > > hostmaster@ > > > > hostmaster is an alias to root anyway (see /etc/aliases). > > > > Martin > > NACK 166, this illegal address is allowed: foo\.bar\.baz\.com > > ACK 167 when 166 is ready. > > rob Are you sure that you quoted the string in shell properly? It likes to eat backslashes when one is not cautious. The zonemgr value, including backslashes, should then show up in `ipa dnszone-show ZONE'. This is my output: # ipa dnszone-mod example.com --admin-email='foo\.bar\.baz\.com' ipa: ERROR: invalid 'admin_email': address domain is not fully qualified ("example.com" instead of just "example") Anyway, attaching a rebased patch (it collided with my patch 120). Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-166-2-improve-zonemgr-validator-and-normalizer.patch Type: text/x-patch Size: 12637 bytes Desc: not available URL: From mkosek at redhat.com Mon Nov 14 10:04:19 2011 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 14 Nov 2011 11:04:19 +0100 Subject: [Freeipa-devel] [PATCH] 162 Fix coverity issues in client CLI tools In-Reply-To: <4EBD6C76.9010600@redhat.com> References: <1320772029.32755.1.camel@balmora.brq.redhat.com> <4EBD6C76.9010600@redhat.com> Message-ID: <1321265059.4805.14.camel@balmora.brq.redhat.com> On Fri, 2011-11-11 at 13:41 -0500, Rob Crittenden wrote: > Martin Kosek wrote: > > This patch fixes 2 coverity issues: > > * ipa-client/config.c: CID 11090: Resource leak > > * ipa-client/ipa-getkeytab.c: CID 11018: Unchecked return value > > > > https://fedorahosted.org/freeipa/ticket/2035 > > You don't need to test a variable before you free it, so you can just > call free(data). Right, I was over-defensive here. Fixed. > > Since we are capturing the kerberos error should it be displayed as well > in ipa-getkeytab? > > rob > Good idea, it should. Unfortunately, this is not done properly across all ipa-getkeytab.c. ipa-rmkeytab.c does this much better. I added error messages at least to the affected function to keep the patch compact. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-162-2-fix-coverity-issues-in-client-cli-tools.patch Type: text/x-patch Size: 3297 bytes Desc: not available URL: From abokovoy at redhat.com Mon Nov 14 10:48:00 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 14 Nov 2011 12:48:00 +0200 Subject: [Freeipa-devel] [PATCH] 0033 Check all LDAP servers during IPA discovery Message-ID: <20111114104800.GA3927@redhat.com> Hi, This is patch proposal, I haven't checked it with multiple servers setup yet. When discovering IPA LDAP servers through DNS records, look through all servers found until first success. A master might be not available or denied access due to anonymous binds disabled, for example, but replica may succeed. Ticket #1827 https://fedorahosted.org/freeipa/ticket/1827 -- / Alexander Bokovoy -------------- next part -------------- >From 3d4d893dc4631184824add70dfdef5dfd9f331c2 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Mon, 14 Nov 2011 12:39:50 +0200 Subject: [PATCH] Check through all LDAP servers in the domain during IPA discovery When discovering IPA LDAP servers through DNS records, look through all servers found until first success. A master might be not available or denied access but replica may succeed. Ticket #1827 https://fedorahosted.org/freeipa/ticket/1827 --- ipa-client/ipaclient/ipadiscovery.py | 34 +++++++++++++++++++++++++--------- 1 files changed, 25 insertions(+), 9 deletions(-) diff --git a/ipa-client/ipaclient/ipadiscovery.py b/ipa-client/ipaclient/ipadiscovery.py index e786e98d8d722b7ac55db6f616effd737f920364..bb450456e987e01098377e698ca53d6cd29648f9 100644 --- a/ipa-client/ipaclient/ipadiscovery.py +++ b/ipa-client/ipaclient/ipadiscovery.py @@ -25,7 +25,7 @@ import tempfile import ldap from ldap import LDAPError from ipapython.ipautil import run, CalledProcessError, valid_ip, get_ipa_basedn, \ - realm_to_suffix, format_netloc + realm_to_suffix, format_netloc, parse_items NOT_FQDN = -1 @@ -170,19 +170,35 @@ class IPADiscovery: self.kdc = krbret[1] logging.debug("[ipacheckldap]") - # check ldap now - ldapret = self.ipacheckldap(self.server, self.realm) + # We may have received multiple servers corresponding to the domain + # Iterate through all of those to check if it is IPA LDAP server + servers = parse_items(self.server) + ldapret = [NOT_IPA_SERVER] + ldapaccess = True + for server in servers: + # check ldap now + ldapret = self.ipacheckldap(server, self.realm) - if ldapret[0] == 0: - self.server = ldapret[1] - self.realm = ldapret[2] + if ldapret[0] == 0: + self.server = ldapret[1] + self.realm = ldapret[2] + break - if ldapret[0] == NO_ACCESS_TO_LDAP and self.realm is None: + if ldapret[0] == NO_ACCESS_TO_LDAP: + ldapaccess = False + + # If one of LDAP servers checked rejects access (may be anonymous + # bind is disabled), assume realm and basedn generated off domain. + # Note that in case ldapret[0] == 0 and ldapaccess == False (one of + # servers didn't provide access but another one succeeded), self.realm + # will be set already to a proper value above, self.basdn will be + # initialized during the LDAP check itself and we'll skip these two checks. + if not ldapaccess and self.realm is None: # Assume realm is the same as domain.upper() self.realm = self.domain.upper() logging.debug("Assuming realm is the same as domain: %s" % self.realm) - if ldapret[0] == NO_ACCESS_TO_LDAP and self.basedn is None: + if not ldapaccess and self.basedn is None: # Generate suffix from realm self.basedn = realm_to_suffix(self.realm) logging.debug("Generate basedn from realm: %s" % self.basedn) @@ -200,7 +216,7 @@ class IPADiscovery: Errno is an error number: 0 means all ok 1 means we could not check the info in LDAP (may happend when - anonymous binds are siabled) + anonymous binds are disabled) 2 means the server is certainly not an IPA server """ -- 1.7.7.2 From ohamada at redhat.com Mon Nov 14 16:20:34 2011 From: ohamada at redhat.com (Ondrej Hamada) Date: Mon, 14 Nov 2011 17:20:34 +0100 Subject: [Freeipa-devel] [PATCH] ipa-client-install with --no-sssd option should check for nss_ldap In-Reply-To: <4EBD294A.20200@redhat.com> References: <4EBD294A.20200@redhat.com> Message-ID: <4EC13FD2.9000203@redhat.com> On 11/11/2011 02:55 PM, Ondrej Hamada wrote: > https://fedorahosted.org/freeipa/ticket/2063 > > In order to check presence of nss_ldap when installing client with > '--no-sssd' option there was added code into ipa-client-install. Check > is base on existence of nss_ldap configuration files. This > configuration could be in 'etc/ldap.conf', '/etc/nss_ldap.conf' or > '/etc/libnss_ldap.conf'. Presence of any of these files is considered > as success otherwise failure. > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel I've rewritten it. Additionally it checks for existence of nss-pam-ldapd and makes the results reusable by configure_{ldap|nslcd}_conf() functions. https://fedorahosted.org/freeipa/ticket/2063 In order to check presence of nss_ldap or nss-pam-ldapd when installing client with '--no-sssd' option there was added code into ipa-client-install. Checking is based on existence of nss_ldap configuration files. This configuration could be in 'etc/ldap.conf', '/etc/nss_ldap.conf' or '/etc/libnss_ldap.conf'. Optionaly the nss_ldap could cooperate with pam_ldap module and hence the presence of it is checked by looking for 'pam_ldap.conf' file. Existence of nss-pam-ldapd is checked against existence of 'nslcd.conf' file. All this checking is done by function nssldap_exists(). Because both main modules are maintained by two different functions, the function returns tuple containing return code and dictionary structure - its key is name of target function and value is list of existing configuration files. Files to check are specified inside the nssldap_exists() function. In order to fit the returned values, the functions configure_{ldap|nslcd}_conf() were slightly modified. They accept one more parameter which is list of existing files. They are not checking existence of above mentioned files anymore. -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ohamada-3-2-Client-install-checks-for-nss_ldap.patch Type: text/x-patch Size: 7009 bytes Desc: not available URL: From mkosek at redhat.com Mon Nov 14 16:24:15 2011 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 14 Nov 2011 17:24:15 +0100 Subject: [Freeipa-devel] [PATCH] 169 Fix LDAP object parameter encoding Message-ID: <1321287855.4805.27.camel@balmora.brq.redhat.com> Tests are now OK, there were a lot of errors related to permissions and hosts (--random was always on). I was also thinking if the encoding step shouldn't be done as late as possible, i.e. after plugin's pre_callbacks, right before LDAP is called. Now, we do encoding right in the beginning of command processing and therefore plugin pre_callback already see the encoded values. If we choose the late-encoding change, we would have to add the encoding step before every LDAP call in baseldap.py or even before custom LDAP calls in user plugins when execute() is overloaded - or write a wrapper for the LDAP calls. That's why I rather chose the proposed solution. ------------ Parameters in LDAP objects missed an information if they are real LDAP attributes or not. Real LDAP attributes are written to entry_attrs dictionary in plugin callbacks and are being encoded. This causes issues when plugin callbacks does not expect that the parameters values are already encoded for submission to LDAP. This patch introduces a new flag "noattribute" used to mark that a parameter is not an LDAP attribute and thus should not be encoded or added to entry_attrs. https://fedorahosted.org/freeipa/ticket/2097 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-169-fix-ldap-object-parameter-encoding.patch Type: text/x-patch Size: 46640 bytes Desc: not available URL: From edewata at redhat.com Mon Nov 14 16:27:15 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 14 Nov 2011 10:27:15 -0600 Subject: [Freeipa-devel] [PATCH] 312 Refactored permission target section. Message-ID: <4EC14163.4030000@redhat.com> The permission target section has been modified to use widgets to create the target selection and handle multiple fields. Ticket #2098 -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-edewata-0312-Refactored-permission-target-section.patch Type: text/x-patch Size: 31211 bytes Desc: not available URL: From edewata at redhat.com Mon Nov 14 16:37:05 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 14 Nov 2011 10:37:05 -0600 Subject: [Freeipa-devel] [PATCH] 313 Removed develop.js. Message-ID: <4EC143B1.4070903@redhat.com> The develop.js is no longer necessary because the code in it has been merged into the main code. An empty extension.js has been added to provide a place for UI customization. Ticket #2099 -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-edewata-0313-Removed-develop.js.patch Type: text/x-patch Size: 16194 bytes Desc: not available URL: From abokovoy at redhat.com Mon Nov 14 17:10:54 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 14 Nov 2011 19:10:54 +0200 Subject: [Freeipa-devel] [PATCH] 169 Fix LDAP object parameter encoding In-Reply-To: <1321287855.4805.27.camel@balmora.brq.redhat.com> References: <1321287855.4805.27.camel@balmora.brq.redhat.com> Message-ID: <20111114171054.GA14807@redhat.com> On Mon, 14 Nov 2011, Martin Kosek wrote: > Tests are now OK, there were a lot of errors related to permissions and > hosts (--random was always on). > > I was also thinking if the encoding step shouldn't be done as late as > possible, i.e. after plugin's pre_callbacks, right before LDAP is > called. Now, we do encoding right in the beginning of command processing > and therefore plugin pre_callback already see the encoded values. > > If we choose the late-encoding change, we would have to add the encoding > step before every LDAP call in baseldap.py or even before custom LDAP > calls in user plugins when execute() is overloaded - or write a wrapper > for the LDAP calls. That's why I rather chose the proposed solution. I don't like it but on further thinking it seems to unavoidable. One minor suggestion is to select more descriptive name because 'attribute' and 'noattribute' are confusing, to say at least. Or maybe we should document them better? Right now there is no documentation for a majority of those flags. -- / Alexander Bokovoy From edewata at redhat.com Mon Nov 14 21:35:35 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 14 Nov 2011 15:35:35 -0600 Subject: [Freeipa-devel] [PATCH] 313 Removed develop.js. In-Reply-To: <4EC143B1.4070903@redhat.com> References: <4EC143B1.4070903@redhat.com> Message-ID: <4EC189A7.5010202@redhat.com> On 11/14/2011 10:37 AM, Endi Sukma Dewata wrote: > The develop.js is no longer necessary because the code in it has > been merged into the main code. > > An empty extension.js has been added to provide a place for UI > customization. > > Ticket #2099 Replaced the regex with a variable IPA.url for detecting live/static site. By default the IPA.url is set to "/ipa/ui/" which is the standard path for live deployment. If the current path matches the IPA.url the UI will know that it's running on a live server. Otherwise it will use the fixtures. If the demo site needs to use the same path as the live path, the IPA.url can be set to null in the extension.js to force the UI to use fixtures. -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-edewata-0313-2-Removed-develop.js.patch Type: text/x-patch Size: 16328 bytes Desc: not available URL: From jfenal at gmail.com Mon Nov 14 22:03:23 2011 From: jfenal at gmail.com (=?UTF-8?B?SsOpcsO0bWUgRmVuYWw=?=) Date: Mon, 14 Nov 2011 23:03:23 +0100 Subject: [Freeipa-devel] [PATCH] Another trivial doc fix Message-ID: diff --git a/ipalib/plugins/group.py b/ipalib/plugins/group.py index cd4a054..4bf77f8 100644 --- a/ipalib/plugins/group.py +++ b/ipalib/plugins/group.py @@ -28,8 +28,8 @@ Groups of users Manage groups of users. By default, new groups are POSIX groups. You can add the --nonposix option to the group-add command to mark a new group -as non-POSIX, and you can use the same argument to the group-mod command -to convert a non-POSIX group to a POSIX group. POSIX groups cannot be +as non-POSIX. You can use the --posix argument to the group-mod command +to convert a non-POSIX group into a POSIX group. POSIX groups cannot be converted to non-POSIX groups. Every group must have a description. -- J?r?me Fenal - jfenal AT gmail.com - http://fenal.org/ Paris.pm - http://paris.mongueurs.net/ From ayoung at redhat.com Mon Nov 14 22:23:17 2011 From: ayoung at redhat.com (Adam Young) Date: Mon, 14 Nov 2011 17:23:17 -0500 Subject: [Freeipa-devel] [PATCH] 312 Refactored permission target section. In-Reply-To: <4EC14163.4030000@redhat.com> References: <4EC14163.4030000@redhat.com> Message-ID: <4EC194D5.3070603@redhat.com> On 11/14/2011 11:27 AM, Endi Sukma Dewata wrote: > The permission target section has been modified to use widgets > to create the target selection and handle multiple fields. > > Ticket #2098 > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel ACK. Pushed to master -------------- next part -------------- An HTML attachment was scrubbed... URL: From ayoung at redhat.com Mon Nov 14 22:23:46 2011 From: ayoung at redhat.com (Adam Young) Date: Mon, 14 Nov 2011 17:23:46 -0500 Subject: [Freeipa-devel] [PATCH] 313 Removed develop.js. In-Reply-To: <4EC189A7.5010202@redhat.com> References: <4EC143B1.4070903@redhat.com> <4EC189A7.5010202@redhat.com> Message-ID: <4EC194F2.1090607@redhat.com> On 11/14/2011 04:35 PM, Endi Sukma Dewata wrote: > On 11/14/2011 10:37 AM, Endi Sukma Dewata wrote: >> The develop.js is no longer necessary because the code in it has >> been merged into the main code. >> >> An empty extension.js has been added to provide a place for UI >> customization. >> >> Ticket #2099 > > Replaced the regex with a variable IPA.url for detecting live/static > site. By default the IPA.url is set to "/ipa/ui/" which is the > standard path for live deployment. If the current path matches the > IPA.url the UI will know that it's running on a live server. Otherwise > it will use the fixtures. > > If the demo site needs to use the same path as the live path, the > IPA.url can be set to null in the extension.js to force the UI to use > fixtures. > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel ACK (version 2) and pushed to master -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcritten at redhat.com Tue Nov 15 03:32:12 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 14 Nov 2011 22:32:12 -0500 Subject: [Freeipa-devel] [PATCH] 880 don't check for existing 389-ds instances In-Reply-To: <1319720203.26374.17.camel@balmora.brq.redhat.com> References: <4E7A369B.1030704@redhat.com> <4EA1C7FF.5010300@redhat.com> <1319720203.26374.17.camel@balmora.brq.redhat.com> Message-ID: <4EC1DD3C.6080003@redhat.com> Martin Kosek wrote: > On Fri, 2011-10-21 at 15:29 -0400, Rob Crittenden wrote: >> Rob Crittenden wrote: >>> We don't need to prohibit existing 389-ds instances when installing IPA, >>> just that the ports we need are available. Remove this check. >>> >>> For master only. >>> >>> rob >> >> Re-based patch against master. >> >> rob > > Works ok, I was able to install IPA server next to other dirsrv sitting > on another port. The installation could get to trouble if another DS > would sit on a standard port but would be stopped at the moment, i.e. > ports 389 and 636 would be free. But I think this is a risk we can take. > > My only concern is with uninstallation - we shutdown all dirsrv > instances during the process. Thus, in the end the custom dirsrv > instance remains stopped. > > Martin We track state of existing instance installations and restart them if necessary (though there was a bug preventing this, my patch fixes it). Instances installed afterward are another matter. This is about the best we can do. We do our best to restore state but we can only take it so far. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-880-3-dsinstance.patch Type: text/x-patch Size: 5450 bytes Desc: not available URL: From rcritten at redhat.com Tue Nov 15 03:40:53 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 14 Nov 2011 22:40:53 -0500 Subject: [Freeipa-devel] [PATCH] #1794 - Speed up replica setup In-Reply-To: References: <1317673256.17819.13.camel@willson.li.ssimo.org> <1317680239.17819.19.camel@willson.li.ssimo.org> <1318011288.31149.10.camel@willson.li.ssimo.org> Message-ID: <4EC1DF45.80307@redhat.com> JR Aquino wrote: > On Oct 7, 2011, at 11:14 AM, Simo Sorce wrote: > >> On Mon, 2011-10-03 at 18:17 -0400, Simo Sorce wrote: >>> On Mon, 2011-10-03 at 16:20 -0400, Simo Sorce wrote: >>>> Newer 389ds servers have a new option to have a different set of >>>> filtered attributes from normal replication. >>>> >>>> This has been added in order to allow DS to replicate memberof >>>> attributes only during a total update so that we do not need to run a >>>> fixup memberof task on a replica at install time. >>>> This task is quite inefficient for big database and can take a long >>>> time. By replicating memberof while the DB is locked we are guaranteed >>>> the memberof list is consistent so we do not need a fixup. >>>> >>>> This patch allows to enable this feature dynamically. If the server does >>>> not yet support the new option it falls back to the previous behavior. >>>> >>>> Fixes: https://fedorahosted.org/freeipa/ticket/1794 >>>> >>>> I am sending the patch but it has been jointly developed at various >>>> stages by Nathan, JR, and me. >>>> >>>> Simo. >>> >>> After some thinking I found out that we cannot commit this patch until >>> the memberof plugin is converted to use the new transaction interfaces >>> for plugins, as otherwise it is possible to run into race conditions >>> where the member/memberof relations are not settled if a new replica is >>> installed while member attributes are being changed. >>> >>> Granted the race is quite small and unlikely but real. >>> So please test and ack it, but we need to defer pushing to stable >>> branches until ds copes. >>> I think it is ok to push to master for testing, DS should have the >>> necessary support by the time we make another stable release from master >>> and in our test environments I am sure we will never hit the race. >> >> After some more testing I found a small bug that can cause issues in >> some conditions, new patch attached. >> >> Simo. > > ACK with 389-ds-base-1.2.10-0.4.a4 Pushed to master From mkosek at redhat.com Tue Nov 15 10:07:13 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 Nov 2011 11:07:13 +0100 Subject: [Freeipa-devel] [PATCH] 169 Fix LDAP object parameter encoding In-Reply-To: <20111114171054.GA14807@redhat.com> References: <1321287855.4805.27.camel@balmora.brq.redhat.com> <20111114171054.GA14807@redhat.com> Message-ID: <1321351633.19993.3.camel@balmora.brq.redhat.com> On Mon, 2011-11-14 at 19:10 +0200, Alexander Bokovoy wrote: > On Mon, 14 Nov 2011, Martin Kosek wrote: > > Tests are now OK, there were a lot of errors related to permissions and > > hosts (--random was always on). > > > > I was also thinking if the encoding step shouldn't be done as late as > > possible, i.e. after plugin's pre_callbacks, right before LDAP is > > called. Now, we do encoding right in the beginning of command processing > > and therefore plugin pre_callback already see the encoded values. > > > > If we choose the late-encoding change, we would have to add the encoding > > step before every LDAP call in baseldap.py or even before custom LDAP > > calls in user plugins when execute() is overloaded - or write a wrapper > > for the LDAP calls. That's why I rather chose the proposed solution. > I don't like it but on further thinking it seems to unavoidable. > One minor suggestion is to select more descriptive name because > 'attribute' and 'noattribute' are confusing, to say at least. > > Or maybe we should document them better? Right now there is no > documentation for a majority of those flags. > I renamed it to 'virtual_attribute' (as an opposite to physical LDAP attribute), I hope it makes more sense. You are right that we miss a documentation for most of Param attributes and classes which may confuse especially newcomers to FreeIPA project. I added a quick overview to Param class doc. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-169-2-fix-ldap-object-parameter-encoding.patch Type: text/x-patch Size: 51382 bytes Desc: not available URL: From abokovoy at redhat.com Tue Nov 15 10:33:55 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 15 Nov 2011 12:33:55 +0200 Subject: [Freeipa-devel] [PATCH] 169 Fix LDAP object parameter encoding In-Reply-To: <1321351633.19993.3.camel@balmora.brq.redhat.com> References: <1321287855.4805.27.camel@balmora.brq.redhat.com> <20111114171054.GA14807@redhat.com> <1321351633.19993.3.camel@balmora.brq.redhat.com> Message-ID: <20111115103355.GB27400@redhat.com> On Tue, 15 Nov 2011, Martin Kosek wrote: > > Or maybe we should document them better? Right now there is no > > documentation for a majority of those flags. > > > > I renamed it to 'virtual_attribute' (as an opposite to physical LDAP > attribute), I hope it makes more sense. > > You are right that we miss a documentation for most of Param attributes > and classes which may confuse especially newcomers to FreeIPA project. I > added a quick overview to Param class doc. ACK for the bulk of changes except API.txt. I know, it is autogenerated but can we do something with API.txt to not leak attributes that make no difference to the client side? For example, the fact that certain param is virtual or LDAP attribute is irrelevant on the client side, so maybe this detail could be ignored in API.txt? We've raised this topic before regarding label names changes which force API break as well but as this patch shows, we'll get a lot of trouble on compatibility without filtering the attributes that don't really matter on the client side... -- / Alexander Bokovoy From mkosek at redhat.com Tue Nov 15 12:20:39 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 Nov 2011 13:20:39 +0100 Subject: [Freeipa-devel] [PATCH] 169 Fix LDAP object parameter encoding In-Reply-To: <20111115103355.GB27400@redhat.com> References: <1321287855.4805.27.camel@balmora.brq.redhat.com> <20111114171054.GA14807@redhat.com> <1321351633.19993.3.camel@balmora.brq.redhat.com> <20111115103355.GB27400@redhat.com> Message-ID: <1321359639.19993.5.camel@balmora.brq.redhat.com> On Tue, 2011-11-15 at 12:33 +0200, Alexander Bokovoy wrote: > On Tue, 15 Nov 2011, Martin Kosek wrote: > > > Or maybe we should document them better? Right now there is no > > > documentation for a majority of those flags. > > > > > > > I renamed it to 'virtual_attribute' (as an opposite to physical LDAP > > attribute), I hope it makes more sense. > > > > You are right that we miss a documentation for most of Param attributes > > and classes which may confuse especially newcomers to FreeIPA project. I > > added a quick overview to Param class doc. > ACK for the bulk of changes except API.txt. I know, it is > autogenerated but can we do something with API.txt to not leak > attributes that make no difference to the client side? For example, > the fact that certain param is virtual or LDAP attribute is irrelevant > on the client side, so maybe this detail could be ignored in API.txt? > > We've raised this topic before regarding label names changes which > force API break as well but as this patch shows, we'll get a lot of > trouble on compatibility without filtering the attributes that don't > really matter on the client side... Pushed to master. As discussed on IRC, I filed a ticket for the API filtering part: https://fedorahosted.org/freeipa/ticket/2097 Martin From mkosek at redhat.com Tue Nov 15 14:06:16 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 Nov 2011 15:06:16 +0100 Subject: [Freeipa-devel] [PATCH] 170 Remove redundant information from API.txt Message-ID: <1321365976.19993.12.camel@balmora.brq.redhat.com> Some Param or Output attributes do not cause API incompatibility (e.g. doc, label or callables) and does not need to be included in API.txt. When these attributes are modified, a lot of bogus changes may get in API.txt - making the real API changes less detectable. https://fedorahosted.org/freeipa/ticket/2107 ------------------- The list of ignored attributes can be further discussed. I left just the attributes that IMO really influences the Param/Output behavior and that we should be cautious when they are changed. I had to gzip the patch as the API.txt change is "quite" large. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-170-remove-redundant-information-from-API.txt.patch.tgz Type: application/x-compressed-tar Size: 23861 bytes Desc: not available URL: From abokovoy at redhat.com Tue Nov 15 15:42:38 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 15 Nov 2011 17:42:38 +0200 Subject: [Freeipa-devel] [PATCH] 170 Remove redundant information from API.txt In-Reply-To: <1321365976.19993.12.camel@balmora.brq.redhat.com> References: <1321365976.19993.12.camel@balmora.brq.redhat.com> Message-ID: <20111115154237.GA12639@redhat.com> On Tue, 15 Nov 2011, Martin Kosek wrote: > Some Param or Output attributes do not cause API incompatibility > (e.g. doc, label or callables) and does not need to be included > in API.txt. When these attributes are modified, a lot of bogus > changes may get in API.txt - making the real API changes less > detectable. > > https://fedorahosted.org/freeipa/ticket/2107 > > ------------------- > > The list of ignored attributes can be further discussed. I left just the > attributes that IMO really influences the Param/Output behavior and that > we should be cautious when they are changed. > > I had to gzip the patch as the API.txt change is "quite" large. ACK, the makeapi patch looks good to me. You didn't need to use tar, though, gzip would have been enough. Now real question is how to achieve fine-grained compatibility with older releases which I'll look at in ticket #2026. -- / Alexander Bokovoy From mkosek at redhat.com Tue Nov 15 17:39:37 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 Nov 2011 18:39:37 +0100 Subject: [Freeipa-devel] [PATCH] 170 Remove redundant information from API.txt In-Reply-To: <20111115154237.GA12639@redhat.com> References: <1321365976.19993.12.camel@balmora.brq.redhat.com> <20111115154237.GA12639@redhat.com> Message-ID: <1321378777.32444.2.camel@balmora.brq.redhat.com> On Tue, 2011-11-15 at 17:42 +0200, Alexander Bokovoy wrote: > On Tue, 15 Nov 2011, Martin Kosek wrote: > > > Some Param or Output attributes do not cause API incompatibility > > (e.g. doc, label or callables) and does not need to be included > > in API.txt. When these attributes are modified, a lot of bogus > > changes may get in API.txt - making the real API changes less > > detectable. > > > > https://fedorahosted.org/freeipa/ticket/2107 > > > > ------------------- > > > > The list of ignored attributes can be further discussed. I left just the > > attributes that IMO really influences the Param/Output behavior and that > > we should be cautious when they are changed. > > > > I had to gzip the patch as the API.txt change is "quite" large. > ACK, the makeapi patch looks good to me. You didn't need to use tar, > though, gzip would have been enough. Right, my fingers just used the command they are used to. I will make sure they don't do it next time :-) Pushed to master. > > Now real question is how to achieve fine-grained compatibility with > older releases which I'll look at in ticket #2026. Yup. Martin From rcritten at redhat.com Tue Nov 15 19:10:43 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 15 Nov 2011 14:10:43 -0500 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <4EBA9066.4070600@redhat.com> References: <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <4EB152A7.5010303@redhat.com> <20111102151114.GD14373@redhat.com> <4EB15E73.8070402@redhat.com> <20111102163720.GA23092@redhat.com> <20111102163848.GB23092@redhat.com> <4EB2614E.8090307@redhat.com> <20111103114352.GB26667@redhat.com> <4EBA54D1.4040700@redhat.com> <20111109105904.GA5529@redhat.com> <4EBA9066.4070600@redhat.com> Message-ID: <4EC2B933.8030902@redhat.com> Jan Cholasta wrote: > Dne 9.11.2011 11:59, Alexander Bokovoy napsal(a): >> On Wed, 09 Nov 2011, Jan Cholasta wrote: >>>>> would't suffer from a facelift - currently it is messy, hard to read >>>>> code, with bits nobody ever used or uses anymore, some of the >>>>> docstrings and comments aren't correct, etc.) >>>> A review of API class is a good idea. However, I think it is currently >>>> enough what is in proposed patch as it gets you 2-3x speed increase >>>> already. >>> >>> I've already started experimenting with the API class, hopefully it >>> will result in something useful :) >> Good. >> >>>> This is something I'd like to keep as it has great value for all >>>> end-users and it requires loading all Python files in ipalib/plugins >>>> (and in ipaserver/plugins for server side). >>> >>> I'm not suggesting we should skip importing the modules. The plugin >>> initialization process consists of these 3 steps: >>> >>> 1. import plugin modules >>> 2. create plugin instances >>> 3. finalize plugin instances >>> >>> What I'm suggesting is that we do steps 2 and 3 on-demand. We can't >>> do step 1 on-demand, because we have no way of knowing what plugins >>> are available without importing the modules. (Both your and my patch >>> does only step 3 on-demand, but I think we can do better than that.) >> Agreed. >> >>>>> Anyway, if we decide to go with your solution, please make sure >>>>> *all* the plugins that are used are finalized (because of the >>>>> locking) and add a new api.env attribute which controls the lazy >>>>> finalization, so that the behavior can be overriden (actually I have >>>>> already done that - see attached patch - just use >>>>> "api.env.plugins_on_demand" instead of "api.env.context == 'cli'"). >>>> Done. >>>> >>> >>> + if not self.__dict__['_Plugin__finalized']: >>> + self.finalize() >>> >>> This doesn't seem right, coding style-wise. If you want to access >>> the property from outside the Plugin class, why did you name-mangle >>> it in the first place? >> It is supposed to be internal detail of a Plugin. I agree it stinks a >> bit here. :) >> >> >>> def finalize(self): >>> """ >>> """ >>> + if not self.__finalized: >>> + self.__finalized = True >>> + else: >>> + return >>> if not is_production_mode(self): >>> lock(self) >>> >>> Finalize is supposed to be called just once, so IMO it would be >>> better not to do the check and let it throw the exception. >> On contrary, I think sequential finalize() calls should do nothing. >> This is at least what I expect from a finalized object -- no-op. > > In my patch, finalize() throws an exception if called more than once, > but ensure_finalized() does nothing for plugins that are already > finalized. So there are both kinds of behavior available. > >> >>> + for i in ('Object', 'Property', 'Backend'): >>> + if i in self: >>> + namespaces.append(self[i]) >>> >>> AFAIK the framework is supposed to be generally usable for other >>> projects in the future. With that in mind, I think we can't afford >>> to hard-code things like this. >> That's true. As you managed to get around without hardcoding, we can >> drop this part. >> >>> Anyway, I've made a patch that uses data descriptors for the trap >>> objects (which is IMHO more elegant than what you do). I've also >>> managed to add on-demand finalization to Object and Attribute >>> subclasses by moving the set-up code from set_api() to finalize() >>> (it doesn't add much performance, though). See attachment. >> I read through the code and I like it! Did you get the whole test >> suite running with it? There are some parts of tests that expect >> non-finalized objects to have None properties while they are now not >> None. > > I always run the test suite ;-) > > All the unit tests succeed (they do when run with my patch 54 applied, > which fixes failures of some unrelated unit tests). > >> >> If so, preliminary ACK for your patch from reading it if you would >> make sure on_finalize() allows multiple calls and would make a better >> commit message. ;) > > on_finalize() shouldn't be called directly (perhaps I should rename it > to _on_finalize to emphasize that...?) > > I'll work on the commit message. As usual, it is the hardest part of the > patch for me. > >> >> I'll try to arrange testing myself today/tomorrow. >> >>> The numbers from my VM: >>> >>> master abbra jcholast >>> $ time ipa >>> real 0m1.288s 0m0.619s 0m0.601s >>> user 0m1.103s 0m0.478s 0m0.451s >>> sys 0m0.161s 0m0.126s 0m0.133s >>> >>> $ time ipa user-find >>> real 0m1.897s 0m1.253s 0m1.235s >>> user 0m1.119s 0m0.486s 0m0.488s >>> sys 0m0.160s 0m0.160s 0m0.136s >>> >>> $ time ipa help >>> real 0m1.299s 0m0.629s 0m0.600s >>> user 0m1.094s 0m0.477s 0m0.446s >>> sys 0m0.183s 0m0.136s 0m0.140s >> Looks good (your VM is supposedly at the same farm as mine so numbers >> are comparable). There is anyway great variation in execution times in >> VMs so 600ms vs 629ms is roughly the same. > > Yep. > >> >> Thanks a lot! I think it is great advance over the original approach. > > Thanks for the kind words :-) > > Honza > Just a couple of questions/clarifications: 1. I think more documentation is needed for on_finalize(). The name isn't particularly descriptive. If I read it properly you are providing an alternate way to override finalize(), right? 2. Changing to a partition in Attribute is not equivalent to the regular expression previously used. Why loosen the requirements? 3. I think you should document in-line where you have block calls to finalize_attr() so that future developers know to add the stub. 4. What is the purpose of the read-lock in finalize? It generally looks good and provides impressive performance improvements. rob From rcritten at redhat.com Tue Nov 15 19:12:09 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 15 Nov 2011 14:12:09 -0500 Subject: [Freeipa-devel] [PATCH] 155 Fix ipa-managed-entries bind procedure In-Reply-To: <1319551888.14006.21.camel@balmora.brq.redhat.com> References: <1319551888.14006.21.camel@balmora.brq.redhat.com> Message-ID: <4EC2B989.8010402@redhat.com> Martin Kosek wrote: > Make sure that when Directory Manager password is entered, > we directly do a simple bind instead of trying binding via GSSAPI. > Also capture ldap.INVALID_CREDENTIALS exception and provide nice > error message than crash. > > https://fedorahosted.org/freeipa/ticket/1927 ACK From jdennis at redhat.com Tue Nov 15 19:38:21 2011 From: jdennis at redhat.com (John Dennis) Date: Tue, 15 Nov 2011 14:38:21 -0500 Subject: [Freeipa-devel] [PATCH 52/52] ticket 2022 - Add log manager module Message-ID: <4EC2BFAD.3030107@redhat.com> ticket 2022 - Add log manager module Browseable online HTML doc can be found here, probably easier to read than the patch (plus it includes the source code) http://jdennis.fedorapeople.org/log_manager_doc/ -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: 0052-ticket-2022-Add-log-manager-module.patch Type: text/x-patch Size: 71603 bytes Desc: not available URL: From jdennis at redhat.com Tue Nov 15 19:41:12 2011 From: jdennis at redhat.com (John Dennis) Date: Tue, 15 Nov 2011 14:41:12 -0500 Subject: [Freeipa-devel] [PATCH 53/53] ticket 2022 - modify codebase to utilize IPALogManager, obsoletes logging Message-ID: <201111151941.pAFJfC5u017345@int-mx09.intmail.prod.int.phx2.redhat.com> -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: 0053-ticket-2022-modify-codebase-to-utilize-IPALogManager.patch Type: text/x-patch Size: 148829 bytes Desc: not available URL: From mkosek at redhat.com Wed Nov 16 07:54:57 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 16 Nov 2011 08:54:57 +0100 Subject: [Freeipa-devel] [PATCH] 155 Fix ipa-managed-entries bind procedure In-Reply-To: <4EC2B989.8010402@redhat.com> References: <1319551888.14006.21.camel@balmora.brq.redhat.com> <4EC2B989.8010402@redhat.com> Message-ID: <1321430097.18210.0.camel@balmora.brq.redhat.com> On Tue, 2011-11-15 at 14:12 -0500, Rob Crittenden wrote: > Martin Kosek wrote: > > Make sure that when Directory Manager password is entered, > > we directly do a simple bind instead of trying binding via GSSAPI. > > Also capture ldap.INVALID_CREDENTIALS exception and provide nice > > error message than crash. > > > > https://fedorahosted.org/freeipa/ticket/1927 > > ACK > Pushed to master, ipa-2-1. Martin From mkosek at redhat.com Wed Nov 16 08:41:29 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 16 Nov 2011 09:41:29 +0100 Subject: [Freeipa-devel] [PATCH] 171 Let PublicError accept Gettext objects Message-ID: <1321432889.18210.5.camel@balmora.brq.redhat.com> Make sure that PublicError does crash when it receives Gettext/NGettext object. Instead of throwing a type error, do the translation to receive the required unicode text. https://fedorahosted.org/freeipa/ticket/2096 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-171-let-publicerror-accept-gettext-objects.patch Type: text/x-patch Size: 1386 bytes Desc: not available URL: From mkosek at redhat.com Wed Nov 16 09:32:01 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 16 Nov 2011 10:32:01 +0100 Subject: [Freeipa-devel] [PATCH] 880 don't check for existing 389-ds instances In-Reply-To: <4EC1DD3C.6080003@redhat.com> References: <4E7A369B.1030704@redhat.com> <4EA1C7FF.5010300@redhat.com> <1319720203.26374.17.camel@balmora.brq.redhat.com> <4EC1DD3C.6080003@redhat.com> Message-ID: <1321435921.18210.7.camel@balmora.brq.redhat.com> On Mon, 2011-11-14 at 22:32 -0500, Rob Crittenden wrote: > Martin Kosek wrote: > > On Fri, 2011-10-21 at 15:29 -0400, Rob Crittenden wrote: > >> Rob Crittenden wrote: > >>> We don't need to prohibit existing 389-ds instances when installing IPA, > >>> just that the ports we need are available. Remove this check. > >>> > >>> For master only. > >>> > >>> rob > >> > >> Re-based patch against master. > >> > >> rob > > > > Works ok, I was able to install IPA server next to other dirsrv sitting > > on another port. The installation could get to trouble if another DS > > would sit on a standard port but would be stopped at the moment, i.e. > > ports 389 and 636 would be free. But I think this is a risk we can take. > > > > My only concern is with uninstallation - we shutdown all dirsrv > > instances during the process. Thus, in the end the custom dirsrv > > instance remains stopped. > > > > Martin > > We track state of existing instance installations and restart them if > necessary (though there was a bug preventing this, my patch fixes it). > > Instances installed afterward are another matter. This is about the best > we can do. We do our best to restore state but we can only take it so far. > > rob > ACK. Pushed to master. Martin From mkosek at redhat.com Wed Nov 16 09:51:25 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 16 Nov 2011 10:51:25 +0100 Subject: [Freeipa-devel] [PATCH] 171 Let PublicError accept Gettext objects In-Reply-To: <1321432889.18210.5.camel@balmora.brq.redhat.com> References: <1321432889.18210.5.camel@balmora.brq.redhat.com> Message-ID: <1321437085.18210.10.camel@balmora.brq.redhat.com> Commit message amended: s/does crash/does not crash/. This patch really does not intend to cause crashes :-) Martin On Wed, 2011-11-16 at 09:41 +0100, Martin Kosek wrote: > Make sure that PublicError does crash when it receives > Gettext/NGettext object. Instead of throwing a type error, do the > translation to receive the required unicode text. > > https://fedorahosted.org/freeipa/ticket/2096 > -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-171-2-let-publicerror-accept-gettext-objects.patch Type: text/x-patch Size: 1390 bytes Desc: not available URL: From abokovoy at redhat.com Wed Nov 16 09:55:17 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 16 Nov 2011 11:55:17 +0200 Subject: [Freeipa-devel] [PATCH] 171 Let PublicError accept Gettext objects In-Reply-To: <1321437085.18210.10.camel@balmora.brq.redhat.com> References: <1321432889.18210.5.camel@balmora.brq.redhat.com> <1321437085.18210.10.camel@balmora.brq.redhat.com> Message-ID: <20111116095517.GA11138@redhat.com> On Wed, 16 Nov 2011, Martin Kosek wrote: > Commit message amended: s/does crash/does not crash/. > > This patch really does not intend to cause crashes :-) > > Martin > > On Wed, 2011-11-16 at 09:41 +0100, Martin Kosek wrote: > > Make sure that PublicError does crash when it receives > > Gettext/NGettext object. Instead of throwing a type error, do the > > translation to receive the required unicode text. > > > > https://fedorahosted.org/freeipa/ticket/2096 ACK now. :) -- / Alexander Bokovoy From mkosek at redhat.com Wed Nov 16 10:01:51 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 16 Nov 2011 11:01:51 +0100 Subject: [Freeipa-devel] [PATCH] 171 Let PublicError accept Gettext objects In-Reply-To: <20111116095517.GA11138@redhat.com> References: <1321432889.18210.5.camel@balmora.brq.redhat.com> <1321437085.18210.10.camel@balmora.brq.redhat.com> <20111116095517.GA11138@redhat.com> Message-ID: <1321437711.18210.11.camel@balmora.brq.redhat.com> On Wed, 2011-11-16 at 11:55 +0200, Alexander Bokovoy wrote: > On Wed, 16 Nov 2011, Martin Kosek wrote: > > > Commit message amended: s/does crash/does not crash/. > > > > This patch really does not intend to cause crashes :-) > > > > Martin > > > > On Wed, 2011-11-16 at 09:41 +0100, Martin Kosek wrote: > > > Make sure that PublicError does crash when it receives > > > Gettext/NGettext object. Instead of throwing a type error, do the > > > translation to receive the required unicode text. > > > > > > https://fedorahosted.org/freeipa/ticket/2096 > ACK now. :) > Thanks :-) Pushed to master, ipa-2-1. Martin From mkosek at redhat.com Wed Nov 16 12:35:53 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 16 Nov 2011 13:35:53 +0100 Subject: [Freeipa-devel] [PATCH 53/53] ticket 2022 - modify codebase to utilize IPALogManager, obsoletes logging In-Reply-To: <201111151941.pAFJfC5u017345@int-mx09.intmail.prod.int.phx2.redhat.com> References: <201111151941.pAFJfC5u017345@int-mx09.intmail.prod.int.phx2.redhat.com> Message-ID: <1321446953.18210.31.camel@balmora.brq.redhat.com> On Tue, 2011-11-15 at 14:41 -0500, John Dennis wrote: > > -- > John Dennis > > Looking to carve out IT costs? > www.redhat.com/carveoutcosts/ Hi John, thanks for your patch, that was a fair amount of work there :-) I tested them both and have few comments: 1) Patch 53 will need rebasing 2) Logging for install tools (ipa-server-install, ...) is now broken. DEBUG level messages should get to /var/log/ipaserver-install.log even when the installer is run without --debug. My ipaserver-install.log/ipaserver-uninstall.log was silent when i run just `ipa-server-install' without --debug flag 3) Patch 53 touches most of our Python sources as it changes all logging.* calls to log_mgr.root_logger.* calls. I think this may cause problems (comment 1 is a good example). For example we may not be able to cherry-pick most of new patches from master branch to ipa-2-1 branch or other branches that do not use log_mgr. Additionally, typing "log_mgr.root_logger.*" whenever I want to log anything seems a bit awkward to me. These issues are avoidable, however. Since all our Python code use just the root_logger, can we just simply export root logger in log_manager.py namespace and that import it as "logging" in other modules? Then we would save *a lot* of changes across all our code. Something like this: log_manager.py: --------------- log_mgr = IPALogManager() log_mgr.configure(dict(default_level='error', handlers=[dict(name='console', stream=sys.stderr)]), configure_state='default') +root_logger = log_mgr.root_logger ipa-server-install: ------------------- +from ipapython.log_manager import root_logger as logginng ... # now, all logging will work as usual: logging.error('Some files have not been restored, see /var/lib/ipa/sysrestore/sysrestore.index') Martin From rcritten at redhat.com Wed Nov 16 14:15:44 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 16 Nov 2011 09:15:44 -0500 Subject: [Freeipa-devel] [PATCH] 171 Let PublicError accept Gettext objects In-Reply-To: <1321432889.18210.5.camel@balmora.brq.redhat.com> References: <1321432889.18210.5.camel@balmora.brq.redhat.com> Message-ID: <4EC3C590.2040704@redhat.com> Martin Kosek wrote: > Make sure that PublicError does crash when it receives > Gettext/NGettext object. Instead of throwing a type error, do the > translation to receive the required unicode text. > > https://fedorahosted.org/freeipa/ticket/2096 ACK From rcritten at redhat.com Wed Nov 16 20:44:55 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 16 Nov 2011 15:44:55 -0500 Subject: [Freeipa-devel] [PATCH] 905 add plugin framework to updater Message-ID: <4EC420C7.40607@redhat.com> There are two reasons for the plugin framework: 1. To provide a way of doing manual/complex LDAP changes without having to keep extending ldapupdate.py (like we did with managed entries). 2. Allows for better control of restarts. There are two types of plugins, preop and postop. A preop plugin runs before any file-based updates are loaded. A postop plugin runs after all file-based updates are applied. A preop plugin may update LDAP directly or craft update entries to be applied with the file-based updates. Either a preop or postop plugin may attempt to restart the dirsrv instance. The instance is only restartable if ipa-ldap-updater is being executed as root. A warning is printed if a restart is requested for a non-root user. Plugins are not executed by default. This is so we can use ldapupdate to apply simple updates in commands like ipa-nis-manage. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-905-update.patch Type: text/x-patch Size: 46617 bytes Desc: not available URL: From abokovoy at redhat.com Thu Nov 17 09:11:40 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 17 Nov 2011 11:11:40 +0200 Subject: [Freeipa-devel] [PATCH] 0034 configure does not check for Python.h Message-ID: <20111117091139.GA16561@redhat.com> py_default_encoding extension depends on Python.h and Python development tools availability. Make sure they are installed. https://fedorahosted.org/freeipa/ticket/1838 -- / Alexander Bokovoy -------------- next part -------------- >From b6b6112cb84f69c41a27bbab1c75fed97be4a61b Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Wed, 16 Nov 2011 15:10:21 +0200 Subject: [PATCH 1/3] Check for Python.h during build of py_default_encoding extension For rare cases when Python development package is not installed, check that Python.h is available and bail out if not. Fixes: https://fedorahosted.org/freeipa/ticket/1838 --- ipapython/py_default_encoding/setup.py | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/ipapython/py_default_encoding/setup.py b/ipapython/py_default_encoding/setup.py index 87f7504e46675fb4c79d6452f23408c445a4824c..de2478c1962aba7a78919efdb266bf0600965905 100644 --- a/ipapython/py_default_encoding/setup.py +++ b/ipapython/py_default_encoding/setup.py @@ -18,6 +18,13 @@ # along with this program. If not, see . from distutils.core import setup, Extension +from distutils.sysconfig import get_python_inc +import sys +import os + +python_header = os.path.join(get_python_inc(plat_specific=1), 'Python.h') +if not os.path.exists(python_header): + sys.exit("Cannot find Python development packages that provide Python.h") default_encoding_utf8 = Extension('default_encoding_utf8', ['default_encoding_utf8.c']) -- 1.7.7.2 From abokovoy at redhat.com Thu Nov 17 09:15:56 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 17 Nov 2011 11:15:56 +0200 Subject: [Freeipa-devel] [PATCH] 0035 Configure should check for libintl Message-ID: <20111117091556.GB16561@redhat.com> There are still distributions with external libintl for gettext support. Thus, extend configure in ipa-client to make sure we don't miss these cases. Instead of using full libintl macro, I opted to a simpler one that does not require anything added into the distribution (libintl m4 code forces some unpleasant additions like config.rpath). Marko M. has spotted this on older Ubuntu versions where eglibc has probably too deviated from glibc, though I'm yet to find any reasonably old distribution based on glibc that doesn't include gettext support in their libc. https://fedorahosted.org/freeipa/ticket/1840 -- / Alexander Bokovoy -------------- next part -------------- >From 456740f0ed5f2085bf64affa59e436ba9724f0d2 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Thu, 17 Nov 2011 10:38:13 +0200 Subject: [PATCH 2/3] Add configure check for libint.h There are some distributions which do not provide gettext support within libc. For these cases checking for libintl is required. https://fedorahosted.org/freeipa/ticket/1840 --- ipa-client/Makefile.am | 3 +++ ipa-client/configure.ac | 15 +++++++++++++++ 2 files changed, 18 insertions(+), 0 deletions(-) diff --git a/ipa-client/Makefile.am b/ipa-client/Makefile.am index d0a3f8463a53411286f52dcc552186b9a36b7b88..f22a2c345ff2356c77b7dccc2e8775950202ae5d 100644 --- a/ipa-client/Makefile.am +++ b/ipa-client/Makefile.am @@ -50,6 +50,7 @@ ipa_getkeytab_LDADD = \ $(OPENLDAP_LIBS) \ $(SASL_LIBS) \ $(POPT_LIBS) \ + $(LIBINTL_LIBS) \ $(NULL) ipa_rmkeytab_SOURCES = \ @@ -60,6 +61,7 @@ ipa_rmkeytab_SOURCES = \ ipa_rmkeytab_LDADD = \ $(KRB5_LIBS) \ $(POPT_LIBS) \ + $(LIBINTL_LIBS) \ $(NULL) ipa_join_SOURCES = \ @@ -75,6 +77,7 @@ ipa_join_LDADD = \ $(CURL_LIBS) \ $(XMLRPC_LIBS) \ $(POPT_LIBS) \ + $(LIBINTL_LIBS) \ $(NULL) SUBDIRS = \ diff --git a/ipa-client/configure.ac b/ipa-client/configure.ac index 75544ae7570c19a1fcbaadc8264b6e94da7fe047..880e9289f3ec92bf195b3330d121567aa27e5a73 100644 --- a/ipa-client/configure.ac +++ b/ipa-client/configure.ac @@ -175,6 +175,21 @@ fi AC_SUBST(XMLRPC_LIBS) dnl --------------------------------------------------------------------------- +dnl - Check for libintl +dnl --------------------------------------------------------------------------- +SAVE_LIBS="$LIBS" +LIBINTL_LIBS= +AC_CHECK_HEADER(libintl.h, [], [AC_MSG_ERROR([libintl.h not found, please install xgettext])]) +AC_SEARCH_LIBS([bindtextdomain], [libintl],[], []) +if test "x$ac_cv_search_bindtextdomain" = "xno" ; then + AC_MSG_ERROR([libintl is not found and your libc does not support gettext, please install xgettext]) +elif test "x$ac_cv_search_bindtextdomain" != "xnone required" ; then + LIBINTL_LIBS="$ac_cv_search_bindtextdomain" +fi +LIBS="$SAVELIBS" +AC_SUBST(LIBINTL_LIBS) + +dnl --------------------------------------------------------------------------- dnl - Set the data install directory since we don't use pkgdatadir dnl --------------------------------------------------------------------------- -- 1.7.7.2 From abokovoy at redhat.com Thu Nov 17 09:20:29 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 17 Nov 2011 11:20:29 +0200 Subject: [Freeipa-devel] [PATCH] 0036 make install doesn't create needed directories Message-ID: <20111117092029.GC16561@redhat.com> When installing client from source with 'make client-install' some essential directories are not created which makes ipa-client-install to fail. I have opted to allow creating /etc/ipa and /var/lib/ipa-client/sysrestore only if DESTDIR is specified. Distributing ipa-client in source form is defeating purpose of FreeIPA, in most cases proper packaging should be preferred. What's important is to hint to packagers what directories should be essential, and thus created in $(DESTDIR). Even source-based distributions have one or another form of packaging that makes this issue less problematic. https://fedorahosted.org/freeipa/ticket/1849 -- / Alexander Bokovoy -------------- next part -------------- >From 3bc661f7fdc8309600fbfd4b946d78c716258172 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Thu, 17 Nov 2011 10:45:15 +0200 Subject: [PATCH 3/3] Create directories for client install When ``make client-install`` is called, create /etc/ipa and /var/lib/ipa-client/sysrestore directories required for successful use of ipa-client-install. Do it only if DESTDIR is set to help packagers to notice that these directories have to exist https://fedorahosted.org/freeipa/ticket/1849 --- Makefile | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-) diff --git a/Makefile b/Makefile index 318364ad2980580764908cd73801f47d80569ba7..36fe5def8b13bb6d1bb104b250af18906ffe7ad8 100644 --- a/Makefile +++ b/Makefile @@ -68,7 +68,7 @@ install: all server-install (cd $$subdir && $(MAKE) $@) || exit 1; \ done -client-install: client +client-install: client client-dirs @for subdir in $(CLIENTDIRS); do \ (cd $$subdir && $(MAKE) install) || exit 1; \ done @@ -79,6 +79,12 @@ client-install: client python setup-client.py install --root $(DESTDIR); \ fi +client-dirs: + if [ "$(DESTDIR)" != "" ] ; then \ + mkdir -p $(DESTDIR)/etc/ipa ; \ + mkdir -p $(DESTDIR)/var/lib/ipa-client/sysrestore ; \ + fi + lint: ./make-lint $(LINT_OPTIONS) -- 1.7.7.2 From jdennis at redhat.com Thu Nov 17 16:17:36 2011 From: jdennis at redhat.com (John Dennis) Date: Thu, 17 Nov 2011 11:17:36 -0500 Subject: [Freeipa-devel] [PATCH 53/53] ticket 2022 - modify codebase to utilize IPALogManager, obsoletes logging In-Reply-To: <1321446953.18210.31.camel@balmora.brq.redhat.com> References: <201111151941.pAFJfC5u017345@int-mx09.intmail.prod.int.phx2.redhat.com> <1321446953.18210.31.camel@balmora.brq.redhat.com> Message-ID: <4EC533A0.1020006@redhat.com> On 11/16/2011 07:35 AM, Martin Kosek wrote: > On Tue, 2011-11-15 at 14:41 -0500, John Dennis wrote: >> >> -- >> John Dennis >> >> Looking to carve out IT costs? >> www.redhat.com/carveoutcosts/ > > Hi John, > > thanks for your patch, that was a fair amount of work there :-) I tested > them both and have few comments: > > 1) Patch 53 will need rebasing O.K. will rebase. > > 2) Logging for install tools (ipa-server-install, ...) is now broken. > DEBUG level messages should get to /var/log/ipaserver-install.log even > when the installer is run without --debug. My > ipaserver-install.log/ipaserver-uninstall.log was silent when i run just > `ipa-server-install' without --debug flag Yes you're right, it's supposed to always log to the file at the debug level. Sorry that slipped by, it's a trivial one word fix in standard_logging_setup(), done. > 3) Patch 53 touches most of our Python sources as it changes all > logging.* calls to log_mgr.root_logger.* calls. I think this may cause > problems (comment 1 is a good example). For example we may not be able > to cherry-pick most of new patches from master branch to ipa-2-1 branch > or other branches that do not use log_mgr. > > Additionally, typing "log_mgr.root_logger.*" whenever I want to log > anything seems a bit awkward to me. These issues are avoidable, however. > Since all our Python code use just the root_logger, can we just simply > export root logger in log_manager.py namespace and that import it as > "logging" in other modules? Then we would save *a lot* of changes across > all our code. Something like this: > > log_manager.py: > --------------- > log_mgr = IPALogManager() > log_mgr.configure(dict(default_level='error', > handlers=[dict(name='console', > stream=sys.stderr)]), > configure_state='default') > +root_logger = log_mgr.root_logger > > ipa-server-install: > ------------------- > +from ipapython.log_manager import root_logger as logginng > > ... > # now, all logging will work as usual: > logging.error('Some files have not been restored, see /var/lib/ipa/sysrestore/sysrestore.index') Cherry picking patches between branches with out performing any editing to apply the patch is a nice goal but perhaps not realistic. However you're absolutely right this change introduces an extra step during patch/rebase. A larger question is should we continue to support the idea of having log functions hung off a module name. Here's my thinking on that. But skip to item 3 if you want a compromise solution. 1) I'm a bit concerned that co-opting or borrowing the logging module name and using it for an entirely different purpose. Importing the name "logging" and not have it be the Python logging module as most Python programmers would be the source of needless confusion. Personally I think I would be annoyed if someone had played that game because nothing would be as I expect. The name "logging" implies the logging module with all sorts of names, constants, objects, classes, sub-modules, etc. associated with it, none of that would retain any of it's conventional meaning. I much prefer clarity and to call something what it is and not obfuscate for temporary expediency. 2) The current Python logging implementation is broken IMHO and was the source of some of our problems, I'm not sure I want to perpetuate that. It exported a set of "convenience" functions bound to a global. It made using the logging module easier to use via globals but at the expense of all sorts of conflicts when more than one client of logging tried to manipulate the same shared global objects. The logging.debug(), etc. names are a bogus mapping of a top level module symbol to a bound method of a specific instantiated object, an object available externally only by calling a defined API to retrieve that object from singleton manager by name. To my mind this is the classic abuse of globals every textbook warns against. It's just bad programming best avoided IMHO. Why is it O.K. to have a global log_mgr object for IPA? a) Because it belongs to us, it's not shared with every piece of Python code which imports the logging module. b) It's not abusing a top level module symbol to be a short circuit reference into some other instantiated object. The root logger belongs to the log_mgr instance shared by loaded IPA components. Think of it as an IPA singleton. 3) Can we have a shorter name as a compromise? Sure, I have no problems with that. We already have the log_mgr as a single imported instance. I don't see why we couldn't have also import the symbol "root_logger" which is log_mgr.root_logger. I'm not keen on shorting the name much more, for instance to "logger" because that's too generic and obscures the fact it's the root logger, an important distinction to maintain. An other option would be to have a LogManager object export the debug(), info(), error(), critical(), & exception() methods where they are bound to the LogManager's root_logger instance. This is somewhat akin to what the standard Python logging module, except they performed the binding on the module level and not on an instance (the source of many problems). Those log methods *must* be bound to a Logger object as such I'm not entirely keen on binding them to a LogManager instance because once again you end up providing an abused short cut which hides what is really going on. I think it would benefit programmers if they knew they were logging to a Logger and if so which one, because after all once you try to customize things like "which loggers emit which which messages" it would be really helpful to know you're dealing with a Logger and which one it is and not obscure which one it is to shorten the reference to it as a convenience via some short cut. But if there is a consensus I could bend on this issue. That means we could have one of the two (I prefer the first) root_logger.debug() -or- log_mgr.debug() # Not a fan of this because it hides the important # concept of debug() being bound to the root_logger. As for: logging.debug() For the reasons pointed out in 1 I'd like to avoid that kind of symbol abuse. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Thu Nov 17 16:43:57 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 17 Nov 2011 11:43:57 -0500 Subject: [Freeipa-devel] [PATCH] 0034 configure does not check for Python.h In-Reply-To: <20111117091139.GA16561@redhat.com> References: <20111117091139.GA16561@redhat.com> Message-ID: <4EC539CD.2010806@redhat.com> Alexander Bokovoy wrote: > py_default_encoding extension depends on Python.h and Python > development tools availability. Make sure they are installed. > > https://fedorahosted.org/freeipa/ticket/1838 ACK pushed to master From rcritten at redhat.com Thu Nov 17 16:44:11 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 17 Nov 2011 11:44:11 -0500 Subject: [Freeipa-devel] [PATCH] 0035 Configure should check for libintl In-Reply-To: <20111117091556.GB16561@redhat.com> References: <20111117091556.GB16561@redhat.com> Message-ID: <4EC539DB.5060701@redhat.com> Alexander Bokovoy wrote: > There are still distributions with external libintl for gettext > support. Thus, extend configure in ipa-client to make sure we don't > miss these cases. > > Instead of using full libintl macro, I opted to a simpler one that > does not require anything added into the distribution (libintl m4 code > forces some unpleasant additions like config.rpath). > > Marko M. has spotted this on older Ubuntu versions where eglibc has > probably too deviated from glibc, though I'm yet to find any > reasonably old distribution based on glibc that doesn't include > gettext support in their libc. > > https://fedorahosted.org/freeipa/ticket/1840 ACK fixed typo in commit msg and pushed to master From rcritten at redhat.com Thu Nov 17 16:48:56 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 17 Nov 2011 11:48:56 -0500 Subject: [Freeipa-devel] [PATCH] 0036 make install doesn't create needed directories In-Reply-To: <20111117092029.GC16561@redhat.com> References: <20111117092029.GC16561@redhat.com> Message-ID: <4EC53AF8.9050306@redhat.com> Alexander Bokovoy wrote: > When installing client from source with 'make client-install' > some essential directories are not created which makes > ipa-client-install to fail. > > I have opted to allow creating /etc/ipa and > /var/lib/ipa-client/sysrestore only if DESTDIR is specified. > Distributing ipa-client in source form is defeating purpose of > FreeIPA, in most cases proper packaging should be preferred. What's > important is to hint to packagers what directories should be > essential, and thus created in $(DESTDIR). Even source-based > distributions have one or another form of packaging that makes this > issue less problematic. > > https://fedorahosted.org/freeipa/ticket/1849 I wonder if we should @echo something about DESTDIR not set so we aren't creating the directories. rob From rcritten at redhat.com Thu Nov 17 16:52:33 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 17 Nov 2011 11:52:33 -0500 Subject: [Freeipa-devel] [PATCH] Another trivial doc fix In-Reply-To: References: Message-ID: <4EC53BD1.1090201@redhat.com> J?r?me Fenal wrote: > diff --git a/ipalib/plugins/group.py b/ipalib/plugins/group.py > index cd4a054..4bf77f8 100644 > --- a/ipalib/plugins/group.py > +++ b/ipalib/plugins/group.py > @@ -28,8 +28,8 @@ Groups of users > > Manage groups of users. By default, new groups are POSIX groups. You > can add the --nonposix option to the group-add command to mark a new group > -as non-POSIX, and you can use the same argument to the group-mod command > -to convert a non-POSIX group to a POSIX group. POSIX groups cannot be > +as non-POSIX. You can use the --posix argument to the group-mod command > +to convert a non-POSIX group into a POSIX group. POSIX groups cannot be > converted to non-POSIX groups. > > Every group must have a description. > > I modified this slightly to say "You can use the --posix argument with the group-mod command...". Pushed to master thanks rob From abokovoy at redhat.com Thu Nov 17 16:56:20 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 17 Nov 2011 18:56:20 +0200 Subject: [Freeipa-devel] [PATCH] 0036 make install doesn't create needed directories In-Reply-To: <4EC53AF8.9050306@redhat.com> References: <20111117092029.GC16561@redhat.com> <4EC53AF8.9050306@redhat.com> Message-ID: <20111117165620.GF16561@redhat.com> On Thu, 17 Nov 2011, Rob Crittenden wrote: > Alexander Bokovoy wrote: > >When installing client from source with 'make client-install' > >some essential directories are not created which makes > >ipa-client-install to fail. > > > >I have opted to allow creating /etc/ipa and > >/var/lib/ipa-client/sysrestore only if DESTDIR is specified. > >Distributing ipa-client in source form is defeating purpose of > >FreeIPA, in most cases proper packaging should be preferred. What's > >important is to hint to packagers what directories should be > >essential, and thus created in $(DESTDIR). Even source-based > >distributions have one or another form of packaging that makes this > >issue less problematic. > > > >https://fedorahosted.org/freeipa/ticket/1849 > > I wonder if we should @echo something about DESTDIR not set so we > aren't creating the directories. How about attached variant? -- / Alexander Bokovoy -------------- next part -------------- >From b685d68ba19d6f9a328e8984a65982d6ca300e2a Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Thu, 17 Nov 2011 10:45:15 +0200 Subject: [PATCH 3/3] Create directories for client install When ``make client-install`` is called, create /etc/ipa and /var/lib/ipa-client/sysrestore directories required for successful use of ipa-client-install. Do it only if DESTDIR is set to help packagers to notice that these directories have to exist https://fedorahosted.org/freeipa/ticket/1849 --- Makefile | 11 ++++++++++- 1 files changed, 10 insertions(+), 1 deletions(-) diff --git a/Makefile b/Makefile index 318364ad2980580764908cd73801f47d80569ba7..baf6eee5c151a88e9a668b3e4c3da3b3b2687886 100644 --- a/Makefile +++ b/Makefile @@ -68,7 +68,7 @@ install: all server-install (cd $$subdir && $(MAKE) $@) || exit 1; \ done -client-install: client +client-install: client client-dirs @for subdir in $(CLIENTDIRS); do \ (cd $$subdir && $(MAKE) install) || exit 1; \ done @@ -79,6 +79,15 @@ client-install: client python setup-client.py install --root $(DESTDIR); \ fi +client-dirs: + @if [ "$(DESTDIR)" != "" ] ; then \ + mkdir -p $(DESTDIR)/etc/ipa ; \ + mkdir -p $(DESTDIR)/var/lib/ipa-client/sysrestore ; \ + else \ + echo "DESTDIR was not set, please create /etc/ipa and /var/lib/ipa-client/sysrestore" ; \ + echo "Without those directories ipa-client-install will fail" ; \ + fi + lint: ./make-lint $(LINT_OPTIONS) -- 1.7.7.2 From rcritten at redhat.com Thu Nov 17 18:07:23 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 17 Nov 2011 13:07:23 -0500 Subject: [Freeipa-devel] [PATCH] 0036 make install doesn't create needed directories In-Reply-To: <20111117165620.GF16561@redhat.com> References: <20111117092029.GC16561@redhat.com> <4EC53AF8.9050306@redhat.com> <20111117165620.GF16561@redhat.com> Message-ID: <4EC54D5B.3090700@redhat.com> Alexander Bokovoy wrote: > On Thu, 17 Nov 2011, Rob Crittenden wrote: > >> Alexander Bokovoy wrote: >>> When installing client from source with 'make client-install' >>> some essential directories are not created which makes >>> ipa-client-install to fail. >>> >>> I have opted to allow creating /etc/ipa and >>> /var/lib/ipa-client/sysrestore only if DESTDIR is specified. >>> Distributing ipa-client in source form is defeating purpose of >>> FreeIPA, in most cases proper packaging should be preferred. What's >>> important is to hint to packagers what directories should be >>> essential, and thus created in $(DESTDIR). Even source-based >>> distributions have one or another form of packaging that makes this >>> issue less problematic. >>> >>> https://fedorahosted.org/freeipa/ticket/1849 >> >> I wonder if we should @echo something about DESTDIR not set so we >> aren't creating the directories. > How about attached variant? > Perfect. Pushed to master. rob From simo at redhat.com Thu Nov 17 18:20:19 2011 From: simo at redhat.com (Simo Sorce) Date: Thu, 17 Nov 2011 13:20:19 -0500 Subject: [Freeipa-devel] [PATCH] missing (C) in one file Message-ID: <1321554019.30630.149.camel@willson.li.ssimo.org> I found out one of the files I was working on these days is missing a proper header. Patch attached. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-missing-copyright-header.patch Type: text/x-patch Size: 1437 bytes Desc: not available URL: From rcritten at redhat.com Thu Nov 17 18:21:18 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 17 Nov 2011 13:21:18 -0500 Subject: [Freeipa-devel] [PATCH] missing (C) in one file In-Reply-To: <1321554019.30630.149.camel@willson.li.ssimo.org> References: <1321554019.30630.149.camel@willson.li.ssimo.org> Message-ID: <4EC5509E.90209@redhat.com> Simo Sorce wrote: > I found out one of the files I was working on these days is missing a > proper header. > > Patch attached. > > Simo. ACK From jdennis at redhat.com Thu Nov 17 19:00:57 2011 From: jdennis at redhat.com (John Dennis) Date: Thu, 17 Nov 2011 14:00:57 -0500 Subject: [Freeipa-devel] [PATCH 53/53] ticket 2022 - modify codebase to utilize IPALogManager, obsoletes logging In-Reply-To: <4EC533A0.1020006@redhat.com> References: <201111151941.pAFJfC5u017345@int-mx09.intmail.prod.int.phx2.redhat.com> <1321446953.18210.31.camel@balmora.brq.redhat.com> <4EC533A0.1020006@redhat.com> Message-ID: <4EC559E9.4020909@redhat.com> On 11/17/2011 11:17 AM, John Dennis wrote: > On 11/16/2011 07:35 AM, Martin Kosek wrote: >> 1) Patch 53 will need rebasing > > O.K. will rebase. O.K. I'm confused. This is a 3.0 feature and the patch is against origin/master which is fully up to date. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From edewata at redhat.com Thu Nov 17 19:25:14 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 17 Nov 2011 13:25:14 -0600 Subject: [Freeipa-devel] [PATCH] 314 Fixed entity definition in test cases. Message-ID: <4EC55F9A.7040400@redhat.com> The test cases have been updated to use the new extensible mechanism for defining and registering entities. Ticket #2043 -- Endi S. Dewata -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freeipa-edewata-0314-Fixed-entity-definition-in-test-cases.patch URL: From edewata at redhat.com Thu Nov 17 19:27:42 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 17 Nov 2011 13:27:42 -0600 Subject: [Freeipa-devel] [PATCH] 315 Added commands into metadata. Message-ID: <4EC5602E.3030004@redhat.com> The json_metadata command has been modified to return the command metadata. The API.txt has been updated as well. Needed by ticket #388 -- Endi S. Dewata -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freeipa-edewata-0315-Added-commands-into-metadata.patch URL: From simo at redhat.com Thu Nov 17 21:15:42 2011 From: simo at redhat.com (Simo Sorce) Date: Thu, 17 Nov 2011 16:15:42 -0500 Subject: [Freeipa-devel] [PATCH] missing (C) in one file In-Reply-To: <4EC5509E.90209@redhat.com> References: <1321554019.30630.149.camel@willson.li.ssimo.org> <4EC5509E.90209@redhat.com> Message-ID: <1321564542.30630.164.camel@willson.li.ssimo.org> On Thu, 2011-11-17 at 13:21 -0500, Rob Crittenden wrote: > Simo Sorce wrote: > > I found out one of the files I was working on these days is missing a > > proper header. > > > > Patch attached. > > > > Simo. > > ACK Pushed to master. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Thu Nov 17 22:00:51 2011 From: simo at redhat.com (Simo Sorce) Date: Thu, 17 Nov 2011 17:00:51 -0500 Subject: [Freeipa-devel] [PATCHES] #1950 Implement CLDAP server for AD trust introperability Message-ID: <1321567251.30630.169.camel@willson.li.ssimo.org> Attached find a series of patches that implement a CLDAP server as a dirsrv plugin. The server right now responds only to a very limited class of requests, as observed on the wire. But it can be easily expanded to respond to additional requests as needed. Tested against windows 2008 with which I had create a trust. To test you need Sumit's uncommitted adtrust code to successfully create the trust and provision the IPA tree with the right data. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-NT-domain-GUID-attribute.patch Type: text/x-patch Size: 2764 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Create-skeleton-CLDAP-server-as-a-DS-plugin.patch Type: text/x-patch Size: 18352 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0003-ipa-cldap-Implement-worker-thread.patch Type: text/x-patch Size: 2524 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0004-ipa-cldap-Decode-CLDAP-request.patch Type: text/x-patch Size: 6399 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0005-ipa-cldap-Create-netlogon-blob.patch Type: text/x-patch Size: 13645 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0006-ipa-cldap-send-cldap-reply.patch Type: text/x-patch Size: 2358 bytes Desc: not available URL: From mkosek at redhat.com Fri Nov 18 08:04:59 2011 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 18 Nov 2011 09:04:59 +0100 Subject: [Freeipa-devel] [PATCH 53/53] ticket 2022 - modify codebase to utilize IPALogManager, obsoletes logging In-Reply-To: <4EC533A0.1020006@redhat.com> References: <201111151941.pAFJfC5u017345@int-mx09.intmail.prod.int.phx2.redhat.com> <1321446953.18210.31.camel@balmora.brq.redhat.com> <4EC533A0.1020006@redhat.com> Message-ID: <1321603499.4192.16.camel@balmora.brq.redhat.com> On Thu, 2011-11-17 at 11:17 -0500, John Dennis wrote: > On 11/16/2011 07:35 AM, Martin Kosek wrote: > > On Tue, 2011-11-15 at 14:41 -0500, John Dennis wrote: > >> > >> -- > >> John Dennis > >> > >> Looking to carve out IT costs? > >> www.redhat.com/carveoutcosts/ > > > > Hi John, > > > > thanks for your patch, that was a fair amount of work there :-) I tested > > them both and have few comments: > > > > 1) Patch 53 will need rebasing > > O.K. will rebase. > > > > 2) Logging for install tools (ipa-server-install, ...) is now broken. > > DEBUG level messages should get to /var/log/ipaserver-install.log even > > when the installer is run without --debug. My > > ipaserver-install.log/ipaserver-uninstall.log was silent when i run just > > `ipa-server-install' without --debug flag > > Yes you're right, it's supposed to always log to the file at the debug > level. Sorry that slipped by, it's a trivial one word fix in > standard_logging_setup(), done. Ok. > > > 3) Patch 53 touches most of our Python sources as it changes all > > logging.* calls to log_mgr.root_logger.* calls. I think this may cause > > problems (comment 1 is a good example). For example we may not be able > > to cherry-pick most of new patches from master branch to ipa-2-1 branch > > or other branches that do not use log_mgr. > > > > Additionally, typing "log_mgr.root_logger.*" whenever I want to log > > anything seems a bit awkward to me. These issues are avoidable, however. > > Since all our Python code use just the root_logger, can we just simply > > export root logger in log_manager.py namespace and that import it as > > "logging" in other modules? Then we would save *a lot* of changes across > > all our code. Something like this: > > > > log_manager.py: > > --------------- > > log_mgr = IPALogManager() > > log_mgr.configure(dict(default_level='error', > > handlers=[dict(name='console', > > stream=sys.stderr)]), > > configure_state='default') > > +root_logger = log_mgr.root_logger > > > > ipa-server-install: > > ------------------- > > +from ipapython.log_manager import root_logger as logginng > > > > ... > > # now, all logging will work as usual: > > logging.error('Some files have not been restored, see /var/lib/ipa/sysrestore/sysrestore.index') > > Cherry picking patches between branches with out performing any editing > to apply the patch is a nice goal but perhaps not realistic. However > you're absolutely right this change introduces an extra step during > patch/rebase. > > A larger question is should we continue to support the idea of having > log functions hung off a module name. Here's my thinking on that. But > skip to item 3 if you want a compromise solution. > > > 1) I'm a bit concerned that co-opting or borrowing the logging module > name and using it for an entirely different purpose. Importing the name > "logging" and not have it be the Python logging module as most Python > programmers would be the source of needless confusion. Personally I > think I would be annoyed if someone had played that game because nothing > would be as I expect. The name "logging" implies the logging module with > all sorts of names, constants, objects, classes, sub-modules, etc. > associated with it, none of that would retain any of it's conventional > meaning. I much prefer clarity and to call something what it is and not > obfuscate for temporary expediency. > > 2) The current Python logging implementation is broken IMHO and was the > source of some of our problems, I'm not sure I want to perpetuate that. > It exported a set of "convenience" functions bound to a global. It made > using the logging module easier to use via globals but at the expense of > all sorts of conflicts when more than one client of logging tried to > manipulate the same shared global objects. > > The logging.debug(), etc. names are a bogus mapping of a top level > module symbol to a bound method of a specific instantiated object, an > object available externally only by calling a defined API to retrieve > that object from singleton manager by name. To my mind this is the > classic abuse of globals every textbook warns against. It's just bad > programming best avoided IMHO. > > Why is it O.K. to have a global log_mgr object for IPA? > > a) Because it belongs to us, it's not shared with every piece of > Python code which imports the logging module. > > b) It's not abusing a top level module symbol to be a short circuit > reference into some other instantiated object. The root logger > belongs to the log_mgr instance shared by loaded IPA components. > Think of it as an IPA singleton. > > 3) Can we have a shorter name as a compromise? Sure, I have no problems > with that. We already have the log_mgr as a single imported instance. > I don't see why we couldn't have also import the symbol > "root_logger" which is log_mgr.root_logger. I'm not keen on shorting > the name much more, for instance to "logger" because that's too > generic and obscures the fact it's the root logger, an important > distinction to maintain. > > An other option would be to have a LogManager object export the > debug(), info(), error(), critical(), & exception() methods where > they are bound to the LogManager's root_logger instance. This is > somewhat akin to what the standard Python logging module, except > they performed the binding on the module level and not on an > instance (the source of many problems). Those log methods *must* > be bound to a Logger object as such I'm not entirely keen on > binding them to a LogManager instance because once again you > end up providing an abused short cut which hides what is really > going on. I think it would benefit programmers if they knew they > were logging to a Logger and if so which one, because after all > once you try to customize things like "which loggers emit which > which messages" it would be really helpful to know you're dealing > with a Logger and which one it is and not obscure which one it is > to shorten the reference to it as a convenience via some short cut. > But if there is a consensus I could bend on this issue. > > That means we could have one of the two (I prefer the first) > > root_logger.debug() > > -or- > > log_mgr.debug() # Not a fan of this because it hides the important > # concept of debug() being bound to the root_logger. > > As for: > > logging.debug() > > For the reasons pointed out in 1 I'd like to avoid that kind of > symbol abuse. > > Ok, these are all valid arguments. I know that abusing "logging" is not right, I was just afraid that diverging master branch from our 2.x branch that much would do more harm (and extra effort whenever a patch from master is backported to 2.x branch) that misusing the "logging" name. I would like some second opinion on that before we make the call. I think having root_logger exported to our tools and calling root_logger.warning(...) etc. looks better than the previous version (log_mgr.root_logger.warning()). Martin From mkosek at redhat.com Fri Nov 18 08:09:45 2011 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 18 Nov 2011 09:09:45 +0100 Subject: [Freeipa-devel] [PATCH 53/53] ticket 2022 - modify codebase to utilize IPALogManager, obsoletes logging In-Reply-To: <4EC559E9.4020909@redhat.com> References: <201111151941.pAFJfC5u017345@int-mx09.intmail.prod.int.phx2.redhat.com> <1321446953.18210.31.camel@balmora.brq.redhat.com> <4EC533A0.1020006@redhat.com> <4EC559E9.4020909@redhat.com> Message-ID: <1321603785.4192.18.camel@balmora.brq.redhat.com> On Thu, 2011-11-17 at 14:00 -0500, John Dennis wrote: > On 11/17/2011 11:17 AM, John Dennis wrote: > > On 11/16/2011 07:35 AM, Martin Kosek wrote: > >> 1) Patch 53 will need rebasing > > > > O.K. will rebase. > > O.K. I'm confused. This is a 3.0 feature and the patch is against > origin/master which is fully up to date. > Hm, this does not work for me on the master branch: $ git pull Already up-to-date. $ git apply ~/0052-ticket-2022-Add-log-manager-module.patch /home/mkosek/0052-ticket-2022-Add-log-manager-module.patch:1766: new blank line at EOF. + warning: 1 line adds whitespace errors. $ git apply ~/0053-ticket-2022-modify-codebase-to-utilize-IPALogManager.patch error: patch failed: install/tools/ipa-server-install:510 error: install/tools/ipa-server-install: patch does not apply error: patch failed: ipaserver/install/dsinstance.py:97 error: ipaserver/install/dsinstance.py: patch does not apply Martin From pvoborni at redhat.com Fri Nov 18 10:29:07 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 18 Nov 2011 11:29:07 +0100 Subject: [Freeipa-devel] [PATCH] 314 Fixed entity definition in test cases. In-Reply-To: <4EC55F9A.7040400@redhat.com> References: <4EC55F9A.7040400@redhat.com> Message-ID: <4EC63373.7030405@redhat.com> On 11/17/2011 08:25 PM, Endi Sukma Dewata wrote: > The test cases have been updated to use the new extensible mechanism > for defining and registering entities. > > Ticket #2043 > 1) In the tests is a call of entity_init method which isn't defined anywhere. It leads to JS error and test fail. -- Petr Vobornik From pvoborni at redhat.com Fri Nov 18 10:58:06 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 18 Nov 2011 11:58:06 +0100 Subject: [Freeipa-devel] [PATCH] 315 Added commands into metadata. In-Reply-To: <4EC5602E.3030004@redhat.com> References: <4EC5602E.3030004@redhat.com> Message-ID: <4EC63A3E.5080200@redhat.com> On 11/17/2011 08:27 PM, Endi Sukma Dewata wrote: > The json_metadata command has been modified to return the command > metadata. The API.txt has been updated as well. > > Needed by ticket #388 > ACK from UI perspective. Ipalib changes seems fine to me, but I'm not 100% sure. Note: this patch depends on edewata-314 which isn't ACKed yet. Note 2: I hope this is really needed. It adds 400KB of data to load at UI start. -- Petr Vobornik From abokovoy at redhat.com Fri Nov 18 11:37:14 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 18 Nov 2011 13:37:14 +0200 Subject: [Freeipa-devel] [PATCH] 315 Added commands into metadata. In-Reply-To: <4EC5602E.3030004@redhat.com> References: <4EC5602E.3030004@redhat.com> Message-ID: <20111118113714.GA8538@redhat.com> On Thu, 17 Nov 2011, Endi Sukma Dewata wrote: > The json_metadata command has been modified to return the command > metadata. The API.txt has been updated as well. > > Needed by ticket #388 ... > --- a/ipalib/plugins/internal.py > +++ b/ipalib/plugins/internal.py > @@ -49,15 +52,25 @@ class json_metadata(Command): > ) > > has_output = ( > + Output('commands', dict, doc=_('Dict of JSON encoded IPA Commands')), > Output('objects', dict, doc=_('Dict of JSON encoded IPA Objects')), > Output('methods', dict, doc=_('Dict of JSON encoded IPA Methods')), > ) > > - def execute(self, objname, methodname): > + def execute(self, cmdname, objname, methodname): > + commands = dict() > objects = dict() > methods = dict() I don't like this particular change of order of arguments but as API.Command.json_metadata is only used in two (three now) places in ipa.js, it is acceptable -- there is practically no old Web UI anywhere as it is downloaded from the server. -- / Alexander Bokovoy From jkuncar at redhat.com Fri Nov 18 13:18:37 2011 From: jkuncar at redhat.com (Jiri Kuncar) Date: Fri, 18 Nov 2011 08:18:37 -0500 (EST) Subject: [Freeipa-devel] [PATCH] Skip invalid records In-Reply-To: Message-ID: <617d88cb-b991-4eb0-90ee-c8ec7119e994@zmail13.collab.prod.int.phx2.redhat.com> Bind-dyndb-ldap: Don't fail to load zone when it contains invalid records. Needed by ticket #36 https://fedorahosted.org/bind-dyndb-ldap/ticket/36 -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Skip-bad-records.patch Type: text/x-patch Size: 1104 bytes Desc: not available URL: From ohamada at redhat.com Fri Nov 18 13:29:20 2011 From: ohamada at redhat.com (Ondrej Hamada) Date: Fri, 18 Nov 2011 14:29:20 +0100 Subject: [Freeipa-devel] [PATCH] 4 ipa-client-install fails when not run as root Message-ID: <4EC65DB0.5010208@redhat.com> https://fedorahosted.org/freeipa/ticket/2123 ipa-client-install was failing and returning traceback when wasn't run by root. It was caused by logging initialization that was taking part before the root privileges check. To correct it, the privileges check was moved before the logging initialization. -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ohamada-4-Client-install-root-privileges-check.patch Type: text/x-patch Size: 1523 bytes Desc: not available URL: From atkac at redhat.com Fri Nov 18 13:45:06 2011 From: atkac at redhat.com (Adam Tkac) Date: Fri, 18 Nov 2011 14:45:06 +0100 Subject: [Freeipa-devel] [PATCH] Skip invalid records In-Reply-To: <617d88cb-b991-4eb0-90ee-c8ec7119e994@zmail13.collab.prod.int.phx2.redhat.com> References: <617d88cb-b991-4eb0-90ee-c8ec7119e994@zmail13.collab.prod.int.phx2.redhat.com> Message-ID: <4EC66162.5080505@redhat.com> On 11/18/2011 02:18 PM, Jiri Kuncar wrote: > Bind-dyndb-ldap: Don't fail to load zone when it contains invalid records. Your patch seems fine for me, thanks. Please check my comments inside the patch. Regards, Adam > > Needed by ticket #36 https://fedorahosted.org/bind-dyndb-ldap/ticket/36 > > 0001-Skip-bad-records.patch > > > From 37a8430b4a5555d33fc727a6b6a19998710b37da Mon Sep 17 00:00:00 2001 > From: Jiri Kuncar > Date: Fri, 18 Nov 2011 08:04:53 -0500 > Subject: [PATCH] Skip bad records. > > Signed-off-by: Jiri Kuncar > --- > src/ldap_helper.c | 8 +++++--- > 1 files changed, 5 insertions(+), 3 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index 6fb57ce..029398c 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -1080,9 +1080,11 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > for (entry = HEAD(ldap_conn->ldap_entries); > entry != NULL; > entry = NEXT(entry, link)) { > - CHECK(ldap_parse_rrentry(mctx, entry, ldap_conn, > - origin, ldap_inst->fake_mname, > - string, rdatalist)); > + if (ISC_R_SUCCESS != ldap_parse_rrentry(mctx, entry, ldap_conn, > + origin, ldap_inst->fake_mname, > + string, rdatalist)) { Please keep consistency with rest of code. Use if (function() != result), not if (result != function()). > + log_error("Failed to parse RR entry"); What about log_error("Failed to parse RR entry (%s)", string); ? This will help admin to track which RR is bad. > + } > } > > /* Cache RRs */ > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From edewata at redhat.com Fri Nov 18 14:57:55 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Fri, 18 Nov 2011 08:57:55 -0600 Subject: [Freeipa-devel] [PATCH] 314 Fixed entity definition in test cases. In-Reply-To: <4EC63373.7030405@redhat.com> References: <4EC55F9A.7040400@redhat.com> <4EC63373.7030405@redhat.com> Message-ID: <4EC67273.6010104@redhat.com> On 11/18/2011 4:29 AM, Petr Vobornik wrote: > 1) In the tests is a call of entity_init method which isn't defined > anywhere. It leads to JS error and test fail. Fixed. It was defined in another patch: diff --git a/install/ui/entity.js b/install/ui/entity.js index f7bf992..4e5c724 100644 --- a/install/ui/entity.js +++ b/install/ui/entity.js @@ -217,6 +217,7 @@ IPA.entity = function(spec) { return [that.name + '-pkey']; }; + that.entity_init = that.init; return that; }; -- Endi S. Dewata -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freeipa-edewata-0314-2-Fixed-entity-definition-in-test-cases.patch URL: From pvoborni at redhat.com Fri Nov 18 15:01:31 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 18 Nov 2011 16:01:31 +0100 Subject: [Freeipa-devel] [PATCH] 314 Fixed entity definition in test cases. In-Reply-To: <4EC67273.6010104@redhat.com> References: <4EC55F9A.7040400@redhat.com> <4EC63373.7030405@redhat.com> <4EC67273.6010104@redhat.com> Message-ID: <4EC6734B.2070801@redhat.com> On 11/18/2011 03:57 PM, Endi Sukma Dewata wrote: > On 11/18/2011 4:29 AM, Petr Vobornik wrote: >> 1) In the tests is a call of entity_init method which isn't defined >> anywhere. It leads to JS error and test fail. > > Fixed. It was defined in another patch: ACK -- Petr Vobornik From sbose at redhat.com Fri Nov 18 15:07:17 2011 From: sbose at redhat.com (Sumit Bose) Date: Fri, 18 Nov 2011 16:07:17 +0100 Subject: [Freeipa-devel] [PATCHES] #1950 Implement CLDAP server for AD trust introperability In-Reply-To: <1321567251.30630.169.camel@willson.li.ssimo.org> References: <1321567251.30630.169.camel@willson.li.ssimo.org> Message-ID: <20111118150716.GF8631@localhost.localdomain> On Thu, Nov 17, 2011 at 05:00:51PM -0500, Simo Sorce wrote: > Attached find a series of patches that implement a CLDAP server as a > dirsrv plugin. > > The server right now responds only to a very limited class of requests, > as observed on the wire. But it can be easily expanded to respond to > additional requests as needed. > > Tested against windows 2008 with which I had create a trust. > To test you need Sumit's uncommitted adtrust code to successfully create > the trust and provision the IPA tree with the right data. The patch prevents dirsrv from shutting down cleanly, so NACK. But otherwise it is working great. I will rebase my patches on top of yours, fix some missing bits and send them here as well. bye, Sumit > > Simo. > > -- > Simo Sorce * Red Hat, Inc * New York From edewata at redhat.com Fri Nov 18 15:36:14 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Fri, 18 Nov 2011 09:36:14 -0600 Subject: [Freeipa-devel] [PATCH] 315 Added commands into metadata. In-Reply-To: <4EC63A3E.5080200@redhat.com> References: <4EC5602E.3030004@redhat.com> <4EC63A3E.5080200@redhat.com> Message-ID: <4EC67B6E.2070409@redhat.com> On 11/18/2011 4:58 AM, Petr Vobornik wrote: > On 11/17/2011 08:27 PM, Endi Sukma Dewata wrote: >> The json_metadata command has been modified to return the command >> metadata. The API.txt has been updated as well. >> >> Needed by ticket #388 > > ACK from UI perspective. Ipalib changes seems fine to me, but I'm not > 100% sure. > > Note: this patch depends on edewata-314 which isn't ACKed yet. > Note 2: I hope this is really needed. It adds 400KB of data to load at > UI start. Kind of. The current json_metadata only returns objects and methods. The hbactest plugin is defined as a Command, so the metadata for this plugin will not be returned unless we add the metadata for all commands. HBAC test's metadata is needed to show translated labels in the UI. We can also redefine it in i18n_messages to avoid adding commands into json_metadata, but that will create duplicate labels. Now the methods metadata seem to be a subset of commands metadata, so we probably can change the UI to use commands metadata and not pull the methods metadata. On 11/18/2011 5:37 AM, Alexander Bokovoy wrote: >> - def execute(self, objname, methodname): >> + def execute(self, cmdname, objname, methodname): > I don't like this particular change of order of arguments but as > API.Command.json_metadata is only used in two (three now) places in > ipa.js, it is acceptable -- there is practically no old Web UI > anywhere as it is downloaded from the server. In the JSON API itself the parameters are specified as options, so the order shouldn't matter to the UI. Is there a way to define the execute() using unordered keywords? I'm trying to avoid changing the method signature again in the future. -- Endi S. Dewata From edewata at redhat.com Fri Nov 18 16:14:40 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Fri, 18 Nov 2011 10:14:40 -0600 Subject: [Freeipa-devel] [PATCH] 314 Fixed entity definition in test cases. In-Reply-To: <4EC6734B.2070801@redhat.com> References: <4EC55F9A.7040400@redhat.com> <4EC63373.7030405@redhat.com> <4EC67273.6010104@redhat.com> <4EC6734B.2070801@redhat.com> Message-ID: <4EC68470.5090403@redhat.com> On 11/18/2011 9:01 AM, Petr Vobornik wrote: >>> 1) In the tests is a call of entity_init method which isn't defined >>> anywhere. It leads to JS error and test fail. >> >> Fixed. It was defined in another patch: > > ACK Pushed to master. -- Endi S. Dewata From simo at redhat.com Fri Nov 18 16:50:47 2011 From: simo at redhat.com (Simo Sorce) Date: Fri, 18 Nov 2011 11:50:47 -0500 Subject: [Freeipa-devel] [PATCHES] #1950 Implement CLDAP server for AD trust introperability In-Reply-To: <20111118150716.GF8631@localhost.localdomain> References: <1321567251.30630.169.camel@willson.li.ssimo.org> <20111118150716.GF8631@localhost.localdomain> Message-ID: <1321635047.30630.191.camel@willson.li.ssimo.org> On Fri, 2011-11-18 at 16:07 +0100, Sumit Bose wrote: > On Thu, Nov 17, 2011 at 05:00:51PM -0500, Simo Sorce wrote: > > Attached find a series of patches that implement a CLDAP server as a > > dirsrv plugin. > > > > The server right now responds only to a very limited class of requests, > > as observed on the wire. But it can be easily expanded to respond to > > additional requests as needed. > > > > Tested against windows 2008 with which I had create a trust. > > To test you need Sumit's uncommitted adtrust code to successfully create > > the trust and provision the IPA tree with the right data. > > The patch prevents dirsrv from shutting down cleanly, so NACK. But > otherwise it is working great. I will rebase my patches on top of yours, > fix some missing bits and send them here as well. Ok here is a rebase/modified patchset. I reworked it to have even less churn between patches (you should see only additions and no deletions. I also added a pipe() to deal with the worker thread shutdown. This allows us to interrupt the poll() at any time and at the same time tell the worker thread it is time to end. I tested a dirsrv shutdown in gdb and it works as expected. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-NT-domain-GUID-attribute.patch Type: text/x-patch Size: 2764 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Create-skeleton-CLDAP-server-as-a-DS-plugin.patch Type: text/x-patch Size: 18817 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0003-ipa-cldap-Implement-worker-thread.patch Type: text/x-patch Size: 3607 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0004-ipa-cldap-Decode-CLDAP-request.patch Type: text/x-patch Size: 5694 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0005-ipa-cldap-Create-netlogon-blob.patch Type: text/x-patch Size: 13556 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0006-ipa-cldap-send-cldap-reply.patch Type: text/x-patch Size: 2326 bytes Desc: not available URL: From jdennis at redhat.com Fri Nov 18 17:54:48 2011 From: jdennis at redhat.com (John Dennis) Date: Fri, 18 Nov 2011 12:54:48 -0500 Subject: [Freeipa-devel] [PATCH 53/53] ticket 2022 - modify codebase to utilize IPALogManager, obsoletes logging In-Reply-To: <1321603499.4192.16.camel@balmora.brq.redhat.com> References: <201111151941.pAFJfC5u017345@int-mx09.intmail.prod.int.phx2.redhat.com> <1321446953.18210.31.camel@balmora.brq.redhat.com> <4EC533A0.1020006@redhat.com> <1321603499.4192.16.camel@balmora.brq.redhat.com> Message-ID: <4EC69BE8.8050608@redhat.com> On 11/18/2011 03:04 AM, Martin Kosek wrote: > Ok, these are all valid arguments. I know that abusing "logging" is not > right, I was just afraid that diverging master branch from our 2.x > branch that much would do more harm (and extra effort whenever a patch > from master is backported to 2.x branch) that misusing the "logging" > name. I would like some second opinion on that before we make the call. > > I think having root_logger exported to our tools and calling > root_logger.warning(...) etc. looks better than the previous version > (log_mgr.root_logger.warning()). I'm happy to change log_mgr.root_logger.warning() to root_logger.warning() and I think it's a good suggestion, thank you. I'll do that and try to figure why there seems to be a rebase issue at the same time. FWIW, we use the root_logger in a lot of places where a dedicated logger would make more sense. For example the install-xxx tool should probably be using a logger named "install-xxx" instead of the unnamed anonymous root logger, but that's a change for another day. It would also clean the excessive use of the root logger. In other words inside install-xxx it would do: logger = log_mgr.get_logger('install-xxx') logger.warning() or if was a class instance: log_manager.get_logger(self, True) self.warning() -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From edewata at redhat.com Fri Nov 18 18:27:17 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Fri, 18 Nov 2011 12:27:17 -0600 Subject: [Freeipa-devel] [PATCH] 315 Added commands into metadata. In-Reply-To: <4EC67B6E.2070409@redhat.com> References: <4EC5602E.3030004@redhat.com> <4EC63A3E.5080200@redhat.com> <4EC67B6E.2070409@redhat.com> Message-ID: <4EC6A385.4050605@redhat.com> On 11/18/2011 9:36 AM, Endi Sukma Dewata wrote: > Now the methods metadata seem to be a subset of commands metadata, so we > probably can change the UI to use commands metadata and not pull the > methods metadata. I did this change in the updated patch. It seems to be working fine. > In the JSON API itself the parameters are specified as options, so the > order shouldn't matter to the UI. Is there a way to define the execute() > using unordered keywords? I'm trying to avoid changing the method > signature again in the future. I replaced takes_args with takes_options which takes care the ordering problem. I verified the old UI way of calling json_metadata still works. -- Endi S. Dewata -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freeipa-edewata-0315-2-Added-commands-into-metadata.patch URL: From sbose at redhat.com Fri Nov 18 20:05:10 2011 From: sbose at redhat.com (Sumit Bose) Date: Fri, 18 Nov 2011 21:05:10 +0100 Subject: [Freeipa-devel] [PATCHES] #1950 Implement CLDAP server for AD trust introperability In-Reply-To: <1321635047.30630.191.camel@willson.li.ssimo.org> References: <1321567251.30630.169.camel@willson.li.ssimo.org> <20111118150716.GF8631@localhost.localdomain> <1321635047.30630.191.camel@willson.li.ssimo.org> Message-ID: <20111118200510.GG8631@localhost.localdomain> On Fri, Nov 18, 2011 at 11:50:47AM -0500, Simo Sorce wrote: > On Fri, 2011-11-18 at 16:07 +0100, Sumit Bose wrote: > > On Thu, Nov 17, 2011 at 05:00:51PM -0500, Simo Sorce wrote: > > > Attached find a series of patches that implement a CLDAP server as a > > > dirsrv plugin. > > > > > > The server right now responds only to a very limited class of requests, > > > as observed on the wire. But it can be easily expanded to respond to > > > additional requests as needed. > > > > > > Tested against windows 2008 with which I had create a trust. > > > To test you need Sumit's uncommitted adtrust code to successfully create > > > the trust and provision the IPA tree with the right data. > > > > The patch prevents dirsrv from shutting down cleanly, so NACK. But > > otherwise it is working great. I will rebase my patches on top of yours, > > fix some missing bits and send them here as well. > > Ok here is a rebase/modified patchset. > I reworked it to have even less churn between patches (you should see > only additions and no deletions. > > I also added a pipe() to deal with the worker thread shutdown. > This allows us to interrupt the poll() at any time and at the same time > tell the worker thread it is time to end. > > I tested a dirsrv shutdown in gdb and it works as expected. ... and it even works without gdb, ACL bye, Sumit > > Simo. > > -- > Simo Sorce * Red Hat, Inc * New York From ayoung at redhat.com Fri Nov 18 21:02:52 2011 From: ayoung at redhat.com (Adam Young) Date: Fri, 18 Nov 2011 16:02:52 -0500 Subject: [Freeipa-devel] Putting the A in IPA Message-ID: <4EC6C7FC.1020300@redhat.com> So the crazy systemd folks are at it again: https://docs.google.com/document/pub?id=1IC9yOXj7j6cdLLxWEBAGRL6wl97tFxgjLUEHIX3MSTs&pli=1 This is a re implementation of logging much the way the systmd was a reimplementation of init. Assume that it is going to get implemented, does this give us a better tool for audit? From dpal at redhat.com Fri Nov 18 21:23:43 2011 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 18 Nov 2011 16:23:43 -0500 Subject: [Freeipa-devel] Putting the A in IPA In-Reply-To: <4EC6C7FC.1020300@redhat.com> References: <4EC6C7FC.1020300@redhat.com> Message-ID: <4EC6CCDF.7090602@redhat.com> On 11/18/2011 04:02 PM, Adam Young wrote: > So the crazy systemd folks are at it again: > > https://docs.google.com/document/pub?id=1IC9yOXj7j6cdLLxWEBAGRL6wl97tFxgjLUEHIX3MSTs&pli=1 > > > > This is a re implementation of logging much the way the systmd was a > reimplementation of init. > > Assume that it is going to get implemented, does this give us a > better tool for audit? > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel I am all over it. Stay tuned. -- Thank you, Dmitri Pal Sr. Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jdennis at redhat.com Fri Nov 18 21:37:41 2011 From: jdennis at redhat.com (John Dennis) Date: Fri, 18 Nov 2011 16:37:41 -0500 Subject: [Freeipa-devel] Putting the A in IPA In-Reply-To: <4EC6C7FC.1020300@redhat.com> References: <4EC6C7FC.1020300@redhat.com> Message-ID: <4EC6D025.7030704@redhat.com> On 11/18/2011 04:02 PM, Adam Young wrote: > So the crazy systemd folks are at it again: > > https://docs.google.com/document/pub?id=1IC9yOXj7j6cdLLxWEBAGRL6wl97tFxgjLUEHIX3MSTs&pli=1 Yup, we've been tracking it. sgrubb is supposed to do a tech talk on "standardized logging & audit" soon claiming there is a industry wide consensus forming on how do portable system neutral logging and audit securely. Holding my comments until after hearing what Steve has to say. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Fri Nov 18 21:54:00 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 18 Nov 2011 16:54:00 -0500 Subject: [Freeipa-devel] [PATCH 53/53] ticket 2022 - modify codebase to utilize IPALogManager, obsoletes logging In-Reply-To: <1321603499.4192.16.camel@balmora.brq.redhat.com> References: <201111151941.pAFJfC5u017345@int-mx09.intmail.prod.int.phx2.redhat.com> <1321446953.18210.31.camel@balmora.brq.redhat.com> <4EC533A0.1020006@redhat.com> <1321603499.4192.16.camel@balmora.brq.redhat.com> Message-ID: <4EC6D3F8.8020405@redhat.com> Martin Kosek wrote: > On Thu, 2011-11-17 at 11:17 -0500, John Dennis wrote: >> On 11/16/2011 07:35 AM, Martin Kosek wrote: >>> On Tue, 2011-11-15 at 14:41 -0500, John Dennis wrote: >>>> >>>> -- >>>> John Dennis >>>> >>>> Looking to carve out IT costs? >>>> www.redhat.com/carveoutcosts/ >>> >>> Hi John, >>> >>> thanks for your patch, that was a fair amount of work there :-) I tested >>> them both and have few comments: >>> >>> 1) Patch 53 will need rebasing >> >> O.K. will rebase. >>> >>> 2) Logging for install tools (ipa-server-install, ...) is now broken. >>> DEBUG level messages should get to /var/log/ipaserver-install.log even >>> when the installer is run without --debug. My >>> ipaserver-install.log/ipaserver-uninstall.log was silent when i run just >>> `ipa-server-install' without --debug flag >> >> Yes you're right, it's supposed to always log to the file at the debug >> level. Sorry that slipped by, it's a trivial one word fix in >> standard_logging_setup(), done. > > Ok. > >> >>> 3) Patch 53 touches most of our Python sources as it changes all >>> logging.* calls to log_mgr.root_logger.* calls. I think this may cause >>> problems (comment 1 is a good example). For example we may not be able >>> to cherry-pick most of new patches from master branch to ipa-2-1 branch >>> or other branches that do not use log_mgr. >>> >>> Additionally, typing "log_mgr.root_logger.*" whenever I want to log >>> anything seems a bit awkward to me. These issues are avoidable, however. >>> Since all our Python code use just the root_logger, can we just simply >>> export root logger in log_manager.py namespace and that import it as >>> "logging" in other modules? Then we would save *a lot* of changes across >>> all our code. Something like this: >>> >>> log_manager.py: >>> --------------- >>> log_mgr = IPALogManager() >>> log_mgr.configure(dict(default_level='error', >>> handlers=[dict(name='console', >>> stream=sys.stderr)]), >>> configure_state='default') >>> +root_logger = log_mgr.root_logger >>> >>> ipa-server-install: >>> ------------------- >>> +from ipapython.log_manager import root_logger as logginng >>> >>> ... >>> # now, all logging will work as usual: >>> logging.error('Some files have not been restored, see /var/lib/ipa/sysrestore/sysrestore.index') >> >> Cherry picking patches between branches with out performing any editing >> to apply the patch is a nice goal but perhaps not realistic. However >> you're absolutely right this change introduces an extra step during >> patch/rebase. >> >> A larger question is should we continue to support the idea of having >> log functions hung off a module name. Here's my thinking on that. But >> skip to item 3 if you want a compromise solution. >> >> >> 1) I'm a bit concerned that co-opting or borrowing the logging module >> name and using it for an entirely different purpose. Importing the name >> "logging" and not have it be the Python logging module as most Python >> programmers would be the source of needless confusion. Personally I >> think I would be annoyed if someone had played that game because nothing >> would be as I expect. The name "logging" implies the logging module with >> all sorts of names, constants, objects, classes, sub-modules, etc. >> associated with it, none of that would retain any of it's conventional >> meaning. I much prefer clarity and to call something what it is and not >> obfuscate for temporary expediency. >> >> 2) The current Python logging implementation is broken IMHO and was the >> source of some of our problems, I'm not sure I want to perpetuate that. >> It exported a set of "convenience" functions bound to a global. It made >> using the logging module easier to use via globals but at the expense of >> all sorts of conflicts when more than one client of logging tried to >> manipulate the same shared global objects. >> >> The logging.debug(), etc. names are a bogus mapping of a top level >> module symbol to a bound method of a specific instantiated object, an >> object available externally only by calling a defined API to retrieve >> that object from singleton manager by name. To my mind this is the >> classic abuse of globals every textbook warns against. It's just bad >> programming best avoided IMHO. >> >> Why is it O.K. to have a global log_mgr object for IPA? >> >> a) Because it belongs to us, it's not shared with every piece of >> Python code which imports the logging module. >> >> b) It's not abusing a top level module symbol to be a short circuit >> reference into some other instantiated object. The root logger >> belongs to the log_mgr instance shared by loaded IPA components. >> Think of it as an IPA singleton. >> >> 3) Can we have a shorter name as a compromise? Sure, I have no problems >> with that. We already have the log_mgr as a single imported instance. >> I don't see why we couldn't have also import the symbol >> "root_logger" which is log_mgr.root_logger. I'm not keen on shorting >> the name much more, for instance to "logger" because that's too >> generic and obscures the fact it's the root logger, an important >> distinction to maintain. >> >> An other option would be to have a LogManager object export the >> debug(), info(), error(), critical(),& exception() methods where >> they are bound to the LogManager's root_logger instance. This is >> somewhat akin to what the standard Python logging module, except >> they performed the binding on the module level and not on an >> instance (the source of many problems). Those log methods *must* >> be bound to a Logger object as such I'm not entirely keen on >> binding them to a LogManager instance because once again you >> end up providing an abused short cut which hides what is really >> going on. I think it would benefit programmers if they knew they >> were logging to a Logger and if so which one, because after all >> once you try to customize things like "which loggers emit which >> which messages" it would be really helpful to know you're dealing >> with a Logger and which one it is and not obscure which one it is >> to shorten the reference to it as a convenience via some short cut. >> But if there is a consensus I could bend on this issue. >> >> That means we could have one of the two (I prefer the first) >> >> root_logger.debug() >> >> -or- >> >> log_mgr.debug() # Not a fan of this because it hides the important >> # concept of debug() being bound to the root_logger. >> >> As for: >> >> logging.debug() >> >> For the reasons pointed out in 1 I'd like to avoid that kind of >> symbol abuse. >> >> > > Ok, these are all valid arguments. I know that abusing "logging" is not > right, I was just afraid that diverging master branch from our 2.x > branch that much would do more harm (and extra effort whenever a patch > from master is backported to 2.x branch) that misusing the "logging" > name. I would like some second opinion on that before we make the call. > > I think having root_logger exported to our tools and calling > root_logger.warning(...) etc. looks better than the previous version > (log_mgr.root_logger.warning()). > I have to agree. Patch management is going to be a pain but we knew this would happen sooner or later. IMHO it is worth it to get saner logging. ACK on root_logger vs log_mgr.root_logger.xxx. rob From jdennis at redhat.com Sat Nov 19 17:50:35 2011 From: jdennis at redhat.com (John Dennis) Date: Sat, 19 Nov 2011 12:50:35 -0500 Subject: [Freeipa-devel] [PATCH 53/53] ticket 2022 - modify codebase to utilize IPALogManager, obsoletes logging In-Reply-To: <201111151941.pAFJfC5u017345@int-mx09.intmail.prod.int.phx2.redhat.com> References: <201111151941.pAFJfC5u017345@int-mx09.intmail.prod.int.phx2.redhat.com> Message-ID: <4EC7EC6B.7000008@redhat.com> New patch attached which addresses all previous comments. * rebased against origin/master * new ipa specific module added, ipa_log_manager which has the IPA specific log manager and exports the root_logger symbol. * all logging to the root logger is now root_logger.xxx() instead of the more verbose log_mgr.root_logger.xxx() * fixed the log level in the file logging, now set to debug. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-ticket-2022-modify-codebase-to-utilize-IPALogManager.patch Type: text/x-patch Size: 164994 bytes Desc: not available URL: From abokovoy at redhat.com Sun Nov 20 17:21:13 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Sun, 20 Nov 2011 19:21:13 +0200 Subject: [Freeipa-devel] [WIP] FreeIPA extensibility guide Message-ID: <20111120172113.GA31881@redhat.com> Hi, Since I've joined FreeIPA project in summer 2011, I tried to understand the framework and its interworkings. Unfortunately, there is always not enough documentation on the code and it is far easier to write the code than to document it all along the way. The result is the following draft -- far from being finished, perhaps far from being correct. I'd love to hear constructive critics to improve the guide. Whole chapters in plain English would be awesome contribution! Examples are a bit stupid ones that is I know, but they serve purpose of intentionally distracting from thinking about the framework only in "LDAP management tool" direction. I have generated two versions, one for online reading, another for printing, if anyone is like me and consumes information from dead trees better than from LCD/LED/eInk. http://abbra.fedorapeople.org/freeipa-extensibility.html http://abbra.fedorapeople.org/freeipa-extensibility.pdf -- / Alexander Bokovoy From abokovoy at redhat.com Mon Nov 21 08:04:08 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 21 Nov 2011 10:04:08 +0200 Subject: [Freeipa-devel] [PATCH] 4 ipa-client-install fails when not run as root In-Reply-To: <4EC65DB0.5010208@redhat.com> References: <4EC65DB0.5010208@redhat.com> Message-ID: <20111121080407.GA13007@redhat.com> On Fri, 18 Nov 2011, Ondrej Hamada wrote: > https://fedorahosted.org/freeipa/ticket/2123 > > ipa-client-install was failing and returning traceback when > wasn't run by root. It was caused by logging initialization that > was taking part before the root privileges check. To correct it, > the privileges check was moved before the logging initialization. ACK. -- / Alexander Bokovoy From abokovoy at redhat.com Mon Nov 21 08:21:39 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 21 Nov 2011 10:21:39 +0200 Subject: [Freeipa-devel] [PATCH] 905 add plugin framework to updater In-Reply-To: <4EC420C7.40607@redhat.com> References: <4EC420C7.40607@redhat.com> Message-ID: <20111121082138.GB13007@redhat.com> On Wed, 16 Nov 2011, Rob Crittenden wrote: > There are two reasons for the plugin framework: > 1. To provide a way of doing manual/complex LDAP changes without having > to keep extending ldapupdate.py (like we did with managed entries). > 2. Allows for better control of restarts. > > There are two types of plugins, preop and postop. A preop plugin runs > before any file-based updates are loaded. A postop plugin runs after > all file-based updates are applied. > > A preop plugin may update LDAP directly or craft update entries to > be applied with the file-based updates. > > Either a preop or postop plugin may attempt to restart the dirsrv > instance. The instance is only restartable if ipa-ldap-updater is > being executed as root. A warning is printed if a restart is > requested for a non-root user. > > Plugins are not executed by default. This is so we can use > ldapupdate to apply simple updates in commands like ipa-nis-manage. > > rob > >From e8c632c0a17c5fad3792d4f741976161d245fec6 Mon Sep 17 00:00:00 2001 > From: Rob Crittenden > Date: Wed, 16 Nov 2011 15:37:56 -0500 > Subject: [PATCH] Add plugin framework to LDAP updates. > > There are two reasons for the plugin framework: > 1. To provide a way of doing manual/complex LDAP changes without having > to keep extending ldapupdate.py (like we did with managed entries). > 2. Allows for better control of restarts. > > There are two types of plugins, preop and postop. A preop plugin runs > before any file-based updates are loaded. A postop plugin runs after > all file-based updates are applied. > > A preop plugin may update LDAP directly or craft update entries to be > applied with the file-based updates. > > Either a preop or postop plugin may attempt to restart the dirsrv instance. > The instance is only restartable if ipa-ldap-updater is being executed > as root. A warning is printed if a restart is requested for a non-root > user. > > Plugins are not executed by default. This is so we can use ldapupdate > to apply simple updates in commands like ipa-nis-manage. ACK though I have few considerations, perhaps asking for a separate patch. This patch adds Updater namespace available in all contexts albeit populated only in 'updates'/'installer' modes. I wonder if we could re-factor the API class code so that until you have called finalize(), you can add more namespaces. This would prevent need to modify basic initialization every time new namespace is added -- especially given that namespaces like Updater are specific to single utility. I know, technically you could simply replace api.register by a new instance and dispose the old one before going to finalize()... Another thought is to allow API to accept list of pairs (context, path) to simplify loading plugins. By default 'ipalib' plugins would be loaded, and for others, API.add_plugins(context, path) would be adding more, then API.load_plugins() would iterate over these pairs and if context is the same as in API.env, load them. This way plugins will be truly generic and we'll gain ability to use per application plugins without any additional changes in the core framework. -- / Alexander Bokovoy From mkosek at redhat.com Mon Nov 21 08:30:51 2011 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 21 Nov 2011 09:30:51 +0100 Subject: [Freeipa-devel] [PATCH 53/53] ticket 2022 - modify codebase to utilize IPALogManager, obsoletes logging In-Reply-To: <4EC7EC6B.7000008@redhat.com> References: <201111151941.pAFJfC5u017345@int-mx09.intmail.prod.int.phx2.redhat.com> <4EC7EC6B.7000008@redhat.com> Message-ID: <1321864251.2698.4.camel@balmora.brq.redhat.com> Hello Jonh, the log calls look OK now. I think it will be much easier to use that way. Your patch 0052 and this new patch 0001 applies to current master now. (btw. it would be great if they followed FreeIPA patch policy, i.e. freeipa-jdennis-52-....patch - it would be easier for me to find and manipulate these files) However, pylint discovered 2 problems, I don't think this will fly: $ ./make-lint ipapython/ipa_log_manager.py:176: [E0602, IPALogManager.configure_from_env] Undefined variable 'parse_log_level' ipapython/ipa_log_manager.py:190: [E0602, IPALogManager.configure_from_env] Undefined variable 'parse_log_level' Martin On Sat, 2011-11-19 at 12:50 -0500, John Dennis wrote: > New patch attached which addresses all previous comments. > > * rebased against origin/master > > * new ipa specific module added, ipa_log_manager which > has the IPA specific log manager and exports the > root_logger symbol. > > * all logging to the root logger is now root_logger.xxx() > instead of the more verbose log_mgr.root_logger.xxx() > > * fixed the log level in the file logging, now set to debug. From abokovoy at redhat.com Mon Nov 21 08:32:13 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 21 Nov 2011 10:32:13 +0200 Subject: [Freeipa-devel] [PATCH] 8 Add DNS service records for Windows In-Reply-To: <20111014191648.GE4622@localhost.localdomain> References: <20111014101557.GC4622@localhost.localdomain> <20111014151202.GD4622@localhost.localdomain> <20111014172150.GA17515@redhat.com> <20111014191648.GE4622@localhost.localdomain> Message-ID: <20111121083211.GD13007@redhat.com> Hi Sumit, On Fri, 14 Oct 2011, Sumit Bose wrote: > > It would make more clear what is the default and that it is really > > optional setting -- I'm thinking from the perspective of maintenance > > of the code in future. > > Thank you for your comments, new version attached. Finally got to test it. ACK. -- / Alexander Bokovoy From mkosek at redhat.com Mon Nov 21 08:41:30 2011 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 21 Nov 2011 09:41:30 +0100 Subject: [Freeipa-devel] [PATCH] 4 ipa-client-install fails when not run as root In-Reply-To: <20111121080407.GA13007@redhat.com> References: <4EC65DB0.5010208@redhat.com> <20111121080407.GA13007@redhat.com> Message-ID: <1321864890.2698.5.camel@balmora.brq.redhat.com> On Mon, 2011-11-21 at 10:04 +0200, Alexander Bokovoy wrote: > On Fri, 18 Nov 2011, Ondrej Hamada wrote: > > https://fedorahosted.org/freeipa/ticket/2123 > > > > ipa-client-install was failing and returning traceback when > > wasn't run by root. It was caused by logging initialization that > > was taking part before the root privileges check. To correct it, > > the privileges check was moved before the logging initialization. > ACK. > > Pushed to master, ipa-2-1. Martin From jkuncar at redhat.com Mon Nov 21 09:55:51 2011 From: jkuncar at redhat.com (Jiri Kuncar) Date: Mon, 21 Nov 2011 04:55:51 -0500 (EST) Subject: [Freeipa-devel] [PATCH] Skip invalid records In-Reply-To: <4EC66162.5080505@redhat.com> Message-ID: Patch was fixed according to coding guidelines and Adam's suggestions. Regards, Jiri ----- Original Message ----- From: "Adam Tkac" To: "Jiri Kuncar" Cc: freeipa-devel at redhat.com Sent: Friday, November 18, 2011 2:45:06 PM Subject: Re: [Freeipa-devel] [PATCH] Skip invalid records On 11/18/2011 02:18 PM, Jiri Kuncar wrote: Bind-dyndb-ldap: Don't fail to load zone when it contains invalid records. Your patch seems fine for me, thanks. Please check my comments inside the patch. Regards, Adam Needed by ticket #36 https://fedorahosted.org/bind-dyndb-ldap/ticket/36 0001-Skip-bad-records.patch >From 37a8430b4a5555d33fc727a6b6a19998710b37da Mon Sep 17 00:00:00 2001 From: Jiri Kuncar Date: Fri, 18 Nov 2011 08:04:53 -0500 Subject: [PATCH] Skip bad records. Signed-off-by: Jiri Kuncar --- src/ldap_helper.c | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/src/ldap_helper.c b/src/ldap_helper.c index 6fb57ce..029398c 100644 --- a/src/ldap_helper.c +++ b/src/ldap_helper.c @@ -1080,9 +1080,11 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na for (entry = HEAD(ldap_conn->ldap_entries); entry != NULL; entry = NEXT(entry, link)) { - CHECK(ldap_parse_rrentry(mctx, entry, ldap_conn, - origin, ldap_inst->fake_mname, - string, rdatalist)); + if (ISC_R_SUCCESS != ldap_parse_rrentry(mctx, entry, ldap_conn, + origin, ldap_inst->fake_mname, + string, rdatalist)) { Please keep consistency with rest of code. Use if (function() != result), not if (result != function()). + log_error("Failed to parse RR entry"); What about log_error("Failed to parse RR entry (%s)", string); ? This will help admin to track which RR is bad. + } } /* Cache RRs */ _______________________________________________ Freeipa-devel mailing list Freeipa-devel at redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Skip-invalid-records.patch Type: text/x-patch Size: 1097 bytes Desc: not available URL: From jcholast at redhat.com Mon Nov 21 13:51:00 2011 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 21 Nov 2011 14:51:00 +0100 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <4EC2B933.8030902@redhat.com> References: <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <4EB152A7.5010303@redhat.com> <20111102151114.GD14373@redhat.com> <4EB15E73.8070402@redhat.com> <20111102163720.GA23092@redhat.com> <20111102163848.GB23092@redhat.com> <4EB2614E.8090307@redhat.com> <20111103114352.GB26667@redhat.com> <4EBA54D1.4040700@redhat.com> <20111109105904.GA5529@redhat.com> <4EBA9066.4070600@redhat.com> <4EC2B933.8030902@redhat.com> Message-ID: <4ECA5744.3090700@redhat.com> Dne 15.11.2011 20:10, Rob Crittenden napsal(a): > Jan Cholasta wrote: >> Dne 9.11.2011 11:59, Alexander Bokovoy napsal(a): >>> On Wed, 09 Nov 2011, Jan Cholasta wrote: >>>>>> would't suffer from a facelift - currently it is messy, hard to read >>>>>> code, with bits nobody ever used or uses anymore, some of the >>>>>> docstrings and comments aren't correct, etc.) >>>>> A review of API class is a good idea. However, I think it is currently >>>>> enough what is in proposed patch as it gets you 2-3x speed increase >>>>> already. >>>> >>>> I've already started experimenting with the API class, hopefully it >>>> will result in something useful :) >>> Good. >>> >>>>> This is something I'd like to keep as it has great value for all >>>>> end-users and it requires loading all Python files in ipalib/plugins >>>>> (and in ipaserver/plugins for server side). >>>> >>>> I'm not suggesting we should skip importing the modules. The plugin >>>> initialization process consists of these 3 steps: >>>> >>>> 1. import plugin modules >>>> 2. create plugin instances >>>> 3. finalize plugin instances >>>> >>>> What I'm suggesting is that we do steps 2 and 3 on-demand. We can't >>>> do step 1 on-demand, because we have no way of knowing what plugins >>>> are available without importing the modules. (Both your and my patch >>>> does only step 3 on-demand, but I think we can do better than that.) >>> Agreed. >>> >>>>>> Anyway, if we decide to go with your solution, please make sure >>>>>> *all* the plugins that are used are finalized (because of the >>>>>> locking) and add a new api.env attribute which controls the lazy >>>>>> finalization, so that the behavior can be overriden (actually I have >>>>>> already done that - see attached patch - just use >>>>>> "api.env.plugins_on_demand" instead of "api.env.context == 'cli'"). >>>>> Done. >>>>> >>>> >>>> + if not self.__dict__['_Plugin__finalized']: >>>> + self.finalize() >>>> >>>> This doesn't seem right, coding style-wise. If you want to access >>>> the property from outside the Plugin class, why did you name-mangle >>>> it in the first place? >>> It is supposed to be internal detail of a Plugin. I agree it stinks a >>> bit here. :) >>> >>> >>>> def finalize(self): >>>> """ >>>> """ >>>> + if not self.__finalized: >>>> + self.__finalized = True >>>> + else: >>>> + return >>>> if not is_production_mode(self): >>>> lock(self) >>>> >>>> Finalize is supposed to be called just once, so IMO it would be >>>> better not to do the check and let it throw the exception. >>> On contrary, I think sequential finalize() calls should do nothing. >>> This is at least what I expect from a finalized object -- no-op. >> >> In my patch, finalize() throws an exception if called more than once, >> but ensure_finalized() does nothing for plugins that are already >> finalized. So there are both kinds of behavior available. >> >>> >>>> + for i in ('Object', 'Property', 'Backend'): >>>> + if i in self: >>>> + namespaces.append(self[i]) >>>> >>>> AFAIK the framework is supposed to be generally usable for other >>>> projects in the future. With that in mind, I think we can't afford >>>> to hard-code things like this. >>> That's true. As you managed to get around without hardcoding, we can >>> drop this part. >>> >>>> Anyway, I've made a patch that uses data descriptors for the trap >>>> objects (which is IMHO more elegant than what you do). I've also >>>> managed to add on-demand finalization to Object and Attribute >>>> subclasses by moving the set-up code from set_api() to finalize() >>>> (it doesn't add much performance, though). See attachment. >>> I read through the code and I like it! Did you get the whole test >>> suite running with it? There are some parts of tests that expect >>> non-finalized objects to have None properties while they are now not >>> None. >> >> I always run the test suite ;-) >> >> All the unit tests succeed (they do when run with my patch 54 applied, >> which fixes failures of some unrelated unit tests). >> >>> >>> If so, preliminary ACK for your patch from reading it if you would >>> make sure on_finalize() allows multiple calls and would make a better >>> commit message. ;) >> >> on_finalize() shouldn't be called directly (perhaps I should rename it >> to _on_finalize to emphasize that...?) >> >> I'll work on the commit message. As usual, it is the hardest part of the >> patch for me. >> >>> >>> I'll try to arrange testing myself today/tomorrow. >>> >>>> The numbers from my VM: >>>> >>>> master abbra jcholast >>>> $ time ipa >>>> real 0m1.288s 0m0.619s 0m0.601s >>>> user 0m1.103s 0m0.478s 0m0.451s >>>> sys 0m0.161s 0m0.126s 0m0.133s >>>> >>>> $ time ipa user-find >>>> real 0m1.897s 0m1.253s 0m1.235s >>>> user 0m1.119s 0m0.486s 0m0.488s >>>> sys 0m0.160s 0m0.160s 0m0.136s >>>> >>>> $ time ipa help >>>> real 0m1.299s 0m0.629s 0m0.600s >>>> user 0m1.094s 0m0.477s 0m0.446s >>>> sys 0m0.183s 0m0.136s 0m0.140s >>> Looks good (your VM is supposedly at the same farm as mine so numbers >>> are comparable). There is anyway great variation in execution times in >>> VMs so 600ms vs 629ms is roughly the same. >> >> Yep. >> >>> >>> Thanks a lot! I think it is great advance over the original approach. >> >> Thanks for the kind words :-) >> >> Honza >> > > > Just a couple of questions/clarifications: > > 1. I think more documentation is needed for on_finalize(). The name > isn't particularly descriptive. If I read it properly you are providing > an alternate way to override finalize(), right? Yes. I have split finalize() to finalize() (the method to be called to finalize the plugin) and on_finalize() (the method to be overridden by plugin developer to implement custom finalization), so that I can guard the finalizing code from recursively calling itself (which happens when you try to read an unfinalized finalize_attr attribute from within a finalize() call). > > 2. Changing to a partition in Attribute is not equivalent to the regular > expression previously used. Why loosen the requirements? Well, I didn't think it through. I have reverted the change. > > 3. I think you should document in-line where you have block calls to > finalize_attr() so that future developers know to add the stub. OK. > > 4. What is the purpose of the read-lock in finalize? I guess it is the usual - to prevent modifying the object after the API is finalized. The commit that introduced it is , the commit that made it conditional is . > > It generally looks good and provides impressive performance improvements. > > rob Updated patch attached. Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-57.2-lazy-finalize.patch Type: text/x-patch Size: 14791 bytes Desc: not available URL: From jcholast at redhat.com Mon Nov 21 16:10:47 2011 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 21 Nov 2011 17:10:47 +0100 Subject: [Freeipa-devel] [PATCH] 55 Parse comma-separated lists of values in all parameter types In-Reply-To: <4EBD7279.8030106@redhat.com> References: <4EA00403.9060703@redhat.com> <4EA583AB.6030906@redhat.com> <4EA5877B.2020606@redhat.com> <4EA59620.30506@redhat.com> <4EB45837.2020606@redhat.com> <4EB7B023.1030004@redhat.com> <4EBD7279.8030106@redhat.com> Message-ID: <4ECA7807.4080606@redhat.com> Dne 11.11.2011 20:07, Rob Crittenden napsal(a): > Jan Cholasta wrote: >> Dne 4.11.2011 22:25, Rob Crittenden napsal(a): >>> Jan Cholasta wrote: >>>> Dne 24.10.2011 17:42, Rob Crittenden napsal(a): >>>>> Jan Cholasta wrote: >>>>>> Dne 20.10.2011 13:20, Jan Cholasta napsal(a): >>>>>>> Parse comma-separated lists of values in all parameter types. This >>>>>>> can >>>>>>> enabled for a specific parameter by setting the "csvlist" option to >>>>>>> True. >>>>>>> >>>>>>> Remove List parameter type and replace all occurences with Str with >>>>>>> csvlist enabled. >>>>>>> >>>>>>> https://fedorahosted.org/freeipa/ticket/2007 >>>>>>> >>>>>>> This change will be useful for >>>>>>> https://fedorahosted.org/freeipa/ticket/1487 and >>>>>>> https://fedorahosted.org/freeipa/ticket/1847 >>>>>>> >>>>>>> Unit tests show no regressions. >>>>>>> >>>>>>> Honza >>>>>>> >>>>>> >>>>>> Self-NACK - I have noticed that the batch command no longer works. >>>>>> >>>>>> Updated patch attached. >>>>>> >>>>>> Honza >>>>> >>>>> What is the benefit of this over the List parameter type? >>>>> >>>>> rob >>>> >>>> Mainly because the List parameter type is just a hack. This is the >>>> right >>>> thing to do if we want to use comma-separated lists of parameters of >>>> any >>>> type, with all the validation and other parameter type-specific >>>> features. >>>> >>>> For example, I've added a new parameter type for IP addresses in my >>>> patch 46 >>>> (http://www.redhat.com/archives/freeipa-devel/2011-September/msg00187.html) >>>> >>>> >>>> >>>> and use it for A and AAAA DNS records. Without this patch, we can >>>> either >>>> use List for the record parameters and lose validation in >>>> dnsrecord-find >>>> (because it is based on crud.Search, which strips all the custom >>>> validation rules - like _validate_ipaddr - from the command parameters, >>>> which is one of the causes of #1627) or use IPAddress for the record >>>> parameters and lose the ability to specify them as comma-separated list >>>> of values. With this patch, we can have both comma-separated lists and >>>> validation at the same time. >>>> >>>> Besides, the patch is not as big as it looks like, all the interesting >>>> stuff is in ipalib/parameters.py, everything else is just >>>> search-and-replace. Also I need it to fix #1487 and #1847 without doing >>>> ugly hacks. >>>> >>>> Honza >>>> >>> >>> I think this would constitute a major version change. >> >> I'm not sure about that, as the patch doesn't break API compatibility - >> a string containing a comma-separated list of values is used for list >> parameters both with and without the patch. >> >>> >>> One downside is you can no longer tell in the help with arguments take a >>> CSV and which don't. >> >> Why not? A simple look at csvlist value should provide enough >> information. >> >>> >>> I think the CSV-related Parameter options should all begin with csv, >>> separator and skipspace. >> >> OK. >> >>> >>> The batch command may eventually be made into a command, how will that >>> affect the Any type? >> >> Batch currently uses List for the "methods" parameter not because of its >> CSV capability, but because it doesn't do any type conversion and >> validation on the values. This allows it to use values of the form >> "{u'params': [args_list, kwargs_dict], u'method': method_name}". The Any >> parameter type exists so that this can still be done without List - it's >> basically a single-valued version of List (i.e. Any(csvlist=True) is >> equivalent to List()). >> >> IMO if batch is ever made into a command, it would have to be redesigned >> not to use List/Any, so that it can be used from CLI with validation of >> the parameter values. Any can then be easily removed. >> >>> >>> It otherwise seems to work in my spot-testing. >>> >>> rob >> >> Honza >> > > Given that we want to maintain backward API compatibility can you make > sure older clients will work with this? My gut tells me it will since > this is really a marshaling issue but I don't want to assume. Just tested a few commands that use CSV (selfservice-find, dnszone-add, dnszone-del) from an unpatched client and everything seems to be working fine. > > thanks > > rob I have updated the patch with your suggestions and rebased it on top of current master. See attachment. Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-55.2-comma-separated-list.patch Type: text/x-patch Size: 131954 bytes Desc: not available URL: From edewata at redhat.com Mon Nov 21 18:18:22 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 21 Nov 2011 12:18:22 -0600 Subject: [Freeipa-devel] [PATCH] 316 Added support for radio buttons in table widget. Message-ID: <4ECA95EE.7080205@redhat.com> The table widget has been modified to support single-valued attribute using radio buttons needed by some facets in HBAC Test. The widget now uses 'pagination' flag to determine whether to show the pagination control. The test data has also been updated. Ticket #388 -- Endi S. Dewata -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freeipa-edewata-0316-Added-support-for-radio-buttons-in-table-widget.patch URL: From edewata at redhat.com Mon Nov 21 18:23:41 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 21 Nov 2011 12:23:41 -0600 Subject: [Freeipa-devel] [PATCH] 317 Fixed entity metadata resolution. Message-ID: <4ECA972D.7030904@redhat.com> The current code assumes that an entity will always have a corresponding LDAPObject on the server, so it looks for the metadata in a fixed location. This assumption doesn't work for HBAC Test since it is a Command, not an LDAPObject, so the metadata has to be obtained from a different location. A new method get_default_metadata() has been added to allow each entity to find the metadata from the correct location. Ticket #388 -- Endi S. Dewata -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freeipa-edewata-0317-Fixed-entity-metadata-resolution.patch URL: From edewata at redhat.com Mon Nov 21 18:29:50 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 21 Nov 2011 12:29:50 -0600 Subject: [Freeipa-devel] [PATCH] 318 Refactored facet.load(). Message-ID: <4ECA989E.8050300@redhat.com> The load() in IPA.facet has been modified to accept the complete data returned by the server instead of just the result. This is needed by HBAC Test to access other attributes returned in the test result. Ticket #388 -- Endi S. Dewata -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freeipa-edewata-0318-Refactored-facet.load.patch URL: From edewata at redhat.com Mon Nov 21 18:38:37 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 21 Nov 2011 12:38:37 -0600 Subject: [Freeipa-devel] [PATCH] 319 Added HBAC Test page. Message-ID: <4ECA9AAD.6070502@redhat.com> This is the initial implementation of HBAC Test page. Currently it can select user, source/target group, service, rules, and execute the test. Other functionalities to be implemented include the search filter, external users/hosts, back/next buttons, validation, styling, and internalization. Ticket #388 Demo: http://edewata.fedorapeople.org/freeipa/install/ui/#hbac=hbactest&policy=hbac&navigation=policy -- Endi S. Dewata From edewata at redhat.com Mon Nov 21 18:39:43 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 21 Nov 2011 12:39:43 -0600 Subject: [Freeipa-devel] [PATCH] 319 Added HBAC Test page. In-Reply-To: <4ECA9AAD.6070502@redhat.com> References: <4ECA9AAD.6070502@redhat.com> Message-ID: <4ECA9AEF.2090506@redhat.com> On 11/21/2011 12:38 PM, Endi Sukma Dewata wrote: > This is the initial implementation of HBAC Test page. Currently it > can select user, source/target group, service, rules, and execute > the test. Other functionalities to be implemented include the search > filter, external users/hosts, back/next buttons, validation, styling, > and internalization. > > Ticket #388 > > Demo: > http://edewata.fedorapeople.org/freeipa/install/ui/#hbac=hbactest&policy=hbac&navigation=policy Patch attached. -- Endi S. Dewata -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freeipa-edewata-0319-Added-HBAC-Test-page.patch URL: From dpal at redhat.com Mon Nov 21 19:39:40 2011 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 21 Nov 2011 14:39:40 -0500 Subject: [Freeipa-devel] [PATCH] 319 Added HBAC Test page. In-Reply-To: <4ECA9AAD.6070502@redhat.com> References: <4ECA9AAD.6070502@redhat.com> Message-ID: <4ECAA8FC.8040502@redhat.com> On 11/21/2011 01:38 PM, Endi Sukma Dewata wrote: > This is the initial implementation of HBAC Test page. Currently it > can select user, source/target group, service, rules, and execute > the test. Other functionalities to be implemented include the search > filter, external users/hosts, back/next buttons, validation, styling, > and internalization. > > Ticket #388 > > Demo: > http://edewata.fedorapeople.org/freeipa/install/ui/#hbac=hbactest&policy=hbac&navigation=policy > > It is unclear what Back and Next buttons mean on the "who" screen. The height of the scrollable part is smaller then on the user page. Just click Identity panel to see the difference. In the user list and any other list the size of the scollable part should be adjusted to the number of lines in the view. Play with Ctrl+ and Ctrl- to see ho the line at the bottom cats through an item in the list. Looks inaccurate. -- Thank you, Dmitri Pal Sr. Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From edewata at redhat.com Mon Nov 21 20:58:44 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 21 Nov 2011 14:58:44 -0600 Subject: [Freeipa-devel] [PATCH] 319 Added HBAC Test page. In-Reply-To: <4ECAA8FC.8040502@redhat.com> References: <4ECA9AAD.6070502@redhat.com> <4ECAA8FC.8040502@redhat.com> Message-ID: <4ECABB84.9010806@redhat.com> On 11/21/2011 1:39 PM, Dmitri Pal wrote: >> Demo: >> http://edewata.fedorapeople.org/freeipa/install/ui/#hbac=hbactest&policy=hbac&navigation=policy > It is unclear what Back and Next buttons mean on the "who" screen. Here are the mockup images: http://edewata.fedorapeople.org/images/hbactest/ There shouldn't be a Back button in the Who screen, and there should be an arrow in the Next button. This will be fixed in the next patch. > The height of the scrollable part is smaller then on the user page. Just > click Identity panel to see the difference. Right now this table is using a fixed height, that's why it's smaller. We can make it taller but the risk is you might need to scroll down to see the Next buttons. To make the table adjust according to the window height like in the user page is going to require some CSS tricks. This will be addressed a little later. > In the user list and any other list the size of the scollable part > should be adjusted to the number of lines in the view. Play with Ctrl+ > and Ctrl- to see ho the line at the bottom cats through an item in the > list. Looks inaccurate. In general the table body (the scrollable part) does adjust according to the window size. But yes, there's a slight overlap between the table body and the footer, probably due to the size of the input field in the footer. This will be fixed separately. -- Endi S. Dewata From simo at redhat.com Mon Nov 21 23:53:26 2011 From: simo at redhat.com (Simo Sorce) Date: Mon, 21 Nov 2011 18:53:26 -0500 Subject: [Freeipa-devel] [PATCHES] #1950 Implement CLDAP server for AD trust introperability In-Reply-To: <20111118200510.GG8631@localhost.localdomain> References: <1321567251.30630.169.camel@willson.li.ssimo.org> <20111118150716.GF8631@localhost.localdomain> <1321635047.30630.191.camel@willson.li.ssimo.org> <20111118200510.GG8631@localhost.localdomain> Message-ID: <1321919606.6750.25.camel@willson.li.ssimo.org> On Fri, 2011-11-18 at 21:05 +0100, Sumit Bose wrote: > On Fri, Nov 18, 2011 at 11:50:47AM -0500, Simo Sorce wrote: > > On Fri, 2011-11-18 at 16:07 +0100, Sumit Bose wrote: > > > On Thu, Nov 17, 2011 at 05:00:51PM -0500, Simo Sorce wrote: > > > > Attached find a series of patches that implement a CLDAP server as a > > > > dirsrv plugin. > > > > > > > > The server right now responds only to a very limited class of requests, > > > > as observed on the wire. But it can be easily expanded to respond to > > > > additional requests as needed. > > > > > > > > Tested against windows 2008 with which I had create a trust. > > > > To test you need Sumit's uncommitted adtrust code to successfully create > > > > the trust and provision the IPA tree with the right data. > > > > > > The patch prevents dirsrv from shutting down cleanly, so NACK. But > > > otherwise it is working great. I will rebase my patches on top of yours, > > > fix some missing bits and send them here as well. > > > > Ok here is a rebase/modified patchset. > > I reworked it to have even less churn between patches (you should see > > only additions and no deletions. > > > > I also added a pipe() to deal with the worker thread shutdown. > > This allows us to interrupt the poll() at any time and at the same time > > tell the worker thread it is time to end. > > > > I tested a dirsrv shutdown in gdb and it works as expected. > > ... and it even works without gdb, ACL Thanks. Pushed to master. Simo. -- Simo Sorce * Red Hat, Inc * New York From edewata at redhat.com Tue Nov 22 01:31:42 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 21 Nov 2011 19:31:42 -0600 Subject: [Freeipa-devel] [PATCH] 320 Fixed navigation buttons for HBAC Test. Message-ID: <4ECAFB7E.1090805@redhat.com> The Back, Next, and New Test buttons in HBAC Test have been fixed to work properly. Ticket #388 -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-edewata-0320-Fixed-navigation-buttons-for-HBAC-Test.patch Type: text/x-patch Size: 9331 bytes Desc: not available URL: From edewata at redhat.com Tue Nov 22 01:33:51 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 21 Nov 2011 19:33:51 -0600 Subject: [Freeipa-devel] [PATCH] 321 Fixed search filter in HBAC Test. Message-ID: <4ECAFBFF.1000305@redhat.com> The search filter in HBAC Test has been fixed to work properly. Ticket #388 -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-edewata-0321-Fixed-search-filter-in-HBAC-Test.patch Type: text/x-patch Size: 1451 bytes Desc: not available URL: From jdennis at redhat.com Tue Nov 22 16:23:31 2011 From: jdennis at redhat.com (John Dennis) Date: Tue, 22 Nov 2011 11:23:31 -0500 Subject: [Freeipa-devel] [PATCH 53/53] ticket 2022 - modify codebase to utilize IPALogManager, obsoletes logging In-Reply-To: <1321864251.2698.4.camel@balmora.brq.redhat.com> References: <201111151941.pAFJfC5u017345@int-mx09.intmail.prod.int.phx2.redhat.com> <4EC7EC6B.7000008@redhat.com> <1321864251.2698.4.camel@balmora.brq.redhat.com> Message-ID: <4ECBCC83.7090106@redhat.com> On 11/21/2011 03:30 AM, Martin Kosek wrote: > Hello Jonh, > > the log calls look OK now. I think it will be much easier to use that > way. > > Your patch 0052 and this new patch 0001 applies to current master now. > (btw. it would be great if they followed FreeIPA patch policy, i.e. > freeipa-jdennis-52-....patch - it would be easier for me to find and > manipulate these files) Fixed. This attachment exactly mirrors the original with the patch number information. > > However, pylint discovered 2 problems, I don't think this will fly: > > $ ./make-lint > ipapython/ipa_log_manager.py:176: [E0602, > IPALogManager.configure_from_env] Undefined variable 'parse_log_level' > ipapython/ipa_log_manager.py:190: [E0602, > IPALogManager.configure_from_env] Undefined variable 'parse_log_level' Fixed. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: 0053-ticket-2022-modify-codebase-to-utilize-IPALogManager.patch Type: text/x-patch Size: 165101 bytes Desc: not available URL: From jdennis at redhat.com Tue Nov 22 19:15:50 2011 From: jdennis at redhat.com (John Dennis) Date: Tue, 22 Nov 2011 14:15:50 -0500 Subject: [Freeipa-devel] [PATCH 29/29] Remove calls to has_managed_entries() Message-ID: <201111221915.pAMJFosL022705@int-mx09.intmail.prod.int.phx2.redhat.com> At one point in time we couldn't depend on the 389-ds having the managed entries plugin so this code was added to support both versions. It is no longer needed. https://fedorahosted.org/freeipa/ticket/1242 -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: 0029-Remove-calls-to-has_managed_entries.patch Type: text/x-patch Size: 3274 bytes Desc: not available URL: From jdennis at redhat.com Tue Nov 22 19:16:32 2011 From: jdennis at redhat.com (John Dennis) Date: Tue, 22 Nov 2011 14:16:32 -0500 Subject: [Freeipa-devel] [PATCH 54/54] ticet 2135 - enhance make-test for debugging Message-ID: <201111221916.pAMJGWjr004962@int-mx01.intmail.prod.int.phx2.redhat.com> Add option to run the unit tests under PDB, but enter PDB before anything executes so you can set breakpoints etc. Add option to disable nosetest from capturing stdout, handy for adding print statements when debugging. Note: the old --pdb arg behavior is now --pdb-errors, --pdb is now used to enter pdb upon test execution. This minor change in command line behavior much more accurately reflects the arg name and the behavior. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: 0054-ticet-2135-enhance-make-test-for-debugging.patch Type: text/x-patch Size: 2329 bytes Desc: not available URL: From jdennis at redhat.com Tue Nov 22 19:17:50 2011 From: jdennis at redhat.com (John Dennis) Date: Tue, 22 Nov 2011 14:17:50 -0500 Subject: [Freeipa-devel] [PATCH 29/29] Remove calls to has_managed_entries() In-Reply-To: <201111221915.pAMJFosL022705@int-mx09.intmail.prod.int.phx2.redhat.com> References: <201111221915.pAMJFosL022705@int-mx09.intmail.prod.int.phx2.redhat.com> Message-ID: <4ECBF55E.1060200@redhat.com> Please disregard this email, it was the result of finger fumble. Send to /dev/null instead. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Tue Nov 22 23:01:29 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 22 Nov 2011 18:01:29 -0500 Subject: [Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib In-Reply-To: <4ECA5744.3090700@redhat.com> References: <4EAEB8A8.7010805@redhat.com> <20111102104206.GB14373@redhat.com> <4EB152A7.5010303@redhat.com> <20111102151114.GD14373@redhat.com> <4EB15E73.8070402@redhat.com> <20111102163720.GA23092@redhat.com> <20111102163848.GB23092@redhat.com> <4EB2614E.8090307@redhat.com> <20111103114352.GB26667@redhat.com> <4EBA54D1.4040700@redhat.com> <20111109105904.GA5529@redhat.com> <4EBA9066.4070600@redhat.com> <4EC2B933.8030902@redhat.com> <4ECA5744.3090700@redhat.com> Message-ID: <4ECC29C9.2000205@redhat.com> Jan Cholasta wrote: > Dne 15.11.2011 20:10, Rob Crittenden napsal(a): >> Jan Cholasta wrote: >>> Dne 9.11.2011 11:59, Alexander Bokovoy napsal(a): >>>> On Wed, 09 Nov 2011, Jan Cholasta wrote: >>>>>>> would't suffer from a facelift - currently it is messy, hard to read >>>>>>> code, with bits nobody ever used or uses anymore, some of the >>>>>>> docstrings and comments aren't correct, etc.) >>>>>> A review of API class is a good idea. However, I think it is >>>>>> currently >>>>>> enough what is in proposed patch as it gets you 2-3x speed increase >>>>>> already. >>>>> >>>>> I've already started experimenting with the API class, hopefully it >>>>> will result in something useful :) >>>> Good. >>>> >>>>>> This is something I'd like to keep as it has great value for all >>>>>> end-users and it requires loading all Python files in ipalib/plugins >>>>>> (and in ipaserver/plugins for server side). >>>>> >>>>> I'm not suggesting we should skip importing the modules. The plugin >>>>> initialization process consists of these 3 steps: >>>>> >>>>> 1. import plugin modules >>>>> 2. create plugin instances >>>>> 3. finalize plugin instances >>>>> >>>>> What I'm suggesting is that we do steps 2 and 3 on-demand. We can't >>>>> do step 1 on-demand, because we have no way of knowing what plugins >>>>> are available without importing the modules. (Both your and my patch >>>>> does only step 3 on-demand, but I think we can do better than that.) >>>> Agreed. >>>> >>>>>>> Anyway, if we decide to go with your solution, please make sure >>>>>>> *all* the plugins that are used are finalized (because of the >>>>>>> locking) and add a new api.env attribute which controls the lazy >>>>>>> finalization, so that the behavior can be overriden (actually I have >>>>>>> already done that - see attached patch - just use >>>>>>> "api.env.plugins_on_demand" instead of "api.env.context == 'cli'"). >>>>>> Done. >>>>>> >>>>> >>>>> + if not self.__dict__['_Plugin__finalized']: >>>>> + self.finalize() >>>>> >>>>> This doesn't seem right, coding style-wise. If you want to access >>>>> the property from outside the Plugin class, why did you name-mangle >>>>> it in the first place? >>>> It is supposed to be internal detail of a Plugin. I agree it stinks a >>>> bit here. :) >>>> >>>> >>>>> def finalize(self): >>>>> """ >>>>> """ >>>>> + if not self.__finalized: >>>>> + self.__finalized = True >>>>> + else: >>>>> + return >>>>> if not is_production_mode(self): >>>>> lock(self) >>>>> >>>>> Finalize is supposed to be called just once, so IMO it would be >>>>> better not to do the check and let it throw the exception. >>>> On contrary, I think sequential finalize() calls should do nothing. >>>> This is at least what I expect from a finalized object -- no-op. >>> >>> In my patch, finalize() throws an exception if called more than once, >>> but ensure_finalized() does nothing for plugins that are already >>> finalized. So there are both kinds of behavior available. >>> >>>> >>>>> + for i in ('Object', 'Property', 'Backend'): >>>>> + if i in self: >>>>> + namespaces.append(self[i]) >>>>> >>>>> AFAIK the framework is supposed to be generally usable for other >>>>> projects in the future. With that in mind, I think we can't afford >>>>> to hard-code things like this. >>>> That's true. As you managed to get around without hardcoding, we can >>>> drop this part. >>>> >>>>> Anyway, I've made a patch that uses data descriptors for the trap >>>>> objects (which is IMHO more elegant than what you do). I've also >>>>> managed to add on-demand finalization to Object and Attribute >>>>> subclasses by moving the set-up code from set_api() to finalize() >>>>> (it doesn't add much performance, though). See attachment. >>>> I read through the code and I like it! Did you get the whole test >>>> suite running with it? There are some parts of tests that expect >>>> non-finalized objects to have None properties while they are now not >>>> None. >>> >>> I always run the test suite ;-) >>> >>> All the unit tests succeed (they do when run with my patch 54 applied, >>> which fixes failures of some unrelated unit tests). >>> >>>> >>>> If so, preliminary ACK for your patch from reading it if you would >>>> make sure on_finalize() allows multiple calls and would make a better >>>> commit message. ;) >>> >>> on_finalize() shouldn't be called directly (perhaps I should rename it >>> to _on_finalize to emphasize that...?) >>> >>> I'll work on the commit message. As usual, it is the hardest part of the >>> patch for me. >>> >>>> >>>> I'll try to arrange testing myself today/tomorrow. >>>> >>>>> The numbers from my VM: >>>>> >>>>> master abbra jcholast >>>>> $ time ipa >>>>> real 0m1.288s 0m0.619s 0m0.601s >>>>> user 0m1.103s 0m0.478s 0m0.451s >>>>> sys 0m0.161s 0m0.126s 0m0.133s >>>>> >>>>> $ time ipa user-find >>>>> real 0m1.897s 0m1.253s 0m1.235s >>>>> user 0m1.119s 0m0.486s 0m0.488s >>>>> sys 0m0.160s 0m0.160s 0m0.136s >>>>> >>>>> $ time ipa help >>>>> real 0m1.299s 0m0.629s 0m0.600s >>>>> user 0m1.094s 0m0.477s 0m0.446s >>>>> sys 0m0.183s 0m0.136s 0m0.140s >>>> Looks good (your VM is supposedly at the same farm as mine so numbers >>>> are comparable). There is anyway great variation in execution times in >>>> VMs so 600ms vs 629ms is roughly the same. >>> >>> Yep. >>> >>>> >>>> Thanks a lot! I think it is great advance over the original approach. >>> >>> Thanks for the kind words :-) >>> >>> Honza >>> >> >> >> Just a couple of questions/clarifications: >> >> 1. I think more documentation is needed for on_finalize(). The name >> isn't particularly descriptive. If I read it properly you are providing >> an alternate way to override finalize(), right? > > Yes. I have split finalize() to finalize() (the method to be called to > finalize the plugin) and on_finalize() (the method to be overridden by > plugin developer to implement custom finalization), so that I can guard > the finalizing code from recursively calling itself (which happens when > you try to read an unfinalized finalize_attr attribute from within a > finalize() call). > >> >> 2. Changing to a partition in Attribute is not equivalent to the regular >> expression previously used. Why loosen the requirements? > > Well, I didn't think it through. I have reverted the change. > >> >> 3. I think you should document in-line where you have block calls to >> finalize_attr() so that future developers know to add the stub. > > OK. > >> >> 4. What is the purpose of the read-lock in finalize? > > I guess it is the usual - to prevent modifying the object after the API > is finalized. The commit that introduced it is > , > the commit that made it conditional is > . > > >> >> It generally looks good and provides impressive performance improvements. >> >> rob > > Updated patch attached. > > Honza > ACK, pushed to master. rob From simo at redhat.com Wed Nov 23 00:10:54 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 22 Nov 2011 19:10:54 -0500 Subject: [Freeipa-devel] [PATCH] #2122 Fix PAC re-signing Message-ID: <1322007054.6750.76.camel@willson.li.ssimo.org> In some cases the KDC will decide to use a different checksum type when re-signing a PAC to include it in a service ticket. This is common in a cross-realm trust with AD as most AD DCs will use a HMAC-MD5-RC4 checksum while IPA's KDC will instead choose to use HMAC-SHA-AES when re-signing the PAC. In current MIT code re-signing a PAC with a signature that differs in length from the original will cause an error. While MIT should handle this properly, we use the workaround of regenerating the PAC from scratch so that there is no trace of the previous signatures. Tested while obtaining a cross-realm ticket from an AD domain against a service belonging to an IPA domain. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-ipa-kdb-Support-re-signing-PAC-with-different-checks.patch Type: text/x-patch Size: 2688 bytes Desc: not available URL: From mkosek at redhat.com Wed Nov 23 08:39:23 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 23 Nov 2011 09:39:23 +0100 Subject: [Freeipa-devel] [PATCH 52/52] ticket 2022 - Add log manager module In-Reply-To: <4EC2BFAD.3030107@redhat.com> References: <4EC2BFAD.3030107@redhat.com> Message-ID: <1322037563.27504.0.camel@balmora.brq.redhat.com> On Tue, 2011-11-15 at 14:38 -0500, John Dennis wrote: > ticket 2022 - Add log manager module > > Browseable online HTML doc can be found here, probably easier to read > than the patch (plus it includes the source code) > > http://jdennis.fedorapeople.org/log_manager_doc/ > ACK. Pushed to master. Martin From mkosek at redhat.com Wed Nov 23 08:39:42 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 23 Nov 2011 09:39:42 +0100 Subject: [Freeipa-devel] [PATCH 53/53] ticket 2022 - modify codebase to utilize IPALogManager, obsoletes logging In-Reply-To: <4ECBCC83.7090106@redhat.com> References: <201111151941.pAFJfC5u017345@int-mx09.intmail.prod.int.phx2.redhat.com> <4EC7EC6B.7000008@redhat.com> <1321864251.2698.4.camel@balmora.brq.redhat.com> <4ECBCC83.7090106@redhat.com> Message-ID: <1322037582.27504.1.camel@balmora.brq.redhat.com> On Tue, 2011-11-22 at 11:23 -0500, John Dennis wrote: > On 11/21/2011 03:30 AM, Martin Kosek wrote: > > Hello Jonh, > > > > the log calls look OK now. I think it will be much easier to use that > > way. > > > > Your patch 0052 and this new patch 0001 applies to current master now. > > (btw. it would be great if they followed FreeIPA patch policy, i.e. > > freeipa-jdennis-52-....patch - it would be easier for me to find and > > manipulate these files) > > Fixed. This attachment exactly mirrors the original with the patch > number information. > > > > > However, pylint discovered 2 problems, I don't think this will fly: > > > > $ ./make-lint > > ipapython/ipa_log_manager.py:176: [E0602, > > IPALogManager.configure_from_env] Undefined variable 'parse_log_level' > > ipapython/ipa_log_manager.py:190: [E0602, > > IPALogManager.configure_from_env] Undefined variable 'parse_log_level' > > Fixed. > > ACK. Pushed to master. Martin From mkosek at redhat.com Wed Nov 23 08:52:39 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 23 Nov 2011 09:52:39 +0100 Subject: [Freeipa-devel] [PATCH] 55 Parse comma-separated lists of values in all parameter types In-Reply-To: <4ECA7807.4080606@redhat.com> References: <4EA00403.9060703@redhat.com> <4EA583AB.6030906@redhat.com> <4EA5877B.2020606@redhat.com> <4EA59620.30506@redhat.com> <4EB45837.2020606@redhat.com> <4EB7B023.1030004@redhat.com> <4EBD7279.8030106@redhat.com> <4ECA7807.4080606@redhat.com> Message-ID: <1322038359.27504.4.camel@balmora.brq.redhat.com> On Mon, 2011-11-21 at 17:10 +0100, Jan Cholasta wrote: > Dne 11.11.2011 20:07, Rob Crittenden napsal(a): > > Jan Cholasta wrote: > >> Dne 4.11.2011 22:25, Rob Crittenden napsal(a): > >>> Jan Cholasta wrote: > >>>> Dne 24.10.2011 17:42, Rob Crittenden napsal(a): > >>>>> Jan Cholasta wrote: > >>>>>> Dne 20.10.2011 13:20, Jan Cholasta napsal(a): > >>>>>>> Parse comma-separated lists of values in all parameter types. This > >>>>>>> can > >>>>>>> enabled for a specific parameter by setting the "csvlist" option to > >>>>>>> True. > >>>>>>> > >>>>>>> Remove List parameter type and replace all occurences with Str with > >>>>>>> csvlist enabled. > >>>>>>> > >>>>>>> https://fedorahosted.org/freeipa/ticket/2007 > >>>>>>> > >>>>>>> This change will be useful for > >>>>>>> https://fedorahosted.org/freeipa/ticket/1487 and > >>>>>>> https://fedorahosted.org/freeipa/ticket/1847 > >>>>>>> > >>>>>>> Unit tests show no regressions. > >>>>>>> > >>>>>>> Honza > >>>>>>> > >>>>>> > >>>>>> Self-NACK - I have noticed that the batch command no longer works. > >>>>>> > >>>>>> Updated patch attached. > >>>>>> > >>>>>> Honza > >>>>> > >>>>> What is the benefit of this over the List parameter type? > >>>>> > >>>>> rob > >>>> > >>>> Mainly because the List parameter type is just a hack. This is the > >>>> right > >>>> thing to do if we want to use comma-separated lists of parameters of > >>>> any > >>>> type, with all the validation and other parameter type-specific > >>>> features. > >>>> > >>>> For example, I've added a new parameter type for IP addresses in my > >>>> patch 46 > >>>> (http://www.redhat.com/archives/freeipa-devel/2011-September/msg00187.html) > >>>> > >>>> > >>>> > >>>> and use it for A and AAAA DNS records. Without this patch, we can > >>>> either > >>>> use List for the record parameters and lose validation in > >>>> dnsrecord-find > >>>> (because it is based on crud.Search, which strips all the custom > >>>> validation rules - like _validate_ipaddr - from the command parameters, > >>>> which is one of the causes of #1627) or use IPAddress for the record > >>>> parameters and lose the ability to specify them as comma-separated list > >>>> of values. With this patch, we can have both comma-separated lists and > >>>> validation at the same time. > >>>> > >>>> Besides, the patch is not as big as it looks like, all the interesting > >>>> stuff is in ipalib/parameters.py, everything else is just > >>>> search-and-replace. Also I need it to fix #1487 and #1847 without doing > >>>> ugly hacks. > >>>> > >>>> Honza > >>>> > >>> > >>> I think this would constitute a major version change. > >> > >> I'm not sure about that, as the patch doesn't break API compatibility - > >> a string containing a comma-separated list of values is used for list > >> parameters both with and without the patch. > >> > >>> > >>> One downside is you can no longer tell in the help with arguments take a > >>> CSV and which don't. > >> > >> Why not? A simple look at csvlist value should provide enough > >> information. > >> > >>> > >>> I think the CSV-related Parameter options should all begin with csv, > >>> separator and skipspace. > >> > >> OK. > >> > >>> > >>> The batch command may eventually be made into a command, how will that > >>> affect the Any type? > >> > >> Batch currently uses List for the "methods" parameter not because of its > >> CSV capability, but because it doesn't do any type conversion and > >> validation on the values. This allows it to use values of the form > >> "{u'params': [args_list, kwargs_dict], u'method': method_name}". The Any > >> parameter type exists so that this can still be done without List - it's > >> basically a single-valued version of List (i.e. Any(csvlist=True) is > >> equivalent to List()). > >> > >> IMO if batch is ever made into a command, it would have to be redesigned > >> not to use List/Any, so that it can be used from CLI with validation of > >> the parameter values. Any can then be easily removed. > >> > >>> > >>> It otherwise seems to work in my spot-testing. > >>> > >>> rob > >> > >> Honza > >> > > > > Given that we want to maintain backward API compatibility can you make > > sure older clients will work with this? My gut tells me it will since > > this is really a marshaling issue but I don't want to assume. > > Just tested a few commands that use CSV (selfservice-find, dnszone-add, > dnszone-del) from an unpatched client and everything seems to be working > fine. > > > > > thanks > > > > rob > > I have updated the patch with your suggestions and rebased it on top of > current master. See attachment. > > Honza > Your patch crashes the ./make-lint script: # git apply freeipa-jcholast-55.2-comma-separated-list.patch # ./make-lint Traceback (most recent call last): File "./make-lint", line 239, in sys.exit(main()) File "./make-lint", line 210, in main linter.check(files) File "/usr/lib/python2.7/site-packages/pylint/lint.py", line 493, in check self.check_astng_module(astng, walker, rawcheckers) File "/usr/lib/python2.7/site-packages/pylint/lint.py", line 565, in check_astng_module walker.walk(astng) File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk self.walk(child) File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk self.walk(child) File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk self.walk(child) File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk self.walk(child) File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk self.walk(child) File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk self.walk(child) File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk self.walk(child) File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk self.walk(child) File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 521, in walk cb(astng) File "./make-lint", line 98, in visit_getattr ignored = self._find_ignored_attrs(owner) File "./make-lint", line 82, in _find_ignored_attrs for klass in self._related_classes(owner): File "./make-lint", line 74, in _related_classes for base in klass.ancestors(): File "/usr/lib/python2.7/site-packages/logilab/astng/bases.py", line 48, in __getattr__ return getattr(self._proxied, name) AttributeError: 'Function' object has no attribute 'ancestors' Besides that, I consider this patch very useful. It is also a prerequisite for my new DNS API patch as I use custom non-List based parameter types and I need CSV functionality for them. Martin From jcholast at redhat.com Wed Nov 23 10:14:01 2011 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 23 Nov 2011 11:14:01 +0100 Subject: [Freeipa-devel] [PATCH] 55 Parse comma-separated lists of values in all parameter types In-Reply-To: <1322038359.27504.4.camel@balmora.brq.redhat.com> References: <4EA00403.9060703@redhat.com> <4EA583AB.6030906@redhat.com> <4EA5877B.2020606@redhat.com> <4EA59620.30506@redhat.com> <4EB45837.2020606@redhat.com> <4EB7B023.1030004@redhat.com> <4EBD7279.8030106@redhat.com> <4ECA7807.4080606@redhat.com> <1322038359.27504.4.camel@balmora.brq.redhat.com> Message-ID: <4ECCC769.7070303@redhat.com> Dne 23.11.2011 09:52, Martin Kosek napsal(a): > On Mon, 2011-11-21 at 17:10 +0100, Jan Cholasta wrote: >> Dne 11.11.2011 20:07, Rob Crittenden napsal(a): >>> Jan Cholasta wrote: >>>> Dne 4.11.2011 22:25, Rob Crittenden napsal(a): >>>>> Jan Cholasta wrote: >>>>>> Dne 24.10.2011 17:42, Rob Crittenden napsal(a): >>>>>>> Jan Cholasta wrote: >>>>>>>> Dne 20.10.2011 13:20, Jan Cholasta napsal(a): >>>>>>>>> Parse comma-separated lists of values in all parameter types. This >>>>>>>>> can >>>>>>>>> enabled for a specific parameter by setting the "csvlist" option to >>>>>>>>> True. >>>>>>>>> >>>>>>>>> Remove List parameter type and replace all occurences with Str with >>>>>>>>> csvlist enabled. >>>>>>>>> >>>>>>>>> https://fedorahosted.org/freeipa/ticket/2007 >>>>>>>>> >>>>>>>>> This change will be useful for >>>>>>>>> https://fedorahosted.org/freeipa/ticket/1487 and >>>>>>>>> https://fedorahosted.org/freeipa/ticket/1847 >>>>>>>>> >>>>>>>>> Unit tests show no regressions. >>>>>>>>> >>>>>>>>> Honza >>>>>>>>> >>>>>>>> >>>>>>>> Self-NACK - I have noticed that the batch command no longer works. >>>>>>>> >>>>>>>> Updated patch attached. >>>>>>>> >>>>>>>> Honza >>>>>>> >>>>>>> What is the benefit of this over the List parameter type? >>>>>>> >>>>>>> rob >>>>>> >>>>>> Mainly because the List parameter type is just a hack. This is the >>>>>> right >>>>>> thing to do if we want to use comma-separated lists of parameters of >>>>>> any >>>>>> type, with all the validation and other parameter type-specific >>>>>> features. >>>>>> >>>>>> For example, I've added a new parameter type for IP addresses in my >>>>>> patch 46 >>>>>> (http://www.redhat.com/archives/freeipa-devel/2011-September/msg00187.html) >>>>>> >>>>>> >>>>>> >>>>>> and use it for A and AAAA DNS records. Without this patch, we can >>>>>> either >>>>>> use List for the record parameters and lose validation in >>>>>> dnsrecord-find >>>>>> (because it is based on crud.Search, which strips all the custom >>>>>> validation rules - like _validate_ipaddr - from the command parameters, >>>>>> which is one of the causes of #1627) or use IPAddress for the record >>>>>> parameters and lose the ability to specify them as comma-separated list >>>>>> of values. With this patch, we can have both comma-separated lists and >>>>>> validation at the same time. >>>>>> >>>>>> Besides, the patch is not as big as it looks like, all the interesting >>>>>> stuff is in ipalib/parameters.py, everything else is just >>>>>> search-and-replace. Also I need it to fix #1487 and #1847 without doing >>>>>> ugly hacks. >>>>>> >>>>>> Honza >>>>>> >>>>> >>>>> I think this would constitute a major version change. >>>> >>>> I'm not sure about that, as the patch doesn't break API compatibility - >>>> a string containing a comma-separated list of values is used for list >>>> parameters both with and without the patch. >>>> >>>>> >>>>> One downside is you can no longer tell in the help with arguments take a >>>>> CSV and which don't. >>>> >>>> Why not? A simple look at csvlist value should provide enough >>>> information. >>>> >>>>> >>>>> I think the CSV-related Parameter options should all begin with csv, >>>>> separator and skipspace. >>>> >>>> OK. >>>> >>>>> >>>>> The batch command may eventually be made into a command, how will that >>>>> affect the Any type? >>>> >>>> Batch currently uses List for the "methods" parameter not because of its >>>> CSV capability, but because it doesn't do any type conversion and >>>> validation on the values. This allows it to use values of the form >>>> "{u'params': [args_list, kwargs_dict], u'method': method_name}". The Any >>>> parameter type exists so that this can still be done without List - it's >>>> basically a single-valued version of List (i.e. Any(csvlist=True) is >>>> equivalent to List()). >>>> >>>> IMO if batch is ever made into a command, it would have to be redesigned >>>> not to use List/Any, so that it can be used from CLI with validation of >>>> the parameter values. Any can then be easily removed. >>>> >>>>> >>>>> It otherwise seems to work in my spot-testing. >>>>> >>>>> rob >>>> >>>> Honza >>>> >>> >>> Given that we want to maintain backward API compatibility can you make >>> sure older clients will work with this? My gut tells me it will since >>> this is really a marshaling issue but I don't want to assume. >> >> Just tested a few commands that use CSV (selfservice-find, dnszone-add, >> dnszone-del) from an unpatched client and everything seems to be working >> fine. >> >>> >>> thanks >>> >>> rob >> >> I have updated the patch with your suggestions and rebased it on top of >> current master. See attachment. >> >> Honza >> > > Your patch crashes the ./make-lint script: > > # git apply freeipa-jcholast-55.2-comma-separated-list.patch > # ./make-lint > Traceback (most recent call last): > File "./make-lint", line 239, in > sys.exit(main()) > File "./make-lint", line 210, in main > linter.check(files) > File "/usr/lib/python2.7/site-packages/pylint/lint.py", line 493, in check > self.check_astng_module(astng, walker, rawcheckers) > File "/usr/lib/python2.7/site-packages/pylint/lint.py", line 565, in check_astng_module > walker.walk(astng) > File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk > self.walk(child) > File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk > self.walk(child) > File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk > self.walk(child) > File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk > self.walk(child) > File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk > self.walk(child) > File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk > self.walk(child) > File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk > self.walk(child) > File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 524, in walk > self.walk(child) > File "/usr/lib/python2.7/site-packages/pylint/utils.py", line 521, in walk > cb(astng) > File "./make-lint", line 98, in visit_getattr > ignored = self._find_ignored_attrs(owner) > File "./make-lint", line 82, in _find_ignored_attrs > for klass in self._related_classes(owner): > File "./make-lint", line 74, in _related_classes > for base in klass.ancestors(): > File "/usr/lib/python2.7/site-packages/logilab/astng/bases.py", line 48, in __getattr__ > return getattr(self._proxied, name) > AttributeError: 'Function' object has no attribute 'ancestors' > Well, the crash is obviously not related to the patch. Fedora 16 uses a newer version of pylint (0.24), which causes the crash (it works fine on Fedora 15 with pylint 0.22). I have opened a ticket to resolve that: https://fedorahosted.org/freeipa/ticket/2136 > > > Besides that, I consider this patch very useful. It is also a > prerequisite for my new DNS API patch as I use custom non-List based > parameter types and I need CSV functionality for them. > > Martin > Let me add that we agreed with Martin that parsing of CSV values should probably be done on client-side, because it's merely a convenience for CLI users, not something the protocol depends (or should depend) on. The downside is that a change like this breaks the API, so we can't do it right now, but it is one of the things that we should do once we decided to bump the major version of the API. Honza -- Jan Cholasta From sbose at redhat.com Wed Nov 23 10:53:11 2011 From: sbose at redhat.com (Sumit Bose) Date: Wed, 23 Nov 2011 11:53:11 +0100 Subject: [Freeipa-devel] [PATCH] #2122 Fix PAC re-signing In-Reply-To: <1322007054.6750.76.camel@willson.li.ssimo.org> References: <1322007054.6750.76.camel@willson.li.ssimo.org> Message-ID: <20111123105311.GG2205@localhost.localdomain> On Tue, Nov 22, 2011 at 07:10:54PM -0500, Simo Sorce wrote: > In some cases the KDC will decide to use a different checksum type when > re-signing a PAC to include it in a service ticket. > > This is common in a cross-realm trust with AD as most AD DCs will use a > HMAC-MD5-RC4 checksum while IPA's KDC will instead choose to use > HMAC-SHA-AES when re-signing the PAC. > > In current MIT code re-signing a PAC with a signature that differs in > length from the original will cause an error. > > While MIT should handle this properly, we use the workaround of > regenerating the PAC from scratch so that there is no trace of the > previous signatures. > > Tested while obtaining a cross-realm ticket from an AD domain against a > service belonging to an IPA domain. I see "authdata (kdb) handling failure: Cannot allocate memory" in krb5kdc.log when trying to log in with putty into the IPA server. Do you already have an idea or shall I start gdb? bye, Sumit > > Simo. > > -- > Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Wed Nov 23 13:34:21 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 23 Nov 2011 14:34:21 +0100 Subject: [Freeipa-devel] [PATCH] 172 Rework dnsrecord processing Message-ID: <1322055261.27504.9.camel@balmora.brq.redhat.com> Current DNS record processing architecture have many flaws, including custom execute() methods which does not take advantage of base LDAP commands or nonstandard and confusing DNS record option processing. This patch reworks DNS record processing with the following improvements: * Every DNS record has now own Parameter type. Each DNS record consists from one or more "parts" which are also Parameters. This architecture will enable much easier implementation of future per-DNS-type API. * Validation is now not written as a separate function for every parameter but is delegated to DNS record parts. * Normalization is also delegated to DNS record parts. * Since standard LDAP base commands execute method is now used, dnsrecord-add and dnsrecord-mod correctly supports --setattr and --addattr options. * In order to prevent confusion unsupported DNS record types are now hidden. They are still present in the plugin so that old clients receive proper validation error. The patch also contains several fixes: * Fix domain-name validation and normalization- allow domain names that are not fully qualified. For example --cname-rec=bar is a valid domain-name for bind which will translate it then as bar.. This change implies, that fully qualified domain names must end with '.'. * Do not let user accidentally remove entire zone with command "ipa dnsrecord-del @ --del-all". * Fix --ttl and --class option processing in dnsrecord-add and dnsrecord-mod. These changes are compatible with old clients, they are still able to add/modify DNS records. https://fedorahosted.org/freeipa/ticket/2082 ------------ This is a major dns plugin rework, please ping me in case of any question. New per-type DNS record API will be built on this patch - and thanks to the new nested DNS record parameter types should not be long. When Honza's patch 55 is in, proper CSV parsing for new DNS record types should work too. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-172-rework-dnsrecord-processing.patch Type: text/x-patch Size: 93116 bytes Desc: not available URL: From mkosek at redhat.com Wed Nov 23 15:17:41 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 23 Nov 2011 16:17:41 +0100 Subject: [Freeipa-devel] [PATCH] 166+167 Zonemgr improvements In-Reply-To: <1321263429.4805.7.camel@balmora.brq.redhat.com> References: <1321021336.4215.15.camel@balmora.brq.redhat.com> <4EBD3DA0.6050908@redhat.com> <1321263429.4805.7.camel@balmora.brq.redhat.com> Message-ID: <1322061461.27504.20.camel@balmora.brq.redhat.com> On Mon, 2011-11-14 at 10:37 +0100, Martin Kosek wrote: > On Fri, 2011-11-11 at 10:22 -0500, Rob Crittenden wrote: > > Martin Kosek wrote: > > > There are 2 patches improving our zone zonemgr: > > > > > > 1) ipa-server-install --zonemgr option validation and normalization + > > > the same thing in ipa dnszone-add/mod --admin-email. They now allow and > > > correctly process '.' in a local-part of the zonemgr e-mail (it is > > > encoded as '\.'. > > > > > > How to test: > > > > > > ipa-server-install -p secret123 -a secret123 --setup-dns --zonemgr=foo.bar at example.com > > > OR if e-mail is passed in SOA format: > > > ipa-server-install -p secret123 -a secret123 --setup-dns --zonemgr='foo\.bar.example.com' > > > > > > In both cases, the zonemgr e-mail will be set to correct format: 'foo > > > \.bar.example.com'. > > > > > > 2) Our default zonemgr is changed to follow RFC 2142 recommendation - > > > hostmaster@ > > > > > > hostmaster is an alias to root anyway (see /etc/aliases). > > > > > > Martin > > > > NACK 166, this illegal address is allowed: foo\.bar\.baz\.com > > > > ACK 167 when 166 is ready. > > > > rob > > Are you sure that you quoted the string in shell properly? It likes to > eat backslashes when one is not cautious. The zonemgr value, including > backslashes, should then show up in `ipa dnszone-show ZONE'. > > This is my output: > > # ipa dnszone-mod example.com --admin-email='foo\.bar\.baz\.com' > ipa: ERROR: invalid 'admin_email': address domain is not fully qualified > ("example.com" instead of just "example") > > Anyway, attaching a rebased patch (it collided with my patch 120). > > Martin I rebased both patches. I also fixed a bug when ipa-replica-prepare $HOST --ip-address=$IP was failing because None was passed as zonemgr. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-166-3-improve-zonemgr-validator-and-normalizer.patch Type: text/x-patch Size: 6949 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-167-2-default-dns-zone-manager.patch Type: text/x-patch Size: 2125 bytes Desc: not available URL: From mkosek at redhat.com Wed Nov 23 15:46:23 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 23 Nov 2011 16:46:23 +0100 Subject: [Freeipa-devel] [PATCH] 55 Parse comma-separated lists of values in all parameter types In-Reply-To: <4ECCC769.7070303@redhat.com> References: <4EA00403.9060703@redhat.com> <4EA583AB.6030906@redhat.com> <4EA5877B.2020606@redhat.com> <4EA59620.30506@redhat.com> <4EB45837.2020606@redhat.com> <4EB7B023.1030004@redhat.com> <4EBD7279.8030106@redhat.com> <4ECA7807.4080606@redhat.com> <1322038359.27504.4.camel@balmora.brq.redhat.com> <4ECCC769.7070303@redhat.com> Message-ID: <1322063183.27504.26.camel@balmora.brq.redhat.com> On Wed, 2011-11-23 at 11:14 +0100, Jan Cholasta wrote: > Dne 23.11.2011 09:52, Martin Kosek napsal(a): ... > > Well, the crash is obviously not related to the patch. Fedora 16 uses a > newer version of pylint (0.24), which causes the crash (it works fine on > Fedora 15 with pylint 0.22). > > I have opened a ticket to resolve that: > https://fedorahosted.org/freeipa/ticket/2136 The crash _is_ related to the patch since the combination of pylint and the changed source code causes it :-) But of course, it does not mean that the patch is wrong. If possible, I would like to push both your acked patch 55 and a patch fixing 2136 at one time. I don't want to confuse F-16 users with this crash when they try to compile our sources. Therefore, I moved 2136 to November milestone. Martin From sbose at redhat.com Wed Nov 23 15:48:02 2011 From: sbose at redhat.com (Sumit Bose) Date: Wed, 23 Nov 2011 16:48:02 +0100 Subject: [Freeipa-devel] [PATCH] Add ipasam samba passdb backend Message-ID: <20111123154802.GI2205@localhost.localdomain> Hi, this set of patches basically adds a samba passwd backend for IPA with can be build in the freeipa tree, plus the needed new objectclasses and attributes and enables the CLDAP service from Simo which is already committed. I compressed "Add-ipasam-samba-passdb-backend" to save some bandwidth. The backend is based on the old IPA passdb backend form the samba tree and various modified parts from the samba LDAP backend to make it work. As the result there are parts of the code which are not very pretty, but will work as planned. I will start refactoring the code together with fixing the first Coverity findings. bye, Sumit -------------- next part -------------- From dac0ae3118475cabc3626a364474e430014f4749 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Mon, 7 Nov 2011 11:56:57 +0100 Subject: [PATCH 1/6] Move our own domain info into cn=etc https://fedorahosted.org/freeipa/ticket/2001 --- ipaserver/install/adtrustinstance.py | 26 +++++++++++++++++--------- 1 files changed, 17 insertions(+), 9 deletions(-) diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index ee50a43061e76bd9e8c6744bc66b13ce10802521..1216f6bd8cf44cb54eb152d69c5001c10628fb92 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -183,17 +183,24 @@ class ADTRUSTInstance(service.Service): except errors.NotFound: pass - try: - self.admin_conn.getEntry(self.trust_dn, ldap.SCOPE_BASE) - except errors.NotFound: - entry = ipaldap.Entry(self.trust_dn) - entry.setValues("objectclass", ["nsContainer"]) - entry.setValues("cn", "trusts") - self.admin_conn.add_s(entry) + for new_dn in (self.trust_dn, \ + "cn=ad,"+self.trust_dn, \ + "cn=ad,cn=etc,"+self.suffix): + try: + self.admin_conn.getEntry(dn, ldap.SCOPE_BASE) + except errors.NotFound: + entry = ipaldap.Entry(dn) + entry.setValues("objectclass", ["nsContainer"]) + name = dn.split('=')[1].split(',')[0] + if not name: + print "Cannot extract RDN attribute value from [%s]" % dn + return + entry.setValues("cn", name) + self.admin_conn.add_s(entry) entry = ipaldap.Entry(self.smb_dom_dn) entry.setValues("objectclass", ["sambaDomain", "nsContainer"]) - entry.setValues("cn", "ad") + entry.setValues("cn", self.domain_name) entry.setValues("sambaDomainName", self.netbios_name) entry.setValues("sambaSID", self.__gen_sid_string()) #TODO: which MAY attributes do we want to set ? @@ -346,7 +353,8 @@ class ADTRUSTInstance(service.Service): self.smb_dn_pwd = ipautil.ipa_generate_password() self.trust_dn = "cn=trusts,%s" % self.suffix - self.smb_dom_dn = "cn=ad,%s" % self.trust_dn + self.smb_dom_dn = "cn=%s,cn=ad,cn=etc,%s" % (self.domain_name, \ + self.suffix) self.__setup_sub_dict() -- 1.7.6 -------------- next part -------------- From be3cea4a6bf16d1a623694fe5315821eefd1c94b Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Wed, 9 Nov 2011 16:38:10 +0100 Subject: [PATCH 2/6] Add trust objectclass and attributes to v3 schema --- install/share/60basev3.ldif | 11 +++++++++++ 1 files changed, 11 insertions(+), 0 deletions(-) diff --git a/install/share/60basev3.ldif b/install/share/60basev3.ldif index f518541586b2df9ed08718098a7f170563aa4e1d..6db644addf298216e2b85dc68b616e8351457cf5 100644 --- a/install/share/60basev3.ldif +++ b/install/share/60basev3.ldif @@ -14,7 +14,18 @@ attributeTypes: (2.16.840.1.113730.3.8.11.7 NAME 'ipaNTProfilePath' DESC 'User P attributeTypes: (2.16.840.1.113730.3.8.11.8 NAME 'ipaNTHomeDirectory' DESC 'User Home Directory Path' EQUALITY caseIgnoreMatch OREDRING caseIgnoreOrderingMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'IPA v3' ) attributeTypes: (2.16.840.1.113730.3.8.11.9 NAME 'ipaNTHomeDirectoryDrive' DESC 'User Home Drive Letter' EQUALITY caseIgnoreMatch OREDRING caseIgnoreOrderingMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'IPA v3' ) attributeTypes: (2.16.840.1.113730.3.8.11.10 NAME 'ipaNTDomainGUID' DESC 'NT Domain GUID' EQUALITY caseIgnoreIA5Match OREDRING caseIgnoreIA5OrderingMatch SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VALUE X-ORIGIN 'IPA v3' ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.11 NAME 'ipaNTTrustType' DESC 'Type of trust' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.12 NAME 'ipaNTTrustAttributes' DESC 'Trust attributes for a trusted domain' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.13 NAME 'ipaNTTrustDirection' DESC 'Direction of a trust' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.14 NAME 'ipaNTTrustPartner' DESC 'Fully qualified name of the domain with which a trust exists' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{128} ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.15 NAME 'ipaNTTrustAuthOutgoing' DESC 'Authentication information for the outgoing portion of a trust' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.16 NAME 'ipaNTTrustAuthIncoming' DESC 'Authentication information for the incoming portion of a trust' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.17 NAME 'ipaNTTrustForestTrustInfo' DESC 'Forest trust information for a trusted domain object' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.18 NAME 'ipaNTTrustPosixOffset' DESC 'POSIX offset of a trust' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.19 NAME 'ipaNTSupportedEncryptionTypes' DESC 'Supported encryption types of a trust' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) objectClasses: (2.16.840.1.113730.3.8.12.1 NAME 'ipaExternalGroup' SUP top STRUCTURAL MUST ( cn ) MAY ( ipaExternalMember $ memberOf $ description $ owner) X-ORIGIN 'IPA v3' ) objectClasses: (2.16.840.1.113730.3.8.12.2 NAME 'ipaNTUserAttrs' SUP top AUXILIARY MUST ( ipaNTSecurityIdentifier ) MAY ( ipaNTHash $ ipaNTLogonScript $ ipaNTProfilePath $ ipaNTHomeDirectory $ ipaNTHomeDirectoryDrive ) X-ORIGIN 'IPA v3' ) objectClasses: (2.16.840.1.113730.3.8.12.3 NAME 'ipaNTGroupAttrs' SUP top AUXILIARY MUST ( ipaNTSecurityIdentifier ) X-ORIGIN 'IPA v3' ) objectClasses: (2.16.840.1.113730.3.8.12.4 NAME 'ipaNTDomainAttrs' SUP top AUXILIARY MUST ( ipaNTSecurityIdentifier $ ipaNTFlatName $ ipaNTDomainGUID ) MAY ( ipaNTFallbackPrimaryGroup ) X-ORIGIN 'IPA v3' ) +objectClasses: (2.16.840.1.113730.3.8.12.5 NAME 'ipaNTTrustedDomain' SUP top STRUCTURAL DESC 'Trusted Domain Object' MUST ( cn ) MAY ( ipaNTTrustType $ ipaNTTrustAttributes $ ipaNTTrustDirection $ ipaNTTrustPartner $ ipaNTFlatName $ ipaNTTrustAuthOutgoing $ ipaNTTrustAuthIncoming $ ipaNTSecurityIdentifier $ ipaNTTrustForestTrustInfo $ ipaNTTrustPosixOffset $ ipaNTSupportedEncryptionTypes) ) + -- 1.7.6 -------------- next part -------------- From 6af8a4054e5c1e0c06897807a5c8420ed8870c90 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Mon, 7 Nov 2011 12:48:10 +0100 Subject: [PATCH 3/6] Use new objectclasses and attributes for trust --- ipaserver/install/adtrustinstance.py | 46 +++++++++++++++++++++++---------- 1 files changed, 32 insertions(+), 14 deletions(-) diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 1216f6bd8cf44cb54eb152d69c5001c10628fb92..78b319fb8f4d793879a41395558d14f72c225647 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -22,7 +22,7 @@ import errno import ldap import service import tempfile -import installutils +import uuid from ipaserver import ipaldap from ipaserver.install.dsinstance import realm_to_serverid from ipaserver.install.bindinstance import get_rr, add_rr, del_rr, \ @@ -75,6 +75,14 @@ def make_netbios_name(s): return ''.join([c for c in s.split('.')[0].upper() if c in allowed_netbios_chars])[:15] class ADTRUSTInstance(service.Service): + + ATTR_SID = "ipaNTSecurityIdentifier" + ATTR_FLAT_NAME = "ipaNTFlatName" + ATTR_GUID = "ipaNTDomainGUID" + OBJC_USER = "ipaNTUserAttrs" + OBJC_GROUP = "ipaNTGroupAttrs" + OBJC_DOMAIN = "ipaNTDomainAttrs" + def __init__(self, fstore=None, dm_password=None): service.Service.__init__(self, "smb", dm_password=dm_password) @@ -107,13 +115,22 @@ class ADTRUSTInstance(service.Service): # Also the premission to create trusted domain objects below the # domain object is granted. mod = [(ldap.MOD_ADD, 'aci', - str('(targetattr = "sambaNTPassword")' \ + str('(targetattr = "ipaNTHash")' \ '(version 3.0; acl "Samba user can read NT passwords";' \ 'allow (read) userdn="ldap:///%s";)' % self.smb_dn)), (ldap.MOD_ADD, 'aci', str('(target = "ldap:///cn=ad,cn=trusts,%s")' \ - '(targetattr = "sambaTrustType || sambaTrustAttributes || sambaTrustDirection || sambaTrustPartner || sambaFlatName || sambaTrustAuthOutgoing || sambaTrustAuthIncoming || sambaSecurityIdentifier || sambaTrustForestTrustInfo || sambaTrustPosixOffset || sambaSupportedEncryptionTypes")' \ - '(version 3.0;acl "Allow samba user to create and delete trust accounts";' \ + '(targetattr = "ipaNTTrustType || ipaNTTrustAttributes || ' \ + 'ipaNTTrustDirection || ' \ + 'ipaNTTrustPartner || ipaNTFlatName || ' \ + 'ipaNTTrustAuthOutgoing || ' \ + 'ipaNTTrustAuthIncoming || ' \ + 'ipaNTSecurityIdentifier || ' \ + 'ipaNTTrustForestTrustInfo || ' \ + 'ipaNTTrustPosixOffset || ' \ + 'ipaNTSupportedEncryptionTypes")' \ + '(version 3.0;acl "Allow samba user to create and delete ' \ + 'trust accounts";' \ 'allow (write,add,delete) userdn = "ldap:///%s";)' % \ (self.suffix, self.smb_dn)))] @@ -137,7 +154,7 @@ class ADTRUSTInstance(service.Service): print "Samba domain object not found" return - dom_sid = dom_entry.getValue("sambaSID") + dom_sid = dom_entry.getValue(self.ATTR_SID) if not dom_sid: print "Samba domain object does not have a SID" return @@ -155,22 +172,22 @@ class ADTRUSTInstance(service.Service): print "IPA admin group object not found" return - if admin_entry.getValue("sambaSID") or \ - admin_group_entry.getValue("sambaSID"): + if admin_entry.getValue(self.ATTR_SID) or \ + admin_group_entry.getValue(self.ATTR_SID): print "Admin SID already set, nothing to do" return try: self.admin_conn.modify_s(admin_dn, \ - [(ldap.MOD_ADD, "objectclass", "sambaSamAccount"), \ - (ldap.MOD_ADD, "sambaSID", dom_sid + "-500")]) + [(ldap.MOD_ADD, "objectclass", self.OBJC_USER), \ + (ldap.MOD_ADD, self.ATTR_SID, dom_sid + "-500")]) except: print "Failed to modify IPA admin object" try: self.admin_conn.modify_s(admin_group_dn, \ - [(ldap.MOD_ADD, "objectclass", "sambaSidEntry"), \ - (ldap.MOD_ADD, "sambaSID", dom_sid + "-512")]) + [(ldap.MOD_ADD, "objectclass", self.OBJC_GROUP), \ + (ldap.MOD_ADD, self.ATTR_SID, dom_sid + "-512")]) except: print "Failed to modify IPA admin group object" @@ -199,10 +216,11 @@ class ADTRUSTInstance(service.Service): self.admin_conn.add_s(entry) entry = ipaldap.Entry(self.smb_dom_dn) - entry.setValues("objectclass", ["sambaDomain", "nsContainer"]) + entry.setValues("objectclass", [self.OBJC_DOMAIN, "nsContainer"]) entry.setValues("cn", self.domain_name) - entry.setValues("sambaDomainName", self.netbios_name) - entry.setValues("sambaSID", self.__gen_sid_string()) + entry.setValues(self.ATTR_FLAT_NAME, self.netbios_name) + entry.setValues(self.ATTR_SID, self.__gen_sid_string()) + entry.setValues(self.ATTR_GUID, str(uuid.uuid4())) #TODO: which MAY attributes do we want to set ? self.admin_conn.add_s(entry) -- 1.7.6 -------------- next part -------------- From ec8f6e42db0ddf172aea48a364278ff2cf277458 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Mon, 7 Nov 2011 12:59:20 +0100 Subject: [PATCH 4/6] Fix some pylint warnings --- install/tools/ipa-adtrust-install | 2 +- ipaserver/install/adtrustinstance.py | 103 +++++++++++++++++++++------------ 2 files changed, 66 insertions(+), 39 deletions(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index 87fecbfb4834d65fdccc3f8536a5665ba75e48a5..c6fd3478a28697301cac317dff1bbf25c6d865ce 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -111,7 +111,7 @@ def main(): print "" # Check if samba packages are installed - if not adtrustinstance.check_inst(options.unattended): + if not adtrustinstance.check_inst(): sys.exit("Aborting installation.") # Initialize the ipalib api diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 78b319fb8f4d793879a41395558d14f72c225647..390ab5efda679ca039189722e46d32b90cd99b6d 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -20,10 +20,11 @@ import os import errno import ldap -import service import tempfile import uuid from ipaserver import ipaldap +from ipaserver.install import installutils +from ipaserver.install import service from ipaserver.install.dsinstance import realm_to_serverid from ipaserver.install.bindinstance import get_rr, add_rr, del_rr, \ dns_zone_exists @@ -32,17 +33,17 @@ from ipapython import sysrestore from ipapython import ipautil from ipapython.ipa_log_manager import * -import random import string import struct -allowed_netbios_chars = string.ascii_uppercase + string.digits +ALLOWED_NETBIOS_CHARS = string.ascii_uppercase + string.digits -def check_inst(unattended): - for f in ['/usr/sbin/smbd', '/usr/bin/net', '/usr/bin/smbpasswd']: - if not os.path.exists(f): - print "%s was not found on this system" % f - print "Please install the 'samba' packages and start the installation again" +def check_inst(): + for smbfile in ['/usr/sbin/smbd', '/usr/bin/net', '/usr/bin/smbpasswd']: + if not os.path.exists(smbfile): + print "%s was not found on this system" % file + print "Please install the 'samba' packages and " \ + "start the installation again" return False #TODO: Add check for needed samba4 libraries @@ -51,13 +52,13 @@ def check_inst(unattended): def ipa_smb_conf_exists(): try: - fd = open('/etc/samba/smb.conf', 'r') - except IOError, e: - if e.errno == errno.ENOENT: + conf_fd = open('/etc/samba/smb.conf', 'r') + except IOError, err: + if err.errno == errno.ENOENT: return False - lines = fd.readlines() - fd.close() + lines = conf_fd.readlines() + conf_fd.close() for line in lines: if line.startswith('### Added by IPA Installer ###'): return True @@ -66,13 +67,15 @@ def ipa_smb_conf_exists(): def check_netbios_name(s): # NetBIOS names may not be longer than 15 allowed characters - if not s or len(s) > 15 or ''.join([c for c in s if c not in allowed_netbios_chars]): + if not s or len(s) > 15 or \ + ''.join([c for c in s if c not in ALLOWED_NETBIOS_CHARS]): return False return True def make_netbios_name(s): - return ''.join([c for c in s.split('.')[0].upper() if c in allowed_netbios_chars])[:15] + return ''.join([c for c in s.split('.')[0].upper() \ + if c in ALLOWED_NETBIOS_CHARS])[:15] class ADTRUSTInstance(service.Service): @@ -84,6 +87,22 @@ class ADTRUSTInstance(service.Service): OBJC_DOMAIN = "ipaNTDomainAttrs" def __init__(self, fstore=None, dm_password=None): + self.fqdn = None + self.ip_address = None + self.realm_name = None + self.domain_name = None + self.netbios_name = None + self.no_msdcs = None + self.smbd_user = None + self.suffix = None + self.ldapi_socket = None + self.smb_conf = None + self.smb_dn = None + self.smb_dn_pwd = None + self.trust_dn = None + self.smb_dom_dn = None + self.sub_dict = None + service.Service.__init__(self, "smb", dm_password=dm_password) if fstore: @@ -97,7 +116,8 @@ class ADTRUSTInstance(service.Service): self.admin_conn.getEntry(self.smb_dn, ldap.SCOPE_BASE) print "Samba user entry exists, resetting password" - self.admin_conn.modify_s(self.smb_dn, [(ldap.MOD_REPLACE, "userPassword", self.smb_dn_pwd)]) + self.admin_conn.modify_s(self.smb_dn, \ + [(ldap.MOD_REPLACE, "userPassword", self.smb_dn_pwd)]) return except errors.NotFound: @@ -204,13 +224,14 @@ class ADTRUSTInstance(service.Service): "cn=ad,"+self.trust_dn, \ "cn=ad,cn=etc,"+self.suffix): try: - self.admin_conn.getEntry(dn, ldap.SCOPE_BASE) + self.admin_conn.getEntry(new_dn, ldap.SCOPE_BASE) except errors.NotFound: - entry = ipaldap.Entry(dn) + entry = ipaldap.Entry(new_dn) entry.setValues("objectclass", ["nsContainer"]) - name = dn.split('=')[1].split(',')[0] + name = new_dn.split('=')[1].split(',')[0] if not name: - print "Cannot extract RDN attribute value from [%s]" % dn + print "Cannot extract RDN attribute value from [%s]" % \ + new_dn return entry.setValues("cn", name) self.admin_conn.add_s(entry) @@ -227,18 +248,18 @@ class ADTRUSTInstance(service.Service): def __write_smb_conf(self): self.fstore.backup_file(self.smb_conf) - fd = open(self.smb_conf, "w") - fd.write('### Added by IPA Installer ###\n') - fd.write('[global]\n') - fd.write('config backend = registry\n') - fd.close() + conf_fd = open(self.smb_conf, "w") + conf_fd.write('### Added by IPA Installer ###\n') + conf_fd.write('[global]\n') + conf_fd.write('config backend = registry\n') + conf_fd.close() def __write_smb_registry(self): template = os.path.join(ipautil.SHARE_DIR, "smb.conf.template") conf = ipautil.template_file(template, self.sub_dict) - [fd, tmp_name] = tempfile.mkstemp() - os.write(fd, conf) - os.close(fd) + [tmp_fd, tmp_name] = tempfile.mkstemp() + os.write(tmp_fd, conf) + os.close(tmp_fd) args = ["/usr/bin/net", "conf", "import", tmp_name] @@ -250,7 +271,8 @@ class ADTRUSTInstance(service.Service): def __set_smb_ldap_password(self): args = ["/usr/bin/smbpasswd", "-c", self.smb_conf, "-s", "-W" ] - ipautil.run(args, stdin = self.smb_dn_pwd + "\n" + self.smb_dn_pwd + "\n" ) + ipautil.run(args, stdin = self.smb_dn_pwd + "\n" + \ + self.smb_dn_pwd + "\n" ) def __setup_principal(self): cifs_principal = "cifs/" + self.fqdn + "@" + self.realm_name @@ -291,7 +313,7 @@ class ADTRUSTInstance(service.Service): ".dc._msdcs") err_msg = None - ret = api.Command.dns_is_enabled() + ret = api.Command['dns_is_enabled']() if not ret['result']: err_msg = "DNS management was not enabled at install time." else: @@ -341,7 +363,8 @@ class ADTRUSTInstance(service.Service): # Instead we reply on the IPA init script to start only enabled # components as found in our LDAP configuration tree try: - self.ldap_enable('ADTRUST', self.fqdn, self.dm_password, self.suffix) + self.ldap_enable('ADTRUST', self.fqdn, self.dm_password, \ + self.suffix) except ldap.ALREADY_EXISTS: root_logger.critical("ADTRUST Service startup entry already exists.") pass @@ -355,7 +378,7 @@ class ADTRUSTInstance(service.Service): def setup(self, fqdn, ip_address, realm_name, domain_name, netbios_name, no_msdcs=False, smbd_user="samba"): - self.fqdn =fqdn + self.fqdn = fqdn self.ip_address = ip_address self.realm_name = realm_name self.domain_name = domain_name @@ -363,7 +386,8 @@ class ADTRUSTInstance(service.Service): self.no_msdcs = no_msdcs self.smbd_user = smbd_user self.suffix = ipautil.realm_to_suffix(self.realm_name) - self.ldapi_socket = "%%2fvar%%2frun%%2fslapd-%s.socket" % realm_to_serverid(self.realm_name) + self.ldapi_socket = "%%2fvar%%2frun%%2fslapd-%s.socket" % \ + realm_to_serverid(self.realm_name) self.smb_conf = "/etc/samba/smb.conf" @@ -383,15 +407,18 @@ class ADTRUSTInstance(service.Service): self.step("stopping smbd", self.__stop) self.step("create samba user", self.__create_samba_user) - self.step("create samba domain object", self.__create_samba_domain_object) + self.step("create samba domain object", \ + self.__create_samba_domain_object) self.step("create samba config registry", self.__write_smb_registry) self.step("writing samba config file", self.__write_smb_conf) - self.step("setting password for the samba user", self.__set_smb_ldap_password) + self.step("setting password for the samba user", \ + self.__set_smb_ldap_password) self.step("Adding cifs Kerberos principal", self.__setup_principal) self.step("Adding admin(group) SIDs", self.__add_admin_sids) self.step("configuring smbd to start on boot", self.__enable) if not self.no_msdcs: - self.step("adding special DNS service records", self.__add_dns_service_records) + self.step("adding special DNS service records", \ + self.__add_dns_service_records) self.step("starting smbd", self.__start) self.start_creation("Configuring smbd:") @@ -408,9 +435,9 @@ class ADTRUSTInstance(service.Service): except: pass - for f in [self.smb_conf]: + for r_file in [self.smb_conf]: try: - self.fstore.restore_file(f) + self.fstore.restore_file(r_file) except ValueError, error: root_logger.debug(error) pass -- 1.7.6 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-sbose-0013-Add-ipasam-samba-passdb-backend.patch.bz2 Type: application/x-bzip2 Size: 16879 bytes Desc: not available URL: -------------- next part -------------- From 3cd9e920377b4f37aaedde08f92346e0a6623337 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Fri, 18 Nov 2011 14:04:09 +0100 Subject: [PATCH 6/6] activate CLDAP --- install/tools/ipa-adtrust-install | 3 +-- ipaserver/install/adtrustinstance.py | 4 ++++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index c6fd3478a28697301cac317dff1bbf25c6d865ce..248ea35eaa86dd59ebbc871b86df780cfd71ccf6 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -214,6 +214,7 @@ def main(): print "\t\tUDP Ports:" print "\t\t * 138: netbios-dgm" print "\t\t * 139: netbios-ssn" + print "\t\t * 389: (C)LDAP" print "\t\t * 445: microsoft-ds" print "" print "\tAdditionally you have to make sure the FreeIPA LDAP server cannot reached" @@ -221,8 +222,6 @@ def main(): print "\tfollowing ports for these servers:" print "\t\tTCP Ports:" print "\t\t * 389, 636: LDAP/LDAPS" - print "\t\tUDP Ports:" - print "\t\t * 389: (C)LDAP" print "\tYou may want to choose to REJECT the network packets instead of DROPing them" print "\tto avoid timeouts on the AD domain controllers." diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 390ab5efda679ca039189722e46d32b90cd99b6d..86599956511a2a1e7695582247d374c9946f743d 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -254,6 +254,9 @@ class ADTRUSTInstance(service.Service): conf_fd.write('config backend = registry\n') conf_fd.close() + def __add_cldap_module(self): + self._ldap_mod("ipa-cldap-conf.ldif", self.sub_dict) + def __write_smb_registry(self): template = os.path.join(ipautil.SHARE_DIR, "smb.conf.template") conf = ipautil.template_file(template, self.sub_dict) @@ -415,6 +418,7 @@ class ADTRUSTInstance(service.Service): self.__set_smb_ldap_password) self.step("Adding cifs Kerberos principal", self.__setup_principal) self.step("Adding admin(group) SIDs", self.__add_admin_sids) + self.step("Activation CLDAP plugin", self.__add_cldap_module) self.step("configuring smbd to start on boot", self.__enable) if not self.no_msdcs: self.step("adding special DNS service records", \ -- 1.7.6 From mkosek at redhat.com Wed Nov 23 16:09:44 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 23 Nov 2011 17:09:44 +0100 Subject: [Freeipa-devel] [PATCH] 157 Add --delattr option to complement --setattr/--addattr In-Reply-To: <1320497005.25589.5.camel@priserak> References: <1319714558.26374.13.camel@balmora.brq.redhat.com> <4EB3E74A.1000905@redhat.com> <1320497005.25589.5.camel@priserak> Message-ID: <1322064584.27504.28.camel@balmora.brq.redhat.com> On Sat, 2011-11-05 at 13:43 +0100, Martin Kosek wrote: > On Fri, 2011-11-04 at 09:23 -0400, Rob Crittenden wrote: > > Martin Kosek wrote: > > > Add a --delattr option to round out multi-valued attribute manipulation. > > > The new option is be available for all LDAPUpdate based commands. > > > > > > --delattr is evaluated last, it can remove any value present either > > > in --addattr/--setattr options or stored in LDAP. > > > > > > https://fedorahosted.org/freeipa/ticket/1929 > > > > Should --delattr raise an error if the value doesn't exist? > > > > I think it probably should. > > > > rob > > You are right, it would be more expected behavior. I fixed that. In the > process of implementing the change I found that current --*attr > processing is not good, so I refactored it completely to one function > available for all baseldap.py commands. > > In the process I found out that we don't have any common class for all > baseldap.py commands and the result is BaseLDAPCommand which can now > handle attribute processing and provide it to other LDAP commands. > > And I also fixed one group unit test. Now all unit tests were OK. > > Martin I rebased the patch (API.txt format was changed) and tested that it still works ok. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-157-3-add-delattr-option-to-complement-setattr-addattr.patch Type: text/x-patch Size: 67927 bytes Desc: not available URL: From abokovoy at redhat.com Wed Nov 23 20:19:40 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 23 Nov 2011 22:19:40 +0200 Subject: [Freeipa-devel] Warning: use of ipa-devel repo would prevent upgrades from F15 to F16 Message-ID: <20111123201940.GA14772@redhat.com> Hi, this is mostly actual for developers but I'm including freeipa-users@ just in case. If you are not using ipa-devel, you can ignore this message. Development repository for FreeIPA-related packages, ipa-devel, does not have pki-ca package built for Fedora 16. This means that after upgrade from Fedora 15 to Fedora 16, a pki-ca package installed from ipa-devel repo will not be updated to Fedora 16-specific pki-ca package from fedora-updates: # rpm -q pki-ca pki-ca-9.0.16-1.20111122T0545z.fc15.noarch # yum install pki-ca-9.0.16-1.fc16 Setting up Install Process Package matching pki-ca-9.0.16-1.fc16.noarch already installed. Checking for update. Nothing to do As result, pki-ca will not have systemd support and FreeIPA in Fedora 16 will not work with integrated Certificate Authority. I've filed ticket 61 for pki: https://fedorahosted.org/pki/ticket/61, but person responsible for the repository is not available until Monday so if you are planning to upgrade from Fedora 15 to Fedora 16 and are using ipa-devel repo, please postpone the upgrade until next week. Additionaly, please be aware that ipa-devel repo is for development and early testing purposes only. No guarantees are given out for packages installed from ipa-devel. Of course, if you are participating in FreeIPA/Dogtag/389-ds development, you already know that. -- / Alexander Bokovoy From rcritten at redhat.com Wed Nov 23 22:01:51 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 23 Nov 2011 17:01:51 -0500 Subject: [Freeipa-devel] [PATCH] 906 Add SELinux user mapping framework. Message-ID: <4ECD6D4F.600@redhat.com> This will allow one to define what SELinux context a given user gets on a given machine. A rule can contain a set of users and hosts or it can point to an existing HBAC rule that defines them. https://fedorahosted.org/freeipa/ticket/755 rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-906-selinux.patch Type: text/x-patch Size: 68634 bytes Desc: not available URL: From rcritten at redhat.com Wed Nov 23 22:05:43 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 23 Nov 2011 17:05:43 -0500 Subject: [Freeipa-devel] [PATCH] 905 add plugin framework to updater In-Reply-To: <20111121082138.GB13007@redhat.com> References: <4EC420C7.40607@redhat.com> <20111121082138.GB13007@redhat.com> Message-ID: <4ECD6E37.4090002@redhat.com> Alexander Bokovoy wrote: > On Wed, 16 Nov 2011, Rob Crittenden wrote: >> There are two reasons for the plugin framework: >> 1. To provide a way of doing manual/complex LDAP changes without having >> to keep extending ldapupdate.py (like we did with managed entries). >> 2. Allows for better control of restarts. >> >> There are two types of plugins, preop and postop. A preop plugin runs >> before any file-based updates are loaded. A postop plugin runs after >> all file-based updates are applied. >> >> A preop plugin may update LDAP directly or craft update entries to >> be applied with the file-based updates. >> >> Either a preop or postop plugin may attempt to restart the dirsrv >> instance. The instance is only restartable if ipa-ldap-updater is >> being executed as root. A warning is printed if a restart is >> requested for a non-root user. >> >> Plugins are not executed by default. This is so we can use >> ldapupdate to apply simple updates in commands like ipa-nis-manage. >> >> rob > >> > From e8c632c0a17c5fad3792d4f741976161d245fec6 Mon Sep 17 00:00:00 2001 >> From: Rob Crittenden >> Date: Wed, 16 Nov 2011 15:37:56 -0500 >> Subject: [PATCH] Add plugin framework to LDAP updates. >> >> There are two reasons for the plugin framework: >> 1. To provide a way of doing manual/complex LDAP changes without having >> to keep extending ldapupdate.py (like we did with managed entries). >> 2. Allows for better control of restarts. >> >> There are two types of plugins, preop and postop. A preop plugin runs >> before any file-based updates are loaded. A postop plugin runs after >> all file-based updates are applied. >> >> A preop plugin may update LDAP directly or craft update entries to be >> applied with the file-based updates. >> >> Either a preop or postop plugin may attempt to restart the dirsrv instance. >> The instance is only restartable if ipa-ldap-updater is being executed >> as root. A warning is printed if a restart is requested for a non-root >> user. >> >> Plugins are not executed by default. This is so we can use ldapupdate >> to apply simple updates in commands like ipa-nis-manage. > ACK though I have few considerations, perhaps asking for a > separate patch. > > This patch adds Updater namespace available in all contexts albeit > populated only in 'updates'/'installer' modes. I wonder if we could > re-factor the API class code so that until you have called finalize(), > you can add more namespaces. This would prevent need to modify basic > initialization every time new namespace is added -- especially given > that namespaces like Updater are specific to single utility. > > I know, technically you could simply replace api.register by a new > instance and dispose the old one before going to finalize()... > > Another thought is to allow API to accept list of pairs (context, > path) to simplify loading plugins. By default 'ipalib' plugins would > be loaded, and for others, API.add_plugins(context, path) would be > adding more, then API.load_plugins() would iterate over these pairs > and if context is the same as in API.env, load them. > > This way plugins will be truly generic and we'll gain ability to use > per application plugins without any additional changes in the core > framework. > My original intention was to have the updater in its own completely separate api, and that is how I initially coded it. I had to abandon it though due to wanting to re-use ldap2.py in ipaserver. We do lots of 'from ipalib import api' and this doesn't work if you have your own api. I considered replacing register as well but gave up on that avenue as well. Rebased, pushed to master. rob From rcritten at redhat.com Wed Nov 23 22:30:02 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 23 Nov 2011 17:30:02 -0500 Subject: [Freeipa-devel] [PATCH] fix issues with rebase Message-ID: <4ECD73EA.3060605@redhat.com> This will teach me to manually rebase some strings, I goofed them. I pushed this patch as a one-liner. rob From rcritten at redhat.com Wed Nov 23 22:33:42 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 23 Nov 2011 17:33:42 -0500 Subject: [Freeipa-devel] [PATCH] 8 Add DNS service records for Windows In-Reply-To: <20111121083211.GD13007@redhat.com> References: <20111014101557.GC4622@localhost.localdomain> <20111014151202.GD4622@localhost.localdomain> <20111014172150.GA17515@redhat.com> <20111014191648.GE4622@localhost.localdomain> <20111121083211.GD13007@redhat.com> Message-ID: <4ECD74C6.5080602@redhat.com> Alexander Bokovoy wrote: > Hi Sumit, > > On Fri, 14 Oct 2011, Sumit Bose wrote: >>> It would make more clear what is the default and that it is really >>> optional setting -- I'm thinking from the perspective of maintenance >>> of the code in future. >> >> Thank you for your comments, new version attached. > Finally got to test it. ACK. > pushed to master. From rcritten at redhat.com Wed Nov 23 22:39:38 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 23 Nov 2011 17:39:38 -0500 Subject: [Freeipa-devel] [PATCH] 162 Fix coverity issues in client CLI tools In-Reply-To: <1321265059.4805.14.camel@balmora.brq.redhat.com> References: <1320772029.32755.1.camel@balmora.brq.redhat.com> <4EBD6C76.9010600@redhat.com> <1321265059.4805.14.camel@balmora.brq.redhat.com> Message-ID: <4ECD762A.2000203@redhat.com> Martin Kosek wrote: > On Fri, 2011-11-11 at 13:41 -0500, Rob Crittenden wrote: >> Martin Kosek wrote: >>> This patch fixes 2 coverity issues: >>> * ipa-client/config.c: CID 11090: Resource leak >>> * ipa-client/ipa-getkeytab.c: CID 11018: Unchecked return value >>> >>> https://fedorahosted.org/freeipa/ticket/2035 >> >> You don't need to test a variable before you free it, so you can just >> call free(data). > > Right, I was over-defensive here. Fixed. > >> >> Since we are capturing the kerberos error should it be displayed as well >> in ipa-getkeytab? >> >> rob >> > > Good idea, it should. Unfortunately, this is not done properly across > all ipa-getkeytab.c. ipa-rmkeytab.c does this much better. > > I added error messages at least to the affected function to keep the > patch compact. > > Martin ack, pushed to master From abokovoy at redhat.com Wed Nov 23 23:00:48 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 24 Nov 2011 01:00:48 +0200 Subject: [Freeipa-devel] [PATCH] 906 Add SELinux user mapping framework. In-Reply-To: <4ECD6D4F.600@redhat.com> References: <4ECD6D4F.600@redhat.com> Message-ID: <20111123230047.GB14772@redhat.com> Hi Rob, On Wed, 23 Nov 2011, Rob Crittenden wrote: > This will allow one to define what SELinux context a given user gets > on a given machine. A rule can contain a set of users and hosts or it > can point to an existing HBAC rule that defines them. > > https://fedorahosted.org/freeipa/ticket/755 I read through the patch, will need to test it later this week. I basically have two minor points: 1. Split charachter in the SE Linux user map order. > + > + Define SELinux user map order: > + ipa config-mod --ipaselinuxusermaporder='guest_u:s0$xguest_u:s0$user_u:s0-s0:c0.c1023$staff_u:s0-s0:c0.c1023$unconfined_u:s0-s0:c0.c1023' > """) $ can be considered 'active' character in all shells in a sense it changes treatment of following characters from the shell perspective and therefore will always require shielding from the shell's influence. This increases likelyhood of error from a user side. Maybe / would be more neutral character? As you said on IRC, people might have religious feeling about separators but tricking users into always thinking about escaping/single quoting is equally bad. 2. We have two possible ways to address named properties in MagicDict and NameSpace objects -- through explicit attribute use and through the dictionary key. I guess for the cases when we know the attribute name in advance, it would perhaps be preferrable to use the former style: > + def pre_callback(self, ldap, dn, *keys, **options): > + kw = dict(seealso=dn) > + _entries = api.Command['selinuxusermap_find'](None, **kw) this would be _entries = api.Command.selinuxusermap_find(None, **kw) Other than those two minor points, the patch looks very good. I'm going to give it a run on Friday. -- / Alexander Bokovoy From sbose at redhat.com Thu Nov 24 12:54:19 2011 From: sbose at redhat.com (Sumit Bose) Date: Thu, 24 Nov 2011 13:54:19 +0100 Subject: [Freeipa-devel] [PATCH] #2122 Fix PAC re-signing In-Reply-To: <20111123105311.GG2205@localhost.localdomain> References: <1322007054.6750.76.camel@willson.li.ssimo.org> <20111123105311.GG2205@localhost.localdomain> Message-ID: <20111124125419.GK2205@localhost.localdomain> On Wed, Nov 23, 2011 at 11:53:11AM +0100, Sumit Bose wrote: > On Tue, Nov 22, 2011 at 07:10:54PM -0500, Simo Sorce wrote: > > In some cases the KDC will decide to use a different checksum type when > > re-signing a PAC to include it in a service ticket. > > > > This is common in a cross-realm trust with AD as most AD DCs will use a > > HMAC-MD5-RC4 checksum while IPA's KDC will instead choose to use > > HMAC-SHA-AES when re-signing the PAC. > > > > In current MIT code re-signing a PAC with a signature that differs in > > length from the original will cause an error. > > > > While MIT should handle this properly, we use the workaround of > > regenerating the PAC from scratch so that there is no trace of the > > previous signatures. > > > > Tested while obtaining a cross-realm ticket from an AD domain against a > > service belonging to an IPA domain. > > I see "authdata (kdb) handling failure: Cannot allocate memory" in > krb5kdc.log when trying to log in with putty into the IPA server. Do you > already have an idea or shall I start gdb? I think I found two issues which should be fixed by the following patch: - krb5_pac_add_buffer() expects krb5_pac and not krb5_pac * as a second argument - your patch copies all buffers, including the checksums, which you wanted to remove from the new pac With this patch applied I do not see any errors in the krb5kdc.log and ssh from AD to IPA server works. HTH bye, Sumit diff --git a/daemons/ipa-kdb/ipa_kdb_mspac.c b/daemons/ipa-kdb/ipa_kdb_mspac.c index 63c18b5..68a347a 100644 --- a/daemons/ipa-kdb/ipa_kdb_mspac.c +++ b/daemons/ipa-kdb/ipa_kdb_mspac.c @@ -28,6 +28,10 @@ #define KRB5INT_PAC_SIGN_AVAILABLE 1 #define KRB5INT_FIND_AUTHDATA_AVAILABLE 1 +#define PAC_SERVER_CHECKSUM 6 /**< Server checksum */ diff --git a/daemons/ipa-kdb/ipa_kdb_mspac.c b/daemons/ipa-kdb/ipa_kdb_mspac.c index 63c18b5..68a347a 100644 --- a/daemons/ipa-kdb/ipa_kdb_mspac.c +++ b/daemons/ipa-kdb/ipa_kdb_mspac.c @@ -28,6 +28,10 @@ #define KRB5INT_PAC_SIGN_AVAILABLE 1 #define KRB5INT_FIND_AUTHDATA_AVAILABLE 1 +#define PAC_SERVER_CHECKSUM 6 /**< Server checksum */ +#define PAC_PRIVSVR_CHECKSUM 7 /**< KDC checksum */ + + #if KRB5INT_PAC_SIGN_AVAILABLE krb5_error_code krb5int_pac_sign(krb5_context context, @@ -604,16 +608,19 @@ static krb5_error_code ipadb_verify_pac(krb5_context context, } for (i = 0; i < num_buffers; i++) { - kerr = krb5_pac_get_buffer(context, old_pac, - buffer_types[i], &data); - if (kerr == 0) { - kerr = krb5_pac_add_buffer(context, &new_pac, - buffer_types[i], &data); - } - krb5_free_data_contents(context, &data); - if (kerr) { - krb5_pac_free(context, new_pac); - goto done; + if (buffer_types[i] != PAC_SERVER_CHECKSUM && + buffer_types[i] != PAC_PRIVSVR_CHECKSUM) { + kerr = krb5_pac_get_buffer(context, old_pac, + buffer_types[i], &data); + if (kerr == 0) { + kerr = krb5_pac_add_buffer(context, new_pac, + buffer_types[i], &data); + } + krb5_free_data_contents(context, &data); + if (kerr) { + krb5_pac_free(context, new_pac); + goto done; + } } } > > bye, > Sumit > > > > > Simo. > > > > -- > > Simo Sorce * Red Hat, Inc * New York > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From ohamada at redhat.com Thu Nov 24 14:54:54 2011 From: ohamada at redhat.com (Ondrej Hamada) Date: Thu, 24 Nov 2011 15:54:54 +0100 Subject: [Freeipa-devel] [PATCH] 5 User-add random password support Message-ID: <4ECE5ABE.3060207@redhat.com> https://fedorahosted.org/freeipa/ticket/1979 I've used code from ipalib/plugins/host.py to add support for random password generation. The '--random' option is now available in user-add and user-mod commands. If both the 'password' and 'random' options are used the 'random' option will be ignored. -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ohamada-5-User-add-random-password-support.patch Type: text/x-patch Size: 7213 bytes Desc: not available URL: From ohamada at redhat.com Thu Nov 24 16:51:22 2011 From: ohamada at redhat.com (Ondrej Hamada) Date: Thu, 24 Nov 2011 17:51:22 +0100 Subject: [Freeipa-devel] [PATCH] 5 User-add random password support In-Reply-To: <4ECE5ABE.3060207@redhat.com> References: <4ECE5ABE.3060207@redhat.com> Message-ID: <4ECE760A.7000401@redhat.com> On 11/24/2011 03:54 PM, Ondrej Hamada wrote: > https://fedorahosted.org/freeipa/ticket/1979 > > I've used code from ipalib/plugins/host.py to add support for random > password generation. The '--random' option is now available in > user-add and user-mod commands. If both the 'password' and 'random' > options are used the 'random' option will be ignored. > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel Turned out there remained problem in user_mod function when running it with both 'random' and 'password' options. Corrected patch attached -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ohamada-5-2-User-add-random-password-support.patch Type: text/x-patch Size: 7413 bytes Desc: not available URL: From jkuncar at redhat.com Fri Nov 25 11:21:37 2011 From: jkuncar at redhat.com (Jiri Kuncar) Date: Fri, 25 Nov 2011 06:21:37 -0500 (EST) Subject: [Freeipa-devel] [PATCH] Update PTR records when A/AAAA records are updated In-Reply-To: Message-ID: <9e6119ca-4c4a-460e-a004-bc811f6a5547@zmail13.collab.prod.int.phx2.redhat.com> Bind-dyndb-ldap keeps PTR record synchronized with coresponding A/AAAA record. https://fedorahosted.org/bind-dyndb-ldap/ticket/33 -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Update-PTR-records-when-A-AAAA-records-are-updated.patch Type: text/x-patch Size: 9946 bytes Desc: not available URL: From sbose at redhat.com Fri Nov 25 17:12:31 2011 From: sbose at redhat.com (Sumit Bose) Date: Fri, 25 Nov 2011 18:12:31 +0100 Subject: [Freeipa-devel] [PATCH] 8 Add DNS service records for Windows In-Reply-To: <4ECD74C6.5080602@redhat.com> References: <20111014101557.GC4622@localhost.localdomain> <20111014151202.GD4622@localhost.localdomain> <20111014172150.GA17515@redhat.com> <20111014191648.GE4622@localhost.localdomain> <20111121083211.GD13007@redhat.com> <4ECD74C6.5080602@redhat.com> Message-ID: <20111125171231.GA2246@localhost.localdomain> On Wed, Nov 23, 2011 at 05:33:42PM -0500, Rob Crittenden wrote: > Alexander Bokovoy wrote: > >Hi Sumit, > > > >On Fri, 14 Oct 2011, Sumit Bose wrote: > >>>It would make more clear what is the default and that it is really > >>>optional setting -- I'm thinking from the perspective of maintenance > >>>of the code in future. > >> > >>Thank you for your comments, new version attached. > >Finally got to test it. ACK. > > > > pushed to master. Sorry, I think you pushed the first version and not -3- which was ACKed by Alexander. bye, Sumit From abokovoy at redhat.com Mon Nov 28 12:26:00 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 28 Nov 2011 14:26:00 +0200 Subject: [Freeipa-devel] [PATCH] 8 Add DNS service records for Windows In-Reply-To: <20111125171231.GA2246@localhost.localdomain> References: <20111014101557.GC4622@localhost.localdomain> <20111014151202.GD4622@localhost.localdomain> <20111014172150.GA17515@redhat.com> <20111014191648.GE4622@localhost.localdomain> <20111121083211.GD13007@redhat.com> <4ECD74C6.5080602@redhat.com> <20111125171231.GA2246@localhost.localdomain> Message-ID: <20111128122600.GA4773@redhat.com> On Fri, 25 Nov 2011, Sumit Bose wrote: > On Wed, Nov 23, 2011 at 05:33:42PM -0500, Rob Crittenden wrote: > > Alexander Bokovoy wrote: > > >Hi Sumit, > > > > > >On Fri, 14 Oct 2011, Sumit Bose wrote: > > >>>It would make more clear what is the default and that it is really > > >>>optional setting -- I'm thinking from the perspective of maintenance > > >>>of the code in future. > > >> > > >>Thank you for your comments, new version attached. > > >Finally got to test it. ACK. > > > > > > > pushed to master. > > Sorry, I think you pushed the first version and not -3- which was ACKed > by Alexander. Hm. Sumit, could you please rebase -3- on top of current master HEAD+(1)? I tried that briefly myself and failed. (1) Right now I'm also getting make-lint failing due to more strict PyLint in F16/Rawhide and those seems to be corect and also affect adtrustinstance. I'm sending the patch shortly, so please rebase on top of it. -- / Alexander Bokovoy From abokovoy at redhat.com Mon Nov 28 12:28:52 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 28 Nov 2011 14:28:52 +0200 Subject: [Freeipa-devel] [PATCH] 0037 make-lint fails on Fedora 16/Rawhide Message-ID: <20111128122851.GB4773@redhat.com> Hi, Attached are fixes for ldap.LDAPObject.add_s(self, dn, modlist) uses which now don't pass 'make-lint' on Fedora 16/Rawhide. -- / Alexander Bokovoy -------------- next part -------------- >From dd866262c98be779a094a617975145e2fb1e0dd1 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Mon, 28 Nov 2011 14:21:17 +0200 Subject: [PATCH] Be more explicit when passing Entry class to ldap.LDAPObject.add_s() ldap.LDAPObject.add_s(self, dn, modlist) requires two positional arguments. We used to pass our Entry class which implements dictionary access that gives proper way to pass the positional arguments, but PyLint in Fedora 16/Rawhide became more strict about that and can't infer dictionary interface through static checking. Thus, we need to explicitly annotate dictionary passing with **entry syntax. This has additional benefit to remind that we deal with multiple arguments here. --- ipaserver/install/adtrustinstance.py | 6 +++--- ipaserver/install/krbinstance.py | 4 ++-- ipaserver/install/replication.py | 14 +++++++------- ipaserver/install/service.py | 2 +- 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 96f99dc9b76370c07b727e8e94ccabfe7afca074..2f6d75bb2921eca455e49cc970ddc68e21e1ce55 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -99,7 +99,7 @@ class ADTRUSTInstance(service.Service): entry.setValues("objectclass", ["account", "simplesecurityobject"]) entry.setValues("uid", "samba") entry.setValues("userPassword", self.smb_dn_pwd) - self.admin_conn.add_s(entry) + self.admin_conn.add_s(**entry) # And finally grant it permission to read NT passwords, we do not want # to support LM passwords so there is no need to allow access to them. @@ -188,7 +188,7 @@ class ADTRUSTInstance(service.Service): entry = ipaldap.Entry(self.trust_dn) entry.setValues("objectclass", ["nsContainer"]) entry.setValues("cn", "trusts") - self.admin_conn.add_s(entry) + self.admin_conn.add_s(**entry) entry = ipaldap.Entry(self.smb_dom_dn) entry.setValues("objectclass", ["sambaDomain", "nsContainer"]) @@ -196,7 +196,7 @@ class ADTRUSTInstance(service.Service): entry.setValues("sambaDomainName", self.netbios_name) entry.setValues("sambaSID", self.__gen_sid_string()) #TODO: which MAY attributes do we want to set ? - self.admin_conn.add_s(entry) + self.admin_conn.add_s(**entry) def __write_smb_conf(self): self.fstore.backup_file(self.smb_conf) diff --git a/ipaserver/install/krbinstance.py b/ipaserver/install/krbinstance.py index 6ed38516223a56e723bcc9cf643ac2b362200da4..72c22d7971a024f19e06c2eae8d4129f6f60cf1e 100644 --- a/ipaserver/install/krbinstance.py +++ b/ipaserver/install/krbinstance.py @@ -259,7 +259,7 @@ class KrbInstance(service.Service): entry.setValues("nsSaslMapFilterTemplate", '(krbPrincipalName=\\1@\\2)') try: - self.admin_conn.add_s(entry) + self.admin_conn.add_s(**entry) except ldap.ALREADY_EXISTS: root_logger.critical("failed to add Full Principal Sasl mapping") raise e @@ -272,7 +272,7 @@ class KrbInstance(service.Service): entry.setValues("nsSaslMapFilterTemplate", '(krbPrincipalName=&@%s)' % self.realm) try: - self.admin_conn.add_s(entry) + self.admin_conn.add_s(**entry) except ldap.ALREADY_EXISTS: root_logger.critical("failed to add Name Only Sasl mapping") raise e diff --git a/ipaserver/install/replication.py b/ipaserver/install/replication.py index a139fd0fbe7168193dcfa6ba5f4d19f20d395c52..7aaf959a3904916101b0454b6df502e2996f2771 100644 --- a/ipaserver/install/replication.py +++ b/ipaserver/install/replication.py @@ -227,7 +227,7 @@ class ReplicationManager(object): ent.setValues("sn", "replication manager pseudo user") try: - conn.add_s(ent) + conn.add_s(**ent) except ldap.ALREADY_EXISTS: conn.modify_s(dn, [(ldap.MOD_REPLACE, "userpassword", pw)]) pass @@ -277,7 +277,7 @@ class ReplicationManager(object): entry.setValues('nsds5replicabinddn', [replica_binddn]) entry.setValues('nsds5replicalegacyconsumer', "off") - conn.add_s(entry) + conn.add_s(**entry) def setup_changelog(self, conn): dn = "cn=changelog5, cn=config" @@ -287,7 +287,7 @@ class ReplicationManager(object): entry.setValues('cn', "changelog5") entry.setValues('nsslapd-changelogdir', dirpath) try: - conn.add_s(entry) + conn.add_s(**entry) except ldap.ALREADY_EXISTS: return @@ -310,7 +310,7 @@ class ReplicationManager(object): entry.setValues('nsmultiplexorbinddn', self.repl_man_dn) entry.setValues('nsmultiplexorcredentials', self.repl_man_passwd) - self.conn.add_s(entry) + conn.add_s(**entry) done = True except ldap.ALREADY_EXISTS: benum += 1 @@ -378,7 +378,7 @@ class ReplicationManager(object): entry.setValues("objectclass", ["account", "simplesecurityobject"]) entry.setValues("uid", "passsync") entry.setValues("userPassword", password) - conn.add_s(entry) + conn.add_s(**entry) # Add it to the list of users allowed to bypass password policy extop_dn = "cn=ipa_pwd_extop,cn=plugins,cn=config" @@ -476,7 +476,7 @@ class ReplicationManager(object): if iswinsync: self.setup_winsync_agmt(entry, win_subtree) - a_conn.add_s(entry) + a_conn.add_s(**entry) try: mod = [(ldap.MOD_ADD, 'nsDS5ReplicatedAttributeListTotal', @@ -765,7 +765,7 @@ class ReplicationManager(object): entry.setValues("ipaConfigString", "winsync:%s" % self.hostname) try: - self.conn.add_s(entry) + self.conn.add_s(**entry) except Exception, e: root_logger.info("Failed to create public entry for winsync replica") diff --git a/ipaserver/install/service.py b/ipaserver/install/service.py index 249727b15b6f3c879336c36d43410fc7234d9d9d..2211ba5946d8a1af99f33044d80e57be8a017992 100644 --- a/ipaserver/install/service.py +++ b/ipaserver/install/service.py @@ -289,7 +289,7 @@ class Service(object): "enabledService", "startOrder " + str(order)) try: - conn.add_s(entry) + conn.add_s(**entry) except ldap.ALREADY_EXISTS, e: root_logger.critical("failed to add %s Service startup entry" % name) raise e -- 1.7.7.3 From abokovoy at redhat.com Mon Nov 28 15:26:11 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 28 Nov 2011 17:26:11 +0200 Subject: [Freeipa-devel] [PATCH] 0037 make-lint fails on Fedora 16/Rawhide In-Reply-To: <20111128122851.GB4773@redhat.com> References: <20111128122851.GB4773@redhat.com> Message-ID: <20111128152609.GA18195@redhat.com> On Mon, 28 Nov 2011, Alexander Bokovoy wrote: > Hi, > > Attached are fixes for ldap.LDAPObject.add_s(self, dn, modlist) uses > which now don't pass 'make-lint' on Fedora 16/Rawhide. > > -- > / Alexander Bokovoy > >From dd866262c98be779a094a617975145e2fb1e0dd1 Mon Sep 17 00:00:00 2001 > From: Alexander Bokovoy > Date: Mon, 28 Nov 2011 14:21:17 +0200 > Subject: [PATCH] Be more explicit when passing Entry class to > ldap.LDAPObject.add_s() > > ldap.LDAPObject.add_s(self, dn, modlist) requires two positional arguments. > We used to pass our Entry class which implements dictionary access that gives > proper way to pass the positional arguments, but PyLint in Fedora 16/Rawhide > became more strict about that and can't infer dictionary interface through > static checking. > > Thus, we need to explicitly annotate dictionary passing with **entry syntax. > This has additional benefit to remind that we deal with multiple arguments here. Self-NACK, doesn't really work this way. I'm still looking at better approach but intermediate solution is to use pylint hints. Conservative patch is attached. -- / Alexander Bokovoy -------------- next part -------------- >From 4eb6c88392f43d57943426968a98fca1a91f302c Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Mon, 28 Nov 2011 17:02:11 +0200 Subject: [PATCH] Disable pylint false positive when passing Entry class to ldap.LDAPObject.add_s() ldap.LDAPObject.add_s(self, dn, modlist) requires two positional arguments. We used to pass our Entry class which implements dictionary access that gives proper way to pass the positional arguments, but PyLint in Fedora 16/Rawhide became more strict about that and can't infer dictionary interface through static checking. Thus, we need to explicitly mark this error as false positive as PyLint is unable to guess that. This has additional benefit to remind that we deal with multiple arguments here. --- ipaserver/install/adtrustinstance.py | 6 +++--- ipaserver/install/krbinstance.py | 4 ++-- ipaserver/install/service.py | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 96f99dc9b76370c07b727e8e94ccabfe7afca074..8a9e3d9375f00fef9eb7aafb0a22e88cd09d7270 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -99,7 +99,7 @@ class ADTRUSTInstance(service.Service): entry.setValues("objectclass", ["account", "simplesecurityobject"]) entry.setValues("uid", "samba") entry.setValues("userPassword", self.smb_dn_pwd) - self.admin_conn.add_s(entry) + self.admin_conn.add_s(entry) #pylint: disable=E1120 # And finally grant it permission to read NT passwords, we do not want # to support LM passwords so there is no need to allow access to them. @@ -188,7 +188,7 @@ class ADTRUSTInstance(service.Service): entry = ipaldap.Entry(self.trust_dn) entry.setValues("objectclass", ["nsContainer"]) entry.setValues("cn", "trusts") - self.admin_conn.add_s(entry) + self.admin_conn.add_s(entry) #pylint: disable=E1120 entry = ipaldap.Entry(self.smb_dom_dn) entry.setValues("objectclass", ["sambaDomain", "nsContainer"]) @@ -196,7 +196,7 @@ class ADTRUSTInstance(service.Service): entry.setValues("sambaDomainName", self.netbios_name) entry.setValues("sambaSID", self.__gen_sid_string()) #TODO: which MAY attributes do we want to set ? - self.admin_conn.add_s(entry) + self.admin_conn.add_s(entry) #pylint: disable=E1120 def __write_smb_conf(self): self.fstore.backup_file(self.smb_conf) diff --git a/ipaserver/install/krbinstance.py b/ipaserver/install/krbinstance.py index 6ed38516223a56e723bcc9cf643ac2b362200da4..193dacd2b406332c83ed8b3b86cd7b3cff20471a 100644 --- a/ipaserver/install/krbinstance.py +++ b/ipaserver/install/krbinstance.py @@ -259,7 +259,7 @@ class KrbInstance(service.Service): entry.setValues("nsSaslMapFilterTemplate", '(krbPrincipalName=\\1@\\2)') try: - self.admin_conn.add_s(entry) + self.admin_conn.add_s(entry) #pylint: disable=E1120 except ldap.ALREADY_EXISTS: root_logger.critical("failed to add Full Principal Sasl mapping") raise e @@ -272,7 +272,7 @@ class KrbInstance(service.Service): entry.setValues("nsSaslMapFilterTemplate", '(krbPrincipalName=&@%s)' % self.realm) try: - self.admin_conn.add_s(entry) + self.admin_conn.add_s(entry) #pylint: disable=E1120 except ldap.ALREADY_EXISTS: root_logger.critical("failed to add Name Only Sasl mapping") raise e diff --git a/ipaserver/install/service.py b/ipaserver/install/service.py index 249727b15b6f3c879336c36d43410fc7234d9d9d..68888bc70fa2e661a01b78a0a74490c56dabfce5 100644 --- a/ipaserver/install/service.py +++ b/ipaserver/install/service.py @@ -289,7 +289,7 @@ class Service(object): "enabledService", "startOrder " + str(order)) try: - conn.add_s(entry) + conn.add_s(entry) #pylint: disable=E1120 except ldap.ALREADY_EXISTS, e: root_logger.critical("failed to add %s Service startup entry" % name) raise e -- 1.7.7.3 From mkosek at redhat.com Mon Nov 28 16:35:36 2011 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 28 Nov 2011 17:35:36 +0100 Subject: [Freeipa-devel] [PATCH] [WIP] 172+173 Create per-type DNS API Message-ID: <1322498136.12794.22.camel@balmora.brq.redhat.com> I have prepared a working prototype of the new structured DNS API. It may still have rough edges (and unit tests are not ready), but it will provide a base for discussion and for WebUI folks - so that they can start development of the new DNS WebUI API. The patch takes advantage of the DNS refactor I did in 172. For all supported non-DNSSEC RR types, the following commands are available: dnsrecord-show ZONE NAME dnsrecord-add ZONE NAME dnsrecord-mod ZONE NAME VALUE This is an example of the new API in action: # ipa dnsrecord-show example.com foo Record name: foo A record: 10.0.0.1 # ipa dnsrecordmx-add example.com foo --exchanger="foo.example.com." MX record: 0 foo.example.com. Preference: 0 Exchanger: foo.example.com. ---------------------------- Number of entries returned 1 ---------------------------- # ipa dnsrecordmx-add example.com foo --preference=1 --exchanger="foo.example.com." MX record: 0 foo.example.com. Preference: 0 Exchanger: foo.example.com. MX record: 1 foo.example.com. Preference: 1 Exchanger: foo.example.com. ---------------------------- Number of entries returned 2 ---------------------------- # ipa dnsrecordmx-show example.com foo MX record: 0 foo.example.com. Preference: 0 Exchanger: foo.example.com. MX record: 1 foo.example.com. Preference: 1 Exchanger: foo.example.com. ---------------------------- Number of entries returned 2 ---------------------------- There is an interactive wizard to help user modify a record without specifying an updated value first. If there is just one (MX) record, no wizard would be run. # ipa dnsrecordmx-mod example.com foo --preference=2 Which MX record would you like to modify? [1]: 0 foo.example.com. [2]: 1 foo.example.com. DNS record number: 2 MX record: 0 foo.example.com. Preference: 0 Exchanger: foo.example.com. MX record: 2 foo.example.com. Preference: 2 Exchanger: foo.example.com. ---------------------------- Number of entries returned 2 ---------------------------- # ipa dnsrecordmx-mod example.com foo "2 foo.example.com." --preference=3 MX record: 0 foo.example.com. Preference: 0 Exchanger: foo.example.com. MX record: 3 foo.example.com. Preference: 3 Exchanger: foo.example.com. ---------------------------- Number of entries returned 2 ---------------------------- There are few open questions I am still thinking about: 1) The commands return a list of structured records (just like *-find commands) instead of returning just one record. I thought that it may be more usable this way and consistent with dnsrecord-add/mod/show commands behavior which returns all records too. Otherwise, we would have to change the show command API and add VALUE argument, which would specify a value to be displayed: dnsrecord-show ZONE NAME VALUE 2) Raw DNS record value is in the output too. I though it would be useful to see the raw DNS record value + its parts at one place. 3) The commands are in format dnsrecord-cmd, for example dnsrecordmx-add. I think dnsrecord-mx-add may be more readable. If we want to go this way, I would have to bend the server framework a little which parses an LDAP object from the command name (LDAP object name is dnsrecordmx in this case). This is doable, although I am not sure if this does not have some implications in WebUI side. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-173-create-per-type-dns-api.patch Type: text/x-patch Size: 60297 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-172-2-rework-dnsrecord-processing.patch Type: text/x-patch Size: 95547 bytes Desc: not available URL: From abokovoy at redhat.com Mon Nov 28 17:10:35 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 28 Nov 2011 19:10:35 +0200 Subject: [Freeipa-devel] [PATCH] 906 Add SELinux user mapping framework. In-Reply-To: <20111123230047.GB14772@redhat.com> References: <4ECD6D4F.600@redhat.com> <20111123230047.GB14772@redhat.com> Message-ID: <20111128171034.GA26611@redhat.com> On Thu, 24 Nov 2011, Alexander Bokovoy wrote: > On Wed, 23 Nov 2011, Rob Crittenden wrote: > > This will allow one to define what SELinux context a given user gets > > on a given machine. A rule can contain a set of users and hosts or it > > can point to an existing HBAC rule that defines them. > > > > https://fedorahosted.org/freeipa/ticket/755 > I read through the patch, will need to test it later this week. I > basically have two minor points: > > 1. Split charachter in the SE Linux user map order. > > + > > + Define SELinux user map order: > > + ipa config-mod --ipaselinuxusermaporder='guest_u:s0$xguest_u:s0$user_u:s0-s0:c0.c1023$staff_u:s0-s0:c0.c1023$unconfined_u:s0-s0:c0.c1023' > > """) > $ can be considered 'active' character in all shells in a sense it > changes treatment of following characters from the shell perspective > and therefore will always require shielding from the shell's > influence. This increases likelyhood of error from a user side. > > Maybe / would be more neutral character? > > As you said on IRC, people might have religious feeling about > separators but tricking users into always thinking about > escaping/single quoting is equally bad. > > 2. We have two possible ways to address named properties in MagicDict > and NameSpace objects -- through explicit attribute use and through > the dictionary key. I guess for the cases when we know the attribute > name in advance, it would perhaps be preferrable to use the former > style: > > > + def pre_callback(self, ldap, dn, *keys, **options): > > + kw = dict(seealso=dn) > > + _entries = api.Command['selinuxusermap_find'](None, **kw) > this would be > _entries = api.Command.selinuxusermap_find(None, **kw) > > Other than those two minor points, the patch looks very good. I'm > going to give it a run on Friday. I tested the patch and it works for me on a new install. On upgrade of existing installation I've got few errors during run of ipa-ldap-updater for SELinux schema changes. Unfortunately, didn't save the log as it was 2.1 -> 2.99 upgrade as well. -- / Alexander Bokovoy From sbose at redhat.com Mon Nov 28 17:16:34 2011 From: sbose at redhat.com (Sumit Bose) Date: Mon, 28 Nov 2011 18:16:34 +0100 Subject: [Freeipa-devel] [PATCH] 8 Add DNS service records for Windows In-Reply-To: <20111128122600.GA4773@redhat.com> References: <20111014101557.GC4622@localhost.localdomain> <20111014151202.GD4622@localhost.localdomain> <20111014172150.GA17515@redhat.com> <20111014191648.GE4622@localhost.localdomain> <20111121083211.GD13007@redhat.com> <4ECD74C6.5080602@redhat.com> <20111125171231.GA2246@localhost.localdomain> <20111128122600.GA4773@redhat.com> Message-ID: <20111128171634.GD2246@localhost.localdomain> On Mon, Nov 28, 2011 at 02:26:00PM +0200, Alexander Bokovoy wrote: > On Fri, 25 Nov 2011, Sumit Bose wrote: > > On Wed, Nov 23, 2011 at 05:33:42PM -0500, Rob Crittenden wrote: > > > Alexander Bokovoy wrote: > > > >Hi Sumit, > > > > > > > >On Fri, 14 Oct 2011, Sumit Bose wrote: > > > >>>It would make more clear what is the default and that it is really > > > >>>optional setting -- I'm thinking from the perspective of maintenance > > > >>>of the code in future. > > > >> > > > >>Thank you for your comments, new version attached. > > > >Finally got to test it. ACK. > > > > > > > > > > pushed to master. > > > > Sorry, I think you pushed the first version and not -3- which was ACKed > > by Alexander. > Hm. Sumit, could you please rebase -3- on top of current master > HEAD+(1)? > > I tried that briefly myself and failed. > > (1) Right now I'm also getting make-lint failing due to more strict > PyLint in F16/Rawhide and those seems to be corect and also affect > adtrustinstance. > > I'm sending the patch shortly, so please rebase on top of it. ok, rebased version is attached. To push this upstream you still have to revert the wrong commit. bye, Sumit > > -- > / Alexander Bokovoy -------------- next part -------------- >From 9d8f35baa97c332e4eb676d01815cc68050f37ef Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Thu, 13 Oct 2011 12:01:57 +0200 Subject: [PATCH] Add DNS service records for Windows https://fedorahosted.org/freeipa/ticket/1939 --- install/tools/ipa-adtrust-install | 5 ++- install/tools/man/ipa-adtrust-install.1 | 3 ++ ipaserver/install/adtrustinstance.py | 59 +++++++++++++++++++++++++++++- 3 files changed, 64 insertions(+), 3 deletions(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index 9a6e61c2c5e8148a13d51718edc4e38a65af1fec..87fecbfb4834d65fdccc3f8536a5665ba75e48a5 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -45,6 +45,9 @@ def parse_options(): type="ip", ip_local=True, help="Master Server IP Address") parser.add_option("--netbios-name", dest="netbios_name", help="NetBIOS name of the IPA domain") + parser.add_option("--no-msdcs", dest="no_msdcs", action="store_true", + default=False, help="Do not create DNS service records " \ + "for Windows in managed DNS server") parser.add_option("-U", "--unattended", dest="unattended", action="store_true", default=False, help="unattended installation never prompts the user") @@ -197,7 +200,7 @@ def main(): api.Backend.ldap2.connect(ccache) smb.setup(api.env.host, ip_address, api.env.realm, api.env.domain, - netbios_name) + netbios_name, options.no_msdcs) smb.create_instance() print "==============================================================================" diff --git a/install/tools/man/ipa-adtrust-install.1 b/install/tools/man/ipa-adtrust-install.1 index a3981adf48d14cc0e540c646fff099490203f862..b61da19088b40d6a9e53784f9a061913ecda4321 100644 --- a/install/tools/man/ipa-adtrust-install.1 +++ b/install/tools/man/ipa-adtrust-install.1 @@ -39,6 +39,9 @@ The IP address of the IPA server. If not provided then this is determined based \fB\-\-netbios\-name\fR=\fINETBIOS_NAME\fR The NetBIOS name for the IPA domain. If not provided then this is determined based on the leading component of the DNS domain name. .TP +\fB\-\-no\-msdcs\fR +Do not create DNS service records for Windows in managed DNS server +.TP \fB\-U\fR, \fB\-\-unattended\fR An unattended installation that will never prompt for user input .SH "EXIT STATUS" diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 1fcf9e03716a1b31aac47e206adc6ee86eee1cd9..bbda11cc752923e010a06daac87aaba532cfbbb4 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -25,7 +25,9 @@ import tempfile import installutils from ipaserver import ipaldap from ipaserver.install.dsinstance import realm_to_serverid -from ipalib import errors +from ipaserver.install.bindinstance import get_rr, add_rr, del_rr, \ + dns_zone_exists +from ipalib import errors, api from ipapython import sysrestore from ipapython import ipautil from ipapython.ipa_log_manager import * @@ -245,6 +247,56 @@ class ADTRUSTInstance(service.Service): except ipautil.CalledProcessError, e: root_logger.critical("Failed to add key for %s" % cifs_principal) + def __add_dns_service_records(self): + """ + Add DNS service records for Windows if DNS is enabled and the DNS zone + is managed. If there are already service records for LDAP and Kerberos + their values are used. Otherwise default values are used. + """ + + zone = self.domain_name + host = self.fqdn.split(".")[0] + + ipa_srv_rec = ( + ("_ldap._tcp", ["0 100 389 %s" % host]), + ("_kerberos._tcp", ["0 100 88 %s" % host]), + ("_kerberos._udp", ["0 100 88 %s" % host]) + ) + win_srv_suffix = (".Default-First-Site-Name._sites.dc._msdcs", + ".dc._msdcs") + + err_msg = None + ret = api.Command.dns_is_enabled() + if not ret['result']: + err_msg = "DNS management was not enabled at install time." + else: + if not dns_zone_exists(zone): + err_msg = "DNS zone %s cannot be managed " \ + "as it is not defined in IPA" % zone + + if err_msg: + print err_msg + print "Add the following service records to your DNS server " \ + "for DNS zone %s: " % zone + for (srv, rdata) in ipa_srv_rec: + for suff in win_srv_suffix: + print " - %s%s" % (srv, suff) + return + + for (srv, rdata) in ipa_srv_rec: + ipa_rdata = get_rr(zone, srv, "SRV") + if not ipa_rdata: + ipa_rdata = rdata + + for suff in win_srv_suffix: + win_srv = srv+suff + win_rdata = get_rr(zone, win_srv, "SRV") + if win_rdata: + for rec in win_rdata: + del_rr(zone, win_srv, "SRV", rec) + for rec in ipa_rdata: + add_rr(zone, win_srv, "SRV", rec) + def __start(self): try: self.start() @@ -277,12 +329,13 @@ class ADTRUSTInstance(service.Service): LDAPI_SOCKET = self.ldapi_socket) def setup(self, fqdn, ip_address, realm_name, domain_name, netbios_name, - smbd_user="samba"): + no_msdcs=False, smbd_user="samba"): self.fqdn =fqdn self.ip_address = ip_address self.realm_name = realm_name self.domain_name = domain_name self.netbios_name = netbios_name + self.no_msdcs = no_msdcs self.smbd_user = smbd_user self.suffix = ipautil.realm_to_suffix(self.realm_name) self.ldapi_socket = "%%2fvar%%2frun%%2fslapd-%s.socket" % realm_to_serverid(self.realm_name) @@ -311,6 +364,8 @@ class ADTRUSTInstance(service.Service): self.step("Adding cifs Kerberos principal", self.__setup_principal) self.step("Adding admin(group) SIDs", self.__add_admin_sids) self.step("configuring smbd to start on boot", self.__enable) + if not self.no_msdcs: + self.step("adding special DNS service records", self.__add_dns_service_records) self.step("starting smbd", self.__start) self.start_creation("Configuring smbd:") -- 1.7.6 From sbose at redhat.com Mon Nov 28 17:22:27 2011 From: sbose at redhat.com (Sumit Bose) Date: Mon, 28 Nov 2011 18:22:27 +0100 Subject: [Freeipa-devel] [PATCH] Add ipasam samba passdb backend In-Reply-To: <20111123154802.GI2205@localhost.localdomain> References: <20111123154802.GI2205@localhost.localdomain> Message-ID: <20111128172227.GE2246@localhost.localdomain> On Wed, Nov 23, 2011 at 04:48:02PM +0100, Sumit Bose wrote: > Hi, > > this set of patches basically adds a samba passwd backend for IPA with > can be build in the freeipa tree, plus the needed new objectclasses and > attributes and enables the CLDAP service from Simo which is already > committed. > > I compressed "Add-ipasam-samba-passdb-backend" to save some bandwidth. > The backend is based on the old IPA passdb backend form the samba tree > and various modified parts from the samba LDAP backend to make it work. > As the result there are parts of the code which are not very pretty, > but will work as planned. I will start refactoring the code together > with fixing the first Coverity findings. > > bye, > Sumit Please find attached a rebased version on top of Alexander's latest patch. bye, Sumit -------------- next part -------------- From 0cb634c5d792f248680215b920239cf77105dd77 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Mon, 7 Nov 2011 11:56:57 +0100 Subject: [PATCH 1/6] Move our own domain info into cn=etc https://fedorahosted.org/freeipa/ticket/2001 --- ipaserver/install/adtrustinstance.py | 26 +++++++++++++++++--------- 1 files changed, 17 insertions(+), 9 deletions(-) diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index bbda11cc752923e010a06daac87aaba532cfbbb4..089ba0bd480ca55c917837832d7c643bd87efe7d 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -183,17 +183,24 @@ class ADTRUSTInstance(service.Service): except errors.NotFound: pass - try: - self.admin_conn.getEntry(self.trust_dn, ldap.SCOPE_BASE) - except errors.NotFound: - entry = ipaldap.Entry(self.trust_dn) - entry.setValues("objectclass", ["nsContainer"]) - entry.setValues("cn", "trusts") - self.admin_conn.add_s(entry) #pylint: disable=E1120 + for new_dn in (self.trust_dn, \ + "cn=ad,"+self.trust_dn, \ + "cn=ad,cn=etc,"+self.suffix): + try: + self.admin_conn.getEntry(dn, ldap.SCOPE_BASE) + except errors.NotFound: + entry = ipaldap.Entry(dn) + entry.setValues("objectclass", ["nsContainer"]) + name = dn.split('=')[1].split(',')[0] + if not name: + print "Cannot extract RDN attribute value from [%s]" % dn + return + entry.setValues("cn", name) + self.admin_conn.add_s(entry) #pylint: disable=E1120 entry = ipaldap.Entry(self.smb_dom_dn) entry.setValues("objectclass", ["sambaDomain", "nsContainer"]) - entry.setValues("cn", "ad") + entry.setValues("cn", self.domain_name) entry.setValues("sambaDomainName", self.netbios_name) entry.setValues("sambaSID", self.__gen_sid_string()) #TODO: which MAY attributes do we want to set ? @@ -346,7 +353,8 @@ class ADTRUSTInstance(service.Service): self.smb_dn_pwd = ipautil.ipa_generate_password() self.trust_dn = "cn=trusts,%s" % self.suffix - self.smb_dom_dn = "cn=ad,%s" % self.trust_dn + self.smb_dom_dn = "cn=%s,cn=ad,cn=etc,%s" % (self.domain_name, \ + self.suffix) self.__setup_sub_dict() -- 1.7.6 -------------- next part -------------- From ffff36474df700147df0e296a355140cc39f5c49 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Wed, 9 Nov 2011 16:38:10 +0100 Subject: [PATCH 2/6] Add trust objectclass and attributes to v3 schema --- install/share/60basev3.ldif | 11 +++++++++++ 1 files changed, 11 insertions(+), 0 deletions(-) diff --git a/install/share/60basev3.ldif b/install/share/60basev3.ldif index f518541586b2df9ed08718098a7f170563aa4e1d..6db644addf298216e2b85dc68b616e8351457cf5 100644 --- a/install/share/60basev3.ldif +++ b/install/share/60basev3.ldif @@ -14,7 +14,18 @@ attributeTypes: (2.16.840.1.113730.3.8.11.7 NAME 'ipaNTProfilePath' DESC 'User P attributeTypes: (2.16.840.1.113730.3.8.11.8 NAME 'ipaNTHomeDirectory' DESC 'User Home Directory Path' EQUALITY caseIgnoreMatch OREDRING caseIgnoreOrderingMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'IPA v3' ) attributeTypes: (2.16.840.1.113730.3.8.11.9 NAME 'ipaNTHomeDirectoryDrive' DESC 'User Home Drive Letter' EQUALITY caseIgnoreMatch OREDRING caseIgnoreOrderingMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'IPA v3' ) attributeTypes: (2.16.840.1.113730.3.8.11.10 NAME 'ipaNTDomainGUID' DESC 'NT Domain GUID' EQUALITY caseIgnoreIA5Match OREDRING caseIgnoreIA5OrderingMatch SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VALUE X-ORIGIN 'IPA v3' ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.11 NAME 'ipaNTTrustType' DESC 'Type of trust' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.12 NAME 'ipaNTTrustAttributes' DESC 'Trust attributes for a trusted domain' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.13 NAME 'ipaNTTrustDirection' DESC 'Direction of a trust' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.14 NAME 'ipaNTTrustPartner' DESC 'Fully qualified name of the domain with which a trust exists' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{128} ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.15 NAME 'ipaNTTrustAuthOutgoing' DESC 'Authentication information for the outgoing portion of a trust' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.16 NAME 'ipaNTTrustAuthIncoming' DESC 'Authentication information for the incoming portion of a trust' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.17 NAME 'ipaNTTrustForestTrustInfo' DESC 'Forest trust information for a trusted domain object' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.18 NAME 'ipaNTTrustPosixOffset' DESC 'POSIX offset of a trust' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.19 NAME 'ipaNTSupportedEncryptionTypes' DESC 'Supported encryption types of a trust' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) objectClasses: (2.16.840.1.113730.3.8.12.1 NAME 'ipaExternalGroup' SUP top STRUCTURAL MUST ( cn ) MAY ( ipaExternalMember $ memberOf $ description $ owner) X-ORIGIN 'IPA v3' ) objectClasses: (2.16.840.1.113730.3.8.12.2 NAME 'ipaNTUserAttrs' SUP top AUXILIARY MUST ( ipaNTSecurityIdentifier ) MAY ( ipaNTHash $ ipaNTLogonScript $ ipaNTProfilePath $ ipaNTHomeDirectory $ ipaNTHomeDirectoryDrive ) X-ORIGIN 'IPA v3' ) objectClasses: (2.16.840.1.113730.3.8.12.3 NAME 'ipaNTGroupAttrs' SUP top AUXILIARY MUST ( ipaNTSecurityIdentifier ) X-ORIGIN 'IPA v3' ) objectClasses: (2.16.840.1.113730.3.8.12.4 NAME 'ipaNTDomainAttrs' SUP top AUXILIARY MUST ( ipaNTSecurityIdentifier $ ipaNTFlatName $ ipaNTDomainGUID ) MAY ( ipaNTFallbackPrimaryGroup ) X-ORIGIN 'IPA v3' ) +objectClasses: (2.16.840.1.113730.3.8.12.5 NAME 'ipaNTTrustedDomain' SUP top STRUCTURAL DESC 'Trusted Domain Object' MUST ( cn ) MAY ( ipaNTTrustType $ ipaNTTrustAttributes $ ipaNTTrustDirection $ ipaNTTrustPartner $ ipaNTFlatName $ ipaNTTrustAuthOutgoing $ ipaNTTrustAuthIncoming $ ipaNTSecurityIdentifier $ ipaNTTrustForestTrustInfo $ ipaNTTrustPosixOffset $ ipaNTSupportedEncryptionTypes) ) + -- 1.7.6 -------------- next part -------------- From 57c92a9392aacb4c600fffc8581b1d8e9ef2d10d Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Mon, 7 Nov 2011 12:48:10 +0100 Subject: [PATCH 3/6] Use new objectclasses and attributes for trust --- ipaserver/install/adtrustinstance.py | 46 +++++++++++++++++++++++---------- 1 files changed, 32 insertions(+), 14 deletions(-) diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 089ba0bd480ca55c917837832d7c643bd87efe7d..98cd84000b8a2b70cfd2f990537e16dffc3b82c2 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -22,7 +22,7 @@ import errno import ldap import service import tempfile -import installutils +import uuid from ipaserver import ipaldap from ipaserver.install.dsinstance import realm_to_serverid from ipaserver.install.bindinstance import get_rr, add_rr, del_rr, \ @@ -75,6 +75,14 @@ def make_netbios_name(s): return ''.join([c for c in s.split('.')[0].upper() if c in allowed_netbios_chars])[:15] class ADTRUSTInstance(service.Service): + + ATTR_SID = "ipaNTSecurityIdentifier" + ATTR_FLAT_NAME = "ipaNTFlatName" + ATTR_GUID = "ipaNTDomainGUID" + OBJC_USER = "ipaNTUserAttrs" + OBJC_GROUP = "ipaNTGroupAttrs" + OBJC_DOMAIN = "ipaNTDomainAttrs" + def __init__(self, fstore=None, dm_password=None): service.Service.__init__(self, "smb", dm_password=dm_password) @@ -107,13 +115,22 @@ class ADTRUSTInstance(service.Service): # Also the premission to create trusted domain objects below the # domain object is granted. mod = [(ldap.MOD_ADD, 'aci', - str('(targetattr = "sambaNTPassword")' \ + str('(targetattr = "ipaNTHash")' \ '(version 3.0; acl "Samba user can read NT passwords";' \ 'allow (read) userdn="ldap:///%s";)' % self.smb_dn)), (ldap.MOD_ADD, 'aci', str('(target = "ldap:///cn=ad,cn=trusts,%s")' \ - '(targetattr = "sambaTrustType || sambaTrustAttributes || sambaTrustDirection || sambaTrustPartner || sambaFlatName || sambaTrustAuthOutgoing || sambaTrustAuthIncoming || sambaSecurityIdentifier || sambaTrustForestTrustInfo || sambaTrustPosixOffset || sambaSupportedEncryptionTypes")' \ - '(version 3.0;acl "Allow samba user to create and delete trust accounts";' \ + '(targetattr = "ipaNTTrustType || ipaNTTrustAttributes || ' \ + 'ipaNTTrustDirection || ' \ + 'ipaNTTrustPartner || ipaNTFlatName || ' \ + 'ipaNTTrustAuthOutgoing || ' \ + 'ipaNTTrustAuthIncoming || ' \ + 'ipaNTSecurityIdentifier || ' \ + 'ipaNTTrustForestTrustInfo || ' \ + 'ipaNTTrustPosixOffset || ' \ + 'ipaNTSupportedEncryptionTypes")' \ + '(version 3.0;acl "Allow samba user to create and delete ' \ + 'trust accounts";' \ 'allow (write,add,delete) userdn = "ldap:///%s";)' % \ (self.suffix, self.smb_dn)))] @@ -137,7 +154,7 @@ class ADTRUSTInstance(service.Service): print "Samba domain object not found" return - dom_sid = dom_entry.getValue("sambaSID") + dom_sid = dom_entry.getValue(self.ATTR_SID) if not dom_sid: print "Samba domain object does not have a SID" return @@ -155,22 +172,22 @@ class ADTRUSTInstance(service.Service): print "IPA admin group object not found" return - if admin_entry.getValue("sambaSID") or \ - admin_group_entry.getValue("sambaSID"): + if admin_entry.getValue(self.ATTR_SID) or \ + admin_group_entry.getValue(self.ATTR_SID): print "Admin SID already set, nothing to do" return try: self.admin_conn.modify_s(admin_dn, \ - [(ldap.MOD_ADD, "objectclass", "sambaSamAccount"), \ - (ldap.MOD_ADD, "sambaSID", dom_sid + "-500")]) + [(ldap.MOD_ADD, "objectclass", self.OBJC_USER), \ + (ldap.MOD_ADD, self.ATTR_SID, dom_sid + "-500")]) except: print "Failed to modify IPA admin object" try: self.admin_conn.modify_s(admin_group_dn, \ - [(ldap.MOD_ADD, "objectclass", "sambaSidEntry"), \ - (ldap.MOD_ADD, "sambaSID", dom_sid + "-512")]) + [(ldap.MOD_ADD, "objectclass", self.OBJC_GROUP), \ + (ldap.MOD_ADD, self.ATTR_SID, dom_sid + "-512")]) except: print "Failed to modify IPA admin group object" @@ -199,10 +216,11 @@ class ADTRUSTInstance(service.Service): self.admin_conn.add_s(entry) #pylint: disable=E1120 entry = ipaldap.Entry(self.smb_dom_dn) - entry.setValues("objectclass", ["sambaDomain", "nsContainer"]) + entry.setValues("objectclass", [self.OBJC_DOMAIN, "nsContainer"]) entry.setValues("cn", self.domain_name) - entry.setValues("sambaDomainName", self.netbios_name) - entry.setValues("sambaSID", self.__gen_sid_string()) + entry.setValues(self.ATTR_FLAT_NAME, self.netbios_name) + entry.setValues(self.ATTR_SID, self.__gen_sid_string()) + entry.setValues(self.ATTR_GUID, str(uuid.uuid4())) #TODO: which MAY attributes do we want to set ? self.admin_conn.add_s(entry) #pylint: disable=E1120 -- 1.7.6 -------------- next part -------------- From 6f9b49521dea3dc9860f60a1cdcae27f62cb9694 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Mon, 7 Nov 2011 12:59:20 +0100 Subject: [PATCH 4/6] Fix some pylint warnings --- install/tools/ipa-adtrust-install | 2 +- ipaserver/install/adtrustinstance.py | 103 +++++++++++++++++++++------------ 2 files changed, 66 insertions(+), 39 deletions(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index 87fecbfb4834d65fdccc3f8536a5665ba75e48a5..c6fd3478a28697301cac317dff1bbf25c6d865ce 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -111,7 +111,7 @@ def main(): print "" # Check if samba packages are installed - if not adtrustinstance.check_inst(options.unattended): + if not adtrustinstance.check_inst(): sys.exit("Aborting installation.") # Initialize the ipalib api diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 98cd84000b8a2b70cfd2f990537e16dffc3b82c2..4b31db81ff86cf63f52f9f507317c7efa3246dbb 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -20,10 +20,11 @@ import os import errno import ldap -import service import tempfile import uuid from ipaserver import ipaldap +from ipaserver.install import installutils +from ipaserver.install import service from ipaserver.install.dsinstance import realm_to_serverid from ipaserver.install.bindinstance import get_rr, add_rr, del_rr, \ dns_zone_exists @@ -32,17 +33,17 @@ from ipapython import sysrestore from ipapython import ipautil from ipapython.ipa_log_manager import * -import random import string import struct -allowed_netbios_chars = string.ascii_uppercase + string.digits +ALLOWED_NETBIOS_CHARS = string.ascii_uppercase + string.digits -def check_inst(unattended): - for f in ['/usr/sbin/smbd', '/usr/bin/net', '/usr/bin/smbpasswd']: - if not os.path.exists(f): - print "%s was not found on this system" % f - print "Please install the 'samba' packages and start the installation again" +def check_inst(): + for smbfile in ['/usr/sbin/smbd', '/usr/bin/net', '/usr/bin/smbpasswd']: + if not os.path.exists(smbfile): + print "%s was not found on this system" % file + print "Please install the 'samba' packages and " \ + "start the installation again" return False #TODO: Add check for needed samba4 libraries @@ -51,13 +52,13 @@ def check_inst(unattended): def ipa_smb_conf_exists(): try: - fd = open('/etc/samba/smb.conf', 'r') - except IOError, e: - if e.errno == errno.ENOENT: + conf_fd = open('/etc/samba/smb.conf', 'r') + except IOError, err: + if err.errno == errno.ENOENT: return False - lines = fd.readlines() - fd.close() + lines = conf_fd.readlines() + conf_fd.close() for line in lines: if line.startswith('### Added by IPA Installer ###'): return True @@ -66,13 +67,15 @@ def ipa_smb_conf_exists(): def check_netbios_name(s): # NetBIOS names may not be longer than 15 allowed characters - if not s or len(s) > 15 or ''.join([c for c in s if c not in allowed_netbios_chars]): + if not s or len(s) > 15 or \ + ''.join([c for c in s if c not in ALLOWED_NETBIOS_CHARS]): return False return True def make_netbios_name(s): - return ''.join([c for c in s.split('.')[0].upper() if c in allowed_netbios_chars])[:15] + return ''.join([c for c in s.split('.')[0].upper() \ + if c in ALLOWED_NETBIOS_CHARS])[:15] class ADTRUSTInstance(service.Service): @@ -84,6 +87,22 @@ class ADTRUSTInstance(service.Service): OBJC_DOMAIN = "ipaNTDomainAttrs" def __init__(self, fstore=None, dm_password=None): + self.fqdn = None + self.ip_address = None + self.realm_name = None + self.domain_name = None + self.netbios_name = None + self.no_msdcs = None + self.smbd_user = None + self.suffix = None + self.ldapi_socket = None + self.smb_conf = None + self.smb_dn = None + self.smb_dn_pwd = None + self.trust_dn = None + self.smb_dom_dn = None + self.sub_dict = None + service.Service.__init__(self, "smb", dm_password=dm_password) if fstore: @@ -97,7 +116,8 @@ class ADTRUSTInstance(service.Service): self.admin_conn.getEntry(self.smb_dn, ldap.SCOPE_BASE) print "Samba user entry exists, resetting password" - self.admin_conn.modify_s(self.smb_dn, [(ldap.MOD_REPLACE, "userPassword", self.smb_dn_pwd)]) + self.admin_conn.modify_s(self.smb_dn, \ + [(ldap.MOD_REPLACE, "userPassword", self.smb_dn_pwd)]) return except errors.NotFound: @@ -204,13 +224,14 @@ class ADTRUSTInstance(service.Service): "cn=ad,"+self.trust_dn, \ "cn=ad,cn=etc,"+self.suffix): try: - self.admin_conn.getEntry(dn, ldap.SCOPE_BASE) + self.admin_conn.getEntry(new_dn, ldap.SCOPE_BASE) except errors.NotFound: - entry = ipaldap.Entry(dn) + entry = ipaldap.Entry(new_dn) entry.setValues("objectclass", ["nsContainer"]) - name = dn.split('=')[1].split(',')[0] + name = new_dn.split('=')[1].split(',')[0] if not name: - print "Cannot extract RDN attribute value from [%s]" % dn + print "Cannot extract RDN attribute value from [%s]" % \ + new_dn return entry.setValues("cn", name) self.admin_conn.add_s(entry) #pylint: disable=E1120 @@ -227,18 +248,18 @@ class ADTRUSTInstance(service.Service): def __write_smb_conf(self): self.fstore.backup_file(self.smb_conf) - fd = open(self.smb_conf, "w") - fd.write('### Added by IPA Installer ###\n') - fd.write('[global]\n') - fd.write('config backend = registry\n') - fd.close() + conf_fd = open(self.smb_conf, "w") + conf_fd.write('### Added by IPA Installer ###\n') + conf_fd.write('[global]\n') + conf_fd.write('config backend = registry\n') + conf_fd.close() def __write_smb_registry(self): template = os.path.join(ipautil.SHARE_DIR, "smb.conf.template") conf = ipautil.template_file(template, self.sub_dict) - [fd, tmp_name] = tempfile.mkstemp() - os.write(fd, conf) - os.close(fd) + [tmp_fd, tmp_name] = tempfile.mkstemp() + os.write(tmp_fd, conf) + os.close(tmp_fd) args = ["/usr/bin/net", "conf", "import", tmp_name] @@ -250,7 +271,8 @@ class ADTRUSTInstance(service.Service): def __set_smb_ldap_password(self): args = ["/usr/bin/smbpasswd", "-c", self.smb_conf, "-s", "-W" ] - ipautil.run(args, stdin = self.smb_dn_pwd + "\n" + self.smb_dn_pwd + "\n" ) + ipautil.run(args, stdin = self.smb_dn_pwd + "\n" + \ + self.smb_dn_pwd + "\n" ) def __setup_principal(self): cifs_principal = "cifs/" + self.fqdn + "@" + self.realm_name @@ -291,7 +313,7 @@ class ADTRUSTInstance(service.Service): ".dc._msdcs") err_msg = None - ret = api.Command.dns_is_enabled() + ret = api.Command['dns_is_enabled']() if not ret['result']: err_msg = "DNS management was not enabled at install time." else: @@ -341,7 +363,8 @@ class ADTRUSTInstance(service.Service): # Instead we reply on the IPA init script to start only enabled # components as found in our LDAP configuration tree try: - self.ldap_enable('ADTRUST', self.fqdn, self.dm_password, self.suffix) + self.ldap_enable('ADTRUST', self.fqdn, self.dm_password, \ + self.suffix) except ldap.ALREADY_EXISTS: root_logger.critical("ADTRUST Service startup entry already exists.") pass @@ -355,7 +378,7 @@ class ADTRUSTInstance(service.Service): def setup(self, fqdn, ip_address, realm_name, domain_name, netbios_name, no_msdcs=False, smbd_user="samba"): - self.fqdn =fqdn + self.fqdn = fqdn self.ip_address = ip_address self.realm_name = realm_name self.domain_name = domain_name @@ -363,7 +386,8 @@ class ADTRUSTInstance(service.Service): self.no_msdcs = no_msdcs self.smbd_user = smbd_user self.suffix = ipautil.realm_to_suffix(self.realm_name) - self.ldapi_socket = "%%2fvar%%2frun%%2fslapd-%s.socket" % realm_to_serverid(self.realm_name) + self.ldapi_socket = "%%2fvar%%2frun%%2fslapd-%s.socket" % \ + realm_to_serverid(self.realm_name) self.smb_conf = "/etc/samba/smb.conf" @@ -383,15 +407,18 @@ class ADTRUSTInstance(service.Service): self.step("stopping smbd", self.__stop) self.step("create samba user", self.__create_samba_user) - self.step("create samba domain object", self.__create_samba_domain_object) + self.step("create samba domain object", \ + self.__create_samba_domain_object) self.step("create samba config registry", self.__write_smb_registry) self.step("writing samba config file", self.__write_smb_conf) - self.step("setting password for the samba user", self.__set_smb_ldap_password) + self.step("setting password for the samba user", \ + self.__set_smb_ldap_password) self.step("Adding cifs Kerberos principal", self.__setup_principal) self.step("Adding admin(group) SIDs", self.__add_admin_sids) self.step("configuring smbd to start on boot", self.__enable) if not self.no_msdcs: - self.step("adding special DNS service records", self.__add_dns_service_records) + self.step("adding special DNS service records", \ + self.__add_dns_service_records) self.step("starting smbd", self.__start) self.start_creation("Configuring smbd:") @@ -408,9 +435,9 @@ class ADTRUSTInstance(service.Service): except: pass - for f in [self.smb_conf]: + for r_file in [self.smb_conf]: try: - self.fstore.restore_file(f) + self.fstore.restore_file(r_file) except ValueError, error: root_logger.debug(error) pass -- 1.7.6 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-sbose-0013-2-Add-ipasam-samba-passdb-backend.patch.bz2 Type: application/x-bzip2 Size: 17030 bytes Desc: not available URL: -------------- next part -------------- From a60f5021596a161d6e0a3fd7357fbab129b74083 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Fri, 18 Nov 2011 14:04:09 +0100 Subject: [PATCH 6/6] activate CLDAP --- install/tools/ipa-adtrust-install | 3 +-- ipaserver/install/adtrustinstance.py | 4 ++++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index c6fd3478a28697301cac317dff1bbf25c6d865ce..248ea35eaa86dd59ebbc871b86df780cfd71ccf6 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -214,6 +214,7 @@ def main(): print "\t\tUDP Ports:" print "\t\t * 138: netbios-dgm" print "\t\t * 139: netbios-ssn" + print "\t\t * 389: (C)LDAP" print "\t\t * 445: microsoft-ds" print "" print "\tAdditionally you have to make sure the FreeIPA LDAP server cannot reached" @@ -221,8 +222,6 @@ def main(): print "\tfollowing ports for these servers:" print "\t\tTCP Ports:" print "\t\t * 389, 636: LDAP/LDAPS" - print "\t\tUDP Ports:" - print "\t\t * 389: (C)LDAP" print "\tYou may want to choose to REJECT the network packets instead of DROPing them" print "\tto avoid timeouts on the AD domain controllers." diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 4b31db81ff86cf63f52f9f507317c7efa3246dbb..93026d78eefe5a930ac8b7fc1551b47137aca213 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -254,6 +254,9 @@ class ADTRUSTInstance(service.Service): conf_fd.write('config backend = registry\n') conf_fd.close() + def __add_cldap_module(self): + self._ldap_mod("ipa-cldap-conf.ldif", self.sub_dict) + def __write_smb_registry(self): template = os.path.join(ipautil.SHARE_DIR, "smb.conf.template") conf = ipautil.template_file(template, self.sub_dict) @@ -415,6 +418,7 @@ class ADTRUSTInstance(service.Service): self.__set_smb_ldap_password) self.step("Adding cifs Kerberos principal", self.__setup_principal) self.step("Adding admin(group) SIDs", self.__add_admin_sids) + self.step("Activation CLDAP plugin", self.__add_cldap_module) self.step("configuring smbd to start on boot", self.__enable) if not self.no_msdcs: self.step("adding special DNS service records", \ -- 1.7.6 From sbose at redhat.com Mon Nov 28 17:28:40 2011 From: sbose at redhat.com (Sumit Bose) Date: Mon, 28 Nov 2011 18:28:40 +0100 Subject: [Freeipa-devel] [PATCH] Make pwd-extop aware of new ipaNTHash attribute Message-ID: <20111128172840.GF2246@localhost.localdomain> Hi, in IPAv3 we introduce a new attribute 'ipaNTHash' to store the NT hash. Currently the plugin handling the change password extended operation only sets and updates 'sambaNTPassword'. This patch add support for the new attribute without removing the support for the old one. bye, Sumit -------------- next part -------------- From 68d66eba4e31a314242322471dbfe698f4493737 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Thu, 24 Nov 2011 18:38:38 +0100 Subject: [PATCH] Make pwd-extop aware of new ipaNTHash attribute --- .../ipa-pwd-extop/ipa_pwd_extop.c | 4 +- daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd.h | 7 ++- .../ipa-pwd-extop/ipapwd_common.c | 38 +++++++++++++--- .../ipa-pwd-extop/ipapwd_encoding.c | 22 ++++++++- .../ipa-pwd-extop/ipapwd_prepost.c | 47 +++++++++++++------ 5 files changed, 89 insertions(+), 29 deletions(-) diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c index 65c5834595f89aee8502347311f247be058c3416..82acc49dd0a48bea9b560b882966e996ae5c4775 100644 --- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c +++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c @@ -156,7 +156,7 @@ static int ipapwd_chpwop(Slapi_PBlock *pb, struct ipapwd_krbcfg *krbcfg) Slapi_Value *objectclass=NULL; char *attrlist[] = {"*", "passwordHistory", NULL }; struct ipapwd_data pwdata; - int is_krb, is_smb; + int is_krb, is_smb, is_ipant; char *principal = NULL; /* Get the ber value of the extended operation */ @@ -365,7 +365,7 @@ parse_req_done: } rc = ipapwd_entry_checks(pb, targetEntry, - &is_root, &is_krb, &is_smb, + &is_root, &is_krb, &is_smb, &is_ipant, SLAPI_USERPWD_ATTR, SLAPI_ACL_WRITE); if (rc) { goto free_and_return; diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd.h b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd.h index 787ed500a080674d4a8e1002468006b020eb1578..0edd2dcad580b25d108a762bd78271b3d8244bc5 100644 --- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd.h +++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd.h @@ -113,7 +113,7 @@ struct ipapwd_krbcfg { }; int ipapwd_entry_checks(Slapi_PBlock *pb, struct slapi_entry *e, - int *is_root, int *is_krb, int *is_smb, + int *is_root, int *is_krb, int *is_smb, int *is_ipant, char *attr, int access); int ipapwd_gen_checks(Slapi_PBlock *pb, char **errMesg, struct ipapwd_krbcfg **config, int check_flags); @@ -144,8 +144,9 @@ void ipapwd_keyset_free(struct ipapwd_keyset **pkset); int ipapwd_gen_hashes(struct ipapwd_krbcfg *krbcfg, struct ipapwd_data *data, char *userpw, - int is_krb, int is_smb, Slapi_Value ***svals, - char **nthash, char **lmhash, char **errMesg); + int is_krb, int is_smb, int is_ipant, + Slapi_Value ***svals, char **nthash, char **lmhash, + Slapi_Value ***ntvals, char **errMesg); /* from ipapwd_prepost.c */ int ipapwd_ext_init(void); diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_common.c b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_common.c index 9e203be2763b13328e2d392c76e8545ba7ab549a..c36189987f785de8e8e97737554b854539b83ea2 100644 --- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_common.c +++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_common.c @@ -498,7 +498,7 @@ done: /*==Common-public-functions=============================================*/ int ipapwd_entry_checks(Slapi_PBlock *pb, struct slapi_entry *e, - int *is_root, int *is_krb, int *is_smb, + int *is_root, int *is_krb, int *is_smb, int *is_ipant, char *attr, int acc) { Slapi_Value *sval; @@ -535,6 +535,15 @@ int ipapwd_entry_checks(Slapi_PBlock *pb, struct slapi_entry *e, *is_smb = slapi_entry_attr_has_syntax_value(e, SLAPI_ATTR_OBJECTCLASS, sval); slapi_value_free(&sval); + sval = slapi_value_new_string("ipaNTUserAttrs"); + if (!sval) { + rc = LDAP_OPERATIONS_ERROR; + goto done; + } + *is_ipant = slapi_entry_attr_has_syntax_value(e, SLAPI_ATTR_OBJECTCLASS, + sval); + slapi_value_free(&sval); + rc = LDAP_SUCCESS; done: @@ -765,14 +774,17 @@ int ipapwd_SetPassword(struct ipapwd_krbcfg *krbcfg, int ret = 0; Slapi_Mods *smods = NULL; Slapi_Value **svals = NULL; + Slapi_Value **ntvals = NULL; Slapi_Value **pwvals = NULL; struct tm utctime; char timestr[GENERALIZED_TIME_LENGTH+1]; char *lm = NULL; char *nt = NULL; int is_smb = 0; + int is_ipant = 0; int is_host = 0; Slapi_Value *sambaSamAccount; + Slapi_Value *ipaNTUserAttrs; Slapi_Value *ipaHost; char *errMesg = NULL; char *modtime = NULL; @@ -782,10 +794,17 @@ int ipapwd_SetPassword(struct ipapwd_krbcfg *krbcfg, sambaSamAccount = slapi_value_new_string("sambaSamAccount"); if (slapi_entry_attr_has_syntax_value(data->target, "objectClass", sambaSamAccount)) { - is_smb = 1;; + is_smb = 1; } slapi_value_free(&sambaSamAccount); + ipaNTUserAttrs = slapi_value_new_string("ipaNTUserAttrs"); + if (slapi_entry_attr_has_syntax_value(data->target, + "objectClass", ipaNTUserAttrs)) { + is_ipant = 1; + } + slapi_value_free(&ipaNTUserAttrs); + ipaHost = slapi_value_new_string("ipaHost"); if (slapi_entry_attr_has_syntax_value(data->target, "objectClass", ipaHost)) { @@ -795,8 +814,8 @@ int ipapwd_SetPassword(struct ipapwd_krbcfg *krbcfg, ret = ipapwd_gen_hashes(krbcfg, data, data->password, - is_krb, is_smb, - &svals, &nt, &lm, &errMesg); + is_krb, is_smb, is_ipant, + &svals, &nt, &lm, &ntvals, &errMesg); if (ret) { goto free_and_return; } @@ -835,15 +854,21 @@ int ipapwd_SetPassword(struct ipapwd_krbcfg *krbcfg, } } - if (lm) { + if (lm && is_smb) { slapi_mods_add_string(smods, LDAP_MOD_REPLACE, "sambaLMPassword", lm); } - if (nt) { + if (nt && is_smb) { slapi_mods_add_string(smods, LDAP_MOD_REPLACE, "sambaNTPassword", nt); } + + if (ntvals && is_ipant) { + slapi_mods_add_mod_values(smods, LDAP_MOD_REPLACE, + "ipaNTHash", ntvals); + } + if (is_smb) { /* with samba integration we need to also set sambaPwdLastSet or * samba will decide the user has to change the password again */ @@ -899,6 +924,7 @@ free_and_return: if (modtime) slapi_ch_free((void **)&modtime); slapi_mods_free(&smods); ipapwd_free_slapi_value_array(&svals); + ipapwd_free_slapi_value_array(&ntvals); ipapwd_free_slapi_value_array(&pwvals); return ret; diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_encoding.c b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_encoding.c index 8e59e19c636299c68415f0e5f346dd47cdf5aaae..5720d3e6e0a7e6b1520e51a5ee319bcc47354fc9 100644 --- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_encoding.c +++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_encoding.c @@ -395,8 +395,9 @@ done: int ipapwd_gen_hashes(struct ipapwd_krbcfg *krbcfg, struct ipapwd_data *data, char *userpw, - int is_krb, int is_smb, Slapi_Value ***svals, - char **nthash, char **lmhash, char **errMesg) + int is_krb, int is_smb, int is_ipant, Slapi_Value ***svals, + char **nthash, char **lmhash, Slapi_Value ***ntvals, + char **errMesg) { int rc; @@ -417,7 +418,7 @@ int ipapwd_gen_hashes(struct ipapwd_krbcfg *krbcfg, } } - if (is_smb) { + if (is_smb || is_ipant) { char lm[33], nt[33]; struct ntlm_keys ntlm; int ret; @@ -442,6 +443,20 @@ int ipapwd_gen_hashes(struct ipapwd_krbcfg *krbcfg, nt[32] = '\0'; *nthash = slapi_ch_strdup(nt); } + + if (is_ipant) { + *ntvals = (Slapi_Value **)calloc(2, sizeof(Slapi_Value *)); + if (!svals) { + LOG_OOM(); + rc = LDAP_OPERATIONS_ERROR; + goto done; + } + (*ntvals)[0] = slapi_value_new(); + if (slapi_value_set((*ntvals)[0], ntlm.nt, 16) == NULL) { + rc = LDAP_OPERATIONS_ERROR; + goto done; + } + } } rc = LDAP_SUCCESS; @@ -451,6 +466,7 @@ done: /* when error, free possibly allocated output parameters */ if (rc) { ipapwd_free_slapi_value_array(svals); + ipapwd_free_slapi_value_array(ntvals); } return rc; diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c index 961678acb4f7a3265b5b221acedb96a63cea5855..a4663c0cce3722da4920fedca5d994966bc6d47d 100644 --- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c +++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c @@ -163,7 +163,7 @@ static int ipapwd_pre_add(Slapi_PBlock *pb) char *dn = NULL; struct ipapwd_operation *pwdop = NULL; void *op; - int is_repl_op, is_root, is_krb, is_smb; + int is_repl_op, is_root, is_krb, is_smb, is_ipant; int ret; int rc = LDAP_SUCCESS; @@ -240,7 +240,7 @@ static int ipapwd_pre_add(Slapi_PBlock *pb) } rc = ipapwd_entry_checks(pb, e, - &is_root, &is_krb, &is_smb, + &is_root, &is_krb, &is_smb, &is_ipant, NULL, SLAPI_ACL_ADD); if (rc != LDAP_SUCCESS) { goto done; @@ -307,17 +307,18 @@ static int ipapwd_pre_add(Slapi_PBlock *pb) goto done; } - if (is_krb || is_smb) { + if (is_krb || is_smb || is_ipant) { Slapi_Value **svals = NULL; + Slapi_Value **ntvals = NULL; char *nt = NULL; char *lm = NULL; pwdop->is_krb = is_krb; rc = ipapwd_gen_hashes(krbcfg, &pwdop->pwdata, - userpw, is_krb, is_smb, - &svals, &nt, &lm, &errMesg); + userpw, is_krb, is_smb, is_ipant, + &svals, &nt, &lm, &ntvals, &errMesg); if (rc != LDAP_SUCCESS) { goto done; } @@ -335,17 +336,22 @@ static int ipapwd_pre_add(Slapi_PBlock *pb) ipapwd_free_slapi_value_array(&svals); } - if (lm) { + if (lm && is_smb) { /* set value */ slapi_entry_attr_set_charptr(e, "sambaLMPassword", lm); slapi_ch_free_string(&lm); } - if (nt) { + if (nt && is_smb) { /* set value */ slapi_entry_attr_set_charptr(e, "sambaNTPassword", nt); slapi_ch_free_string(&nt); } + if (ntvals && is_ipant) { + slapi_entry_attr_replace_sv(e, "ipaNTHash", ntvals); + ipapwd_free_slapi_value_array(&ntvals); + } + if (is_smb) { /* with samba integration we need to also set sambaPwdLastSet or * samba will decide the user has to change the password again */ @@ -397,7 +403,7 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) struct slapi_entry *e = NULL; struct ipapwd_operation *pwdop = NULL; void *op; - int is_repl_op, is_pwd_op, is_root, is_krb, is_smb; + int is_repl_op, is_pwd_op, is_root, is_krb, is_smb, is_ipant; int has_krb_keys = 0; int has_history = 0; int gen_krb_keys = 0; @@ -514,7 +520,7 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) } rc = ipapwd_entry_checks(pb, e, - &is_root, &is_krb, &is_smb, + &is_root, &is_krb, &is_smb, &is_ipant, SLAPI_USERPWD_ATTR, SLAPI_ACL_WRITE); if (rc) { goto done; @@ -585,6 +591,7 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) * flags, so we sero them out and see if they get set again */ is_krb = 0; is_smb = 0; + is_ipant = 0; case LDAP_MOD_ADD: bv = slapi_mod_get_first_value(smod); @@ -598,6 +605,8 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) is_krb = 1; if (0 == strncasecmp("sambaSamAccount", bv->bv_val, bv->bv_len)) is_smb = 1; + if (0 == strncasecmp("ipaNTUserAttrs", bv->bv_val, bv->bv_len)) + is_ipant = 1; } while ((bv = slapi_mod_get_next_value(smod)) != NULL); break; @@ -606,6 +615,7 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) /* can this happen for objectclasses ? */ is_krb = 0; is_smb = 0; + is_ipant = 0; default: break; @@ -654,7 +664,7 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) /* Check this is a clear text password, or refuse operation (only if we need * to comput other hashes */ - if (! unhashedpw && (gen_krb_keys || is_smb)) { + if (! unhashedpw && (gen_krb_keys || is_smb || is_ipant)) { if ('{' == userpw[0]) { if (0 == strncasecmp(userpw, "{CLEAR}", strlen("{CLEAR}"))) { unhashedpw = slapi_ch_strdup(&userpw[strlen("{CLEAR}")]); @@ -746,15 +756,16 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) } } - if (gen_krb_keys || is_smb) { + if (gen_krb_keys || is_smb || is_ipant) { Slapi_Value **svals = NULL; + Slapi_Value **ntvals = NULL; char *nt = NULL; char *lm = NULL; rc = ipapwd_gen_hashes(krbcfg, &pwdop->pwdata, unhashedpw, - gen_krb_keys, is_smb, - &svals, &nt, &lm, &errMesg); + gen_krb_keys, is_smb, is_ipant, + &svals, &nt, &lm, &ntvals, &errMesg); if (rc) { goto done; } @@ -766,19 +777,25 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) ipapwd_free_slapi_value_array(&svals); } - if (lm) { + if (lm && is_smb) { /* replace value */ slapi_mods_add_string(smods, LDAP_MOD_REPLACE, "sambaLMPassword", lm); slapi_ch_free_string(&lm); } - if (nt) { + if (nt && is_smb) { /* replace value */ slapi_mods_add_string(smods, LDAP_MOD_REPLACE, "sambaNTPassword", nt); slapi_ch_free_string(&nt); } + if (ntvals && is_ipant) { + slapi_mods_add_mod_values(smods, LDAP_MOD_REPLACE, + "ipaNTHash", ntvals); + ipapwd_free_slapi_value_array(&ntvals); + } + if (is_smb) { /* with samba integration we need to also set sambaPwdLastSet or * samba will decide the user has to change the password again */ -- 1.7.6 From rcritten at redhat.com Mon Nov 28 18:09:46 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 28 Nov 2011 13:09:46 -0500 Subject: [Freeipa-devel] [PATCH] 908 make some fields required Message-ID: <4ED3CE6A.9000804@redhat.com> Some attributes in the framework were not marked as required even though they are in the schema. These are typically computed values and I think the intention was to not prompt for them. This has the side-effect of them showing as not required in the UI even though they are. Since they all have default values set they won't be prompted for on the CLI so there won't be any practical changes. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-908-required.patch Type: text/x-patch Size: 8360 bytes Desc: not available URL: From ryan at pet.ubc.ca Mon Nov 28 21:11:17 2011 From: ryan at pet.ubc.ca (Ryan Thomson) Date: Mon, 28 Nov 2011 13:11:17 -0800 Subject: [Freeipa-devel] Activation and password reset webapp UI Message-ID: <4ED3F8F5.8070402@pet.ubc.ca> Hello, Attached is my amateur attempt at contributing to FreeIPA. The patch implements an account "activation" and password reset webapp UI to cover use cases where FreeIPA may be acting only as a backend to services such as Samba or other web application that do no expose a method for changing expired kerberos credentials. This code is based on the "migration" webapp UI. See ticket 1907. https://fedorahosted.org/freeipa/ticket/1907 Thanks, -- Ryan Thomson Systems Administrator, UBC PET 2211 Wesbrook Mall Rm G358 UBC Hospital, Koerner Pavilion Vancouver, BC V6T 2B5 Office: 604.822.7605 Mobile: 778.319.4505 -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freeipa-rthomson-0001-Activation-webapp-UI.patch URL: From edewata at redhat.com Mon Nov 28 22:52:50 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 28 Nov 2011 16:52:50 -0600 Subject: [Freeipa-devel] [PATCH] 908 make some fields required In-Reply-To: <4ED3CE6A.9000804@redhat.com> References: <4ED3CE6A.9000804@redhat.com> Message-ID: <4ED410C2.7000905@redhat.com> On 11/28/2011 12:09 PM, Rob Crittenden wrote: > Some attributes in the framework were not marked as required even though > they are in the schema. These are typically computed values and I think > the intention was to not prompt for them. This has the side-effect of > them showing as not required in the UI even though they are. > > Since they all have default values set they won't be prompted for on the > CLI so there won't be any practical changes. This patch fixes the problem with required attributes in DNS Zones and cn, uidNumber, and gidNumber in Users. The UI now shows the required indicators for these attributes. So this patch is ACKed. Some problems mentioned in ticket #2015 are still present: 1. Removing the homeDirectory from a user fails because it's required by posixAccount. 2. Removing the gidNumber from a group fails because it's required by posixGroup. 3. Removing config attributes listed in the ticket generates internal error. I think at least the server should return a proper error message. The required indicator can be hard-coded in the UI if necessary. -- Endi S. Dewata From rcritten at redhat.com Mon Nov 28 23:16:09 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 28 Nov 2011 18:16:09 -0500 Subject: [Freeipa-devel] [PATCH] 152 Enable automember for upgraded servers In-Reply-To: <4EB45E19.6060606@redhat.com> References: <1318854040.26757.8.camel@balmora.brq.redhat.com> <4EB452F3.1050806@redhat.com> <4EB45363.6070303@redhat.com> <1320441997.2811.3.camel@priserak> <4EB45AB3.6010305@redhat.com> <4EB45E19.6060606@redhat.com> Message-ID: <4ED41639.3090408@redhat.com> Nathan Kinder wrote: > On 11/04/2011 02:35 PM, Nathan Kinder wrote: >> On 11/04/2011 02:26 PM, Martin Kosek wrote: >>> On Fri, 2011-11-04 at 14:04 -0700, Nathan Kinder wrote: >>>> On 11/04/2011 02:02 PM, Rob Crittenden wrote: >>>>> Martin Kosek wrote: >>>>>> automember functionality is depends on predefined data is in LDAP. >>>>>> Since we add it for fresh installs only, automember cannot be used >>>>>> for upgraded servers. Make sure that automember LDAP data is added >>>>>> during upgrade too. >>>>>> >>>>>> https://fedorahosted.org/freeipa/ticket/1992 >>>>> I think you need that automember schema as well. Can you check with >>>>> the 389-ds team to see if their upgrade script automatically adds new >>>>> schema or if we have to handle that ourselves? >>>> The new automember schema should be added by 'setup-ds.pl -u', so I >>>> don't expect you need to do anything around schema in FreeIPA. >>> Nathan, when is the "setup-ds.pl -u" executed? When the dirsrv rpm is >>> updated, just like FreeIPA runs ipa-ldap-updater in rpm update %post? Or >>> does it have to be run manually? >> It is run in the the %posttrans stage for 389-ds-base. >>> I am asking because the schema problem seems like the root cause that >>> one user has here (the last post): >>> >>> https://bugzilla.redhat.com/show_bug.cgi?id=746589 >> There should be a >> '/etc/dirsrv/slapd-/schema/10automember-plugin.ldif' file if >> the proper version >> of 389-ds-base is being used and if 'setup-ds.pl -u' successfully >> updated the schema. There should also be >> a '/etc/dirsrv/schema/10automember-plugin.ldif' file present >> regardless of 'setup-ds.pl -u' having run >> successfully. > I just tested running 'setup-ds.pl -u' manually with a master build of > 389-ds-base, and there is a bug that is preventing the updates from > being applied. I logged the following bug for this: > > https://bugzilla.redhat.com/show_bug.cgi?id=751495 > > The fix is a one-liner, and I believe Rich is working on getting a fixed > build out ASAP. ACK, works for me. rob From simo at redhat.com Tue Nov 29 00:43:57 2011 From: simo at redhat.com (Simo Sorce) Date: Mon, 28 Nov 2011 19:43:57 -0500 Subject: [Freeipa-devel] [PATCH] #2122 Fix PAC re-signing In-Reply-To: <20111124125419.GK2205@localhost.localdomain> References: <1322007054.6750.76.camel@willson.li.ssimo.org> <20111123105311.GG2205@localhost.localdomain> <20111124125419.GK2205@localhost.localdomain> Message-ID: <1322527437.2613.42.camel@willson.li.ssimo.org> On Thu, 2011-11-24 at 13:54 +0100, Sumit Bose wrote: > I think I found two issues which should be fixed by the following > patch: > - krb5_pac_add_buffer() expects krb5_pac and not krb5_pac * as a > second > argument good catch > - your patch copies all buffers, including the checksums, which you > wanted to remove from the new pac also good catch > With this patch applied I do not see any errors in the krb5kdc.log and > ssh from AD to IPA server works. I still haven't solved my ssh issue from an AD client to IPA, but I get a ticket on the client now, so it must be unrelated stuff. I have prepared a patch which have a slightly different version of your fixes. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-ipa-kdb-Support-re-signing-PAC-with-different-checks-2.patch Type: text/x-patch Size: 3180 bytes Desc: not available URL: From rcritten at redhat.com Tue Nov 29 04:24:02 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 28 Nov 2011 23:24:02 -0500 Subject: [Freeipa-devel] [PATCH] 157 Add --delattr option to complement --setattr/--addattr In-Reply-To: <1322064584.27504.28.camel@balmora.brq.redhat.com> References: <1319714558.26374.13.camel@balmora.brq.redhat.com> <4EB3E74A.1000905@redhat.com> <1320497005.25589.5.camel@priserak> <1322064584.27504.28.camel@balmora.brq.redhat.com> Message-ID: <4ED45E62.2040609@redhat.com> Martin Kosek wrote: > On Sat, 2011-11-05 at 13:43 +0100, Martin Kosek wrote: >> On Fri, 2011-11-04 at 09:23 -0400, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> Add a --delattr option to round out multi-valued attribute manipulation. >>>> The new option is be available for all LDAPUpdate based commands. >>>> >>>> --delattr is evaluated last, it can remove any value present either >>>> in --addattr/--setattr options or stored in LDAP. >>>> >>>> https://fedorahosted.org/freeipa/ticket/1929 >>> >>> Should --delattr raise an error if the value doesn't exist? >>> >>> I think it probably should. >>> >>> rob >> >> You are right, it would be more expected behavior. I fixed that. In the >> process of implementing the change I found that current --*attr >> processing is not good, so I refactored it completely to one function >> available for all baseldap.py commands. >> >> In the process I found out that we don't have any common class for all >> baseldap.py commands and the result is BaseLDAPCommand which can now >> handle attribute processing and provide it to other LDAP commands. >> >> And I also fixed one group unit test. Now all unit tests were OK. >> >> Martin > > I rebased the patch (API.txt format was changed) and tested that it > still works ok. > > Martin ACK but some of the comments need to be cleaned up before pushing. It will also require a minor rebase in frontend.py. process_attr_options() should probably read: Process all --setattr, --addattr, and --delattr options and add the resulting value to the list of attributes. --setattr is processed first, then --addattr and finally --delattr. When --setattr is not used then the original LDAP object is looked up (of course, not when dn is None) and the changes are applied to old object values. ... AttrValueNotFound exception may be raised when an attribute value was not found either by --setattr and --addattr nor in existing LDAP object. rob From rcritten at redhat.com Tue Nov 29 04:34:39 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 28 Nov 2011 23:34:39 -0500 Subject: [Freeipa-devel] [PATCH] 161 Make ipa-server-install clean after itself In-Reply-To: <1320765254.2629.9.camel@balmora.brq.redhat.com> References: <1320765254.2629.9.camel@balmora.brq.redhat.com> Message-ID: <4ED460DF.2090507@redhat.com> Martin Kosek wrote: > How to test: > > 1) ipa-server-install -p secret123 -a secret123 --hostname > ipa.example.com > 2) Continue in interactive wizard until IP address is requested (as > ipa.example.com cannot be resolved) > 3) When it is entered and ipa-server-install gives this output: > > # ipa-server-install -p kokos123 -a kokos123 --hostname ipa.example.com > --setup-dns > ... > Please confirm the domain name [example.com]: > > Unable to resolve IP address for host name > Please provide the IP address to be used for this host name: 10.16.78.93 > Adding [10.16.78.93 ipa.example.com] to your /etc/hosts file<<<<< > ... > > hit CTRL+C to quit from the installation. > 4) Try running ipa-server-install again. It will fail as /etc/hosts has > been changed - this patch fixes it. ACK. Needs a rebase. rob From mkosek at redhat.com Tue Nov 29 08:04:20 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 Nov 2011 09:04:20 +0100 Subject: [Freeipa-devel] [PATCH] 152 Enable automember for upgraded servers In-Reply-To: <4ED41639.3090408@redhat.com> References: <1318854040.26757.8.camel@balmora.brq.redhat.com> <4EB452F3.1050806@redhat.com> <4EB45363.6070303@redhat.com> <1320441997.2811.3.camel@priserak> <4EB45AB3.6010305@redhat.com> <4EB45E19.6060606@redhat.com> <4ED41639.3090408@redhat.com> Message-ID: <1322553860.32073.0.camel@balmora.brq.redhat.com> On Mon, 2011-11-28 at 18:16 -0500, Rob Crittenden wrote: > Nathan Kinder wrote: > > On 11/04/2011 02:35 PM, Nathan Kinder wrote: > >> On 11/04/2011 02:26 PM, Martin Kosek wrote: > >>> On Fri, 2011-11-04 at 14:04 -0700, Nathan Kinder wrote: > >>>> On 11/04/2011 02:02 PM, Rob Crittenden wrote: > >>>>> Martin Kosek wrote: > >>>>>> automember functionality is depends on predefined data is in LDAP. > >>>>>> Since we add it for fresh installs only, automember cannot be used > >>>>>> for upgraded servers. Make sure that automember LDAP data is added > >>>>>> during upgrade too. > >>>>>> > >>>>>> https://fedorahosted.org/freeipa/ticket/1992 > >>>>> I think you need that automember schema as well. Can you check with > >>>>> the 389-ds team to see if their upgrade script automatically adds new > >>>>> schema or if we have to handle that ourselves? > >>>> The new automember schema should be added by 'setup-ds.pl -u', so I > >>>> don't expect you need to do anything around schema in FreeIPA. > >>> Nathan, when is the "setup-ds.pl -u" executed? When the dirsrv rpm is > >>> updated, just like FreeIPA runs ipa-ldap-updater in rpm update %post? Or > >>> does it have to be run manually? > >> It is run in the the %posttrans stage for 389-ds-base. > >>> I am asking because the schema problem seems like the root cause that > >>> one user has here (the last post): > >>> > >>> https://bugzilla.redhat.com/show_bug.cgi?id=746589 > >> There should be a > >> '/etc/dirsrv/slapd-/schema/10automember-plugin.ldif' file if > >> the proper version > >> of 389-ds-base is being used and if 'setup-ds.pl -u' successfully > >> updated the schema. There should also be > >> a '/etc/dirsrv/schema/10automember-plugin.ldif' file present > >> regardless of 'setup-ds.pl -u' having run > >> successfully. > > I just tested running 'setup-ds.pl -u' manually with a master build of > > 389-ds-base, and there is a bug that is preventing the updates from > > being applied. I logged the following bug for this: > > > > https://bugzilla.redhat.com/show_bug.cgi?id=751495 > > > > The fix is a one-liner, and I believe Rich is working on getting a fixed > > build out ASAP. > > ACK, works for me. > > rob Pushed to master, ipa-2-1. Martin From mkosek at redhat.com Tue Nov 29 08:26:34 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 Nov 2011 09:26:34 +0100 Subject: [Freeipa-devel] [PATCH] 161 Make ipa-server-install clean after itself In-Reply-To: <4ED460DF.2090507@redhat.com> References: <1320765254.2629.9.camel@balmora.brq.redhat.com> <4ED460DF.2090507@redhat.com> Message-ID: <1322555194.32073.1.camel@balmora.brq.redhat.com> On Mon, 2011-11-28 at 23:34 -0500, Rob Crittenden wrote: > Martin Kosek wrote: > > How to test: > > > > 1) ipa-server-install -p secret123 -a secret123 --hostname > > ipa.example.com > > 2) Continue in interactive wizard until IP address is requested (as > > ipa.example.com cannot be resolved) > > 3) When it is entered and ipa-server-install gives this output: > > > > # ipa-server-install -p kokos123 -a kokos123 --hostname ipa.example.com > > --setup-dns > > ... > > Please confirm the domain name [example.com]: > > > > Unable to resolve IP address for host name > > Please provide the IP address to be used for this host name: 10.16.78.93 > > Adding [10.16.78.93 ipa.example.com] to your /etc/hosts file<<<<< > > ... > > > > hit CTRL+C to quit from the installation. > > 4) Try running ipa-server-install again. It will fail as /etc/hosts has > > been changed - this patch fixes it. > > ACK. Needs a rebase. > > rob Rebased for master branch. Pushed to master, ipa-2-1. Martin From mkosek at redhat.com Tue Nov 29 09:11:33 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 Nov 2011 10:11:33 +0100 Subject: [Freeipa-devel] [PATCH] 157 Add --delattr option to complement --setattr/--addattr In-Reply-To: <4ED45E62.2040609@redhat.com> References: <1319714558.26374.13.camel@balmora.brq.redhat.com> <4EB3E74A.1000905@redhat.com> <1320497005.25589.5.camel@priserak> <1322064584.27504.28.camel@balmora.brq.redhat.com> <4ED45E62.2040609@redhat.com> Message-ID: <1322557893.32073.7.camel@balmora.brq.redhat.com> On Mon, 2011-11-28 at 23:24 -0500, Rob Crittenden wrote: > Martin Kosek wrote: > > On Sat, 2011-11-05 at 13:43 +0100, Martin Kosek wrote: > >> On Fri, 2011-11-04 at 09:23 -0400, Rob Crittenden wrote: > >>> Martin Kosek wrote: > >>>> Add a --delattr option to round out multi-valued attribute manipulation. > >>>> The new option is be available for all LDAPUpdate based commands. > >>>> > >>>> --delattr is evaluated last, it can remove any value present either > >>>> in --addattr/--setattr options or stored in LDAP. > >>>> > >>>> https://fedorahosted.org/freeipa/ticket/1929 > >>> > >>> Should --delattr raise an error if the value doesn't exist? > >>> > >>> I think it probably should. > >>> > >>> rob > >> > >> You are right, it would be more expected behavior. I fixed that. In the > >> process of implementing the change I found that current --*attr > >> processing is not good, so I refactored it completely to one function > >> available for all baseldap.py commands. > >> > >> In the process I found out that we don't have any common class for all > >> baseldap.py commands and the result is BaseLDAPCommand which can now > >> handle attribute processing and provide it to other LDAP commands. > >> > >> And I also fixed one group unit test. Now all unit tests were OK. > >> > >> Martin > > > > I rebased the patch (API.txt format was changed) and tested that it > > still works ok. > > > > Martin > > ACK but some of the comments need to be cleaned up before pushing. It > will also require a minor rebase in frontend.py. > > process_attr_options() should probably read: > > Process all --setattr, --addattr, and --delattr options and add the > resulting value to the list of attributes. --setattr is processed first, > then --addattr and finally --delattr. > > When --setattr is not used then the original LDAP object is looked up > (of course, not when dn is None) and the changes are applied to old > object values. > > ... > AttrValueNotFound exception may be raised when an attribute value was > not found either by --setattr and --addattr nor in existing LDAP object. > > rob I fixed the comment, it is now much more readable. I see you improved __convert_2_dict function (which is moved to baseldap.py in this patch) in your update plugin framework patch. This has been properly merged. Pushed to master. Martin From sbose at redhat.com Tue Nov 29 09:13:11 2011 From: sbose at redhat.com (Sumit Bose) Date: Tue, 29 Nov 2011 10:13:11 +0100 Subject: [Freeipa-devel] [PATCH] #2122 Fix PAC re-signing In-Reply-To: <1322527437.2613.42.camel@willson.li.ssimo.org> References: <1322007054.6750.76.camel@willson.li.ssimo.org> <20111123105311.GG2205@localhost.localdomain> <20111124125419.GK2205@localhost.localdomain> <1322527437.2613.42.camel@willson.li.ssimo.org> Message-ID: <20111129091311.GG2246@localhost.localdomain> On Mon, Nov 28, 2011 at 07:43:57PM -0500, Simo Sorce wrote: > On Thu, 2011-11-24 at 13:54 +0100, Sumit Bose wrote: > > I think I found two issues which should be fixed by the following > > patch: > > - krb5_pac_add_buffer() expects krb5_pac and not krb5_pac * as a > > second > > argument > > good catch > > > - your patch copies all buffers, including the checksums, which you > > wanted to remove from the new pac > > also good catch > > > With this patch applied I do not see any errors in the krb5kdc.log and > > ssh from AD to IPA server works. > > I still haven't solved my ssh issue from an AD client to IPA, but I get > a ticket on the client now, so it must be unrelated stuff. > > I have prepared a patch which have a slightly different version of your > fixes. The patch looks fine and works for me. ACK bye, Sumit > > Simo. > > -- > Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Tue Nov 29 09:31:22 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 Nov 2011 10:31:22 +0100 Subject: [Freeipa-devel] [PATCH] 5 User-add random password support In-Reply-To: <4ECE760A.7000401@redhat.com> References: <4ECE5ABE.3060207@redhat.com> <4ECE760A.7000401@redhat.com> Message-ID: <1322559082.32073.13.camel@balmora.brq.redhat.com> On Thu, 2011-11-24 at 17:51 +0100, Ondrej Hamada wrote: > On 11/24/2011 03:54 PM, Ondrej Hamada wrote: > > https://fedorahosted.org/freeipa/ticket/1979 > > > > I've used code from ipalib/plugins/host.py to add support for > > random > > password generation. The '--random' option is now available in > > user-add and user-mod commands. If both the 'password' and 'random' > > options are used the 'random' option will be ignored. > > > > Functionally, it works OK. I would just like to propose few improvements: 1) Minor API version in VERSION file should be bumped since you add a new option 2) We should add some tests exercising this new functionality so that we can detect regressions early 3) (optional) I am thinking if the passwords we generate are not very user friendly. I would love to see user's face when he is told that his new password is 5QU;8l2%]y"? . While this is may be OK for hosts bulk passwords which are only manipulated by admins, we may want to develop more user friendly passwords in the user plugin. Martin From mkosek at redhat.com Tue Nov 29 09:44:40 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 Nov 2011 10:44:40 +0100 Subject: [Freeipa-devel] [PATCH] 0037 make-lint fails on Fedora 16/Rawhide In-Reply-To: <20111128152609.GA18195@redhat.com> References: <20111128122851.GB4773@redhat.com> <20111128152609.GA18195@redhat.com> Message-ID: <1322559880.32073.17.camel@balmora.brq.redhat.com> On Mon, 2011-11-28 at 17:26 +0200, Alexander Bokovoy wrote: > On Mon, 28 Nov 2011, Alexander Bokovoy wrote: > > > Hi, > > > > Attached are fixes for ldap.LDAPObject.add_s(self, dn, modlist) uses > > which now don't pass 'make-lint' on Fedora 16/Rawhide. > > > > -- > > / Alexander Bokovoy > > > >From dd866262c98be779a094a617975145e2fb1e0dd1 Mon Sep 17 00:00:00 2001 > > From: Alexander Bokovoy > > Date: Mon, 28 Nov 2011 14:21:17 +0200 > > Subject: [PATCH] Be more explicit when passing Entry class to > > ldap.LDAPObject.add_s() > > > > ldap.LDAPObject.add_s(self, dn, modlist) requires two positional arguments. > > We used to pass our Entry class which implements dictionary access that gives > > proper way to pass the positional arguments, but PyLint in Fedora 16/Rawhide > > became more strict about that and can't infer dictionary interface through > > static checking. > > > > Thus, we need to explicitly annotate dictionary passing with **entry syntax. > > This has additional benefit to remind that we deal with multiple arguments here. > Self-NACK, doesn't really work this way. I'm still looking at better > approach but intermediate solution is to use pylint hints. > > Conservative patch is attached. > What about other add_s(entry) calls? I see we call it this way on more places, especially in replication.py: $ git grep "add_s(e" ipaserver/install/cainstance.py: ld.add_s(entry_dn, entry) ipaserver/install/krbinstance.py: self.admin_conn.add_s(entry) #pylint: disable=E1120 ipaserver/install/krbinstance.py: self.admin_conn.add_s(entry) #pylint: disable=E1120 ipaserver/install/replication.py: conn.add_s(ent) ipaserver/install/replication.py: conn.add_s(entry) ipaserver/install/replication.py: conn.add_s(entry) ipaserver/install/replication.py: self.conn.add_s(entry) ipaserver/install/replication.py: conn.add_s(entry) ipaserver/install/replication.py: a_conn.add_s(entry) ipaserver/install/replication.py: self.conn.add_s(entry) ipaserver/install/service.py: conn.add_s(entry) #pylint: disable=E1120 Should we patch ipa-2-1 branch as well? If we do another release for F-16 we want to have pylint check clean. We would need a rebased patch for ipa-2-1 branch in this case. Martin From abokovoy at redhat.com Tue Nov 29 09:52:25 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 29 Nov 2011 11:52:25 +0200 Subject: [Freeipa-devel] [PATCH] 0037 make-lint fails on Fedora 16/Rawhide In-Reply-To: <1322559880.32073.17.camel@balmora.brq.redhat.com> References: <20111128122851.GB4773@redhat.com> <20111128152609.GA18195@redhat.com> <1322559880.32073.17.camel@balmora.brq.redhat.com> Message-ID: <20111129095224.GA5725@redhat.com> On Tue, 29 Nov 2011, Martin Kosek wrote: > > Conservative patch is attached. > > > > What about other add_s(entry) calls? I see we call it this way on more > places, especially in replication.py: > > $ git grep "add_s(e" > ipaserver/install/cainstance.py: ld.add_s(entry_dn, entry) > ipaserver/install/krbinstance.py: self.admin_conn.add_s(entry) #pylint: disable=E1120 > ipaserver/install/krbinstance.py: self.admin_conn.add_s(entry) #pylint: disable=E1120 > ipaserver/install/replication.py: conn.add_s(ent) > ipaserver/install/replication.py: conn.add_s(entry) > ipaserver/install/replication.py: conn.add_s(entry) > ipaserver/install/replication.py: self.conn.add_s(entry) > ipaserver/install/replication.py: conn.add_s(entry) > ipaserver/install/replication.py: a_conn.add_s(entry) > ipaserver/install/replication.py: self.conn.add_s(entry) > ipaserver/install/service.py: conn.add_s(entry) #pylint: disable=E1120 > > Should we patch ipa-2-1 branch as well? If we do another release for > F-16 we want to have pylint check clean. We would need a rebased patch > for ipa-2-1 branch in this case. I think this patch also can go away, we found out with Rob yesterday night that John has addressed these issues in his patches 48 and 49 in September. The patches were ACKed but not merged. I'm doing their testing now and will push them after that. Look at John's 48/49 patches in meanwhile. -- / Alexander Bokovoy From abokovoy at redhat.com Tue Nov 29 11:54:41 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 29 Nov 2011 13:54:41 +0200 Subject: [Freeipa-devel] [PATCH] 0037 make-lint fails on Fedora 16/Rawhide In-Reply-To: <20111129095224.GA5725@redhat.com> References: <20111128122851.GB4773@redhat.com> <20111128152609.GA18195@redhat.com> <1322559880.32073.17.camel@balmora.brq.redhat.com> <20111129095224.GA5725@redhat.com> Message-ID: <20111129115439.GB5725@redhat.com> On Tue, 29 Nov 2011, Alexander Bokovoy wrote: > On Tue, 29 Nov 2011, Martin Kosek wrote: > > > Conservative patch is attached. > > > > > > > What about other add_s(entry) calls? I see we call it this way on more > > places, especially in replication.py: > > > > $ git grep "add_s(e" > > ipaserver/install/cainstance.py: ld.add_s(entry_dn, entry) > > ipaserver/install/krbinstance.py: self.admin_conn.add_s(entry) #pylint: disable=E1120 > > ipaserver/install/krbinstance.py: self.admin_conn.add_s(entry) #pylint: disable=E1120 > > ipaserver/install/replication.py: conn.add_s(ent) > > ipaserver/install/replication.py: conn.add_s(entry) > > ipaserver/install/replication.py: conn.add_s(entry) > > ipaserver/install/replication.py: self.conn.add_s(entry) > > ipaserver/install/replication.py: conn.add_s(entry) > > ipaserver/install/replication.py: a_conn.add_s(entry) > > ipaserver/install/replication.py: self.conn.add_s(entry) > > ipaserver/install/service.py: conn.add_s(entry) #pylint: disable=E1120 > > > > Should we patch ipa-2-1 branch as well? If we do another release for > > F-16 we want to have pylint check clean. We would need a rebased patch > > for ipa-2-1 branch in this case. > I think this patch also can go away, we found out with Rob yesterday > night that John has addressed these issues in his patches 48 and 49 in > September. The patches were ACKed but not merged. I'm doing their > testing now and will push them after that. > > Look at John's 48/49 patches in meanwhile. Ok, I've tested patches 48 and 49 and also made versions for ipa-2-1 branch. All attached. -- / Alexander Bokovoy -------------- next part -------------- >From dd2ab5e265da7eead7c8bd1f09551401d968e784 Mon Sep 17 00:00:00 2001 From: John Dennis Date: Mon, 26 Sep 2011 10:39:15 -0400 Subject: [PATCH 1/3] Ticket #1879 - IPAdmin undefined anonymous parameter lists The IPAdmin class in ipaserver/ipaldap.py has methods with anonymous undefined parameter lists. For example: def getList(self,*args): In Python syntax this means you can call getList with any positional parameter list you want. This is bad because: 1) It's not true, *args gets passed to an ldap function with a well defined parameter list, so you really do have to call it with a defined parameter list. *args will let you pass anything, but once it gets passed to the ldap function it will blow up if the parameters do not match (what parameters are those you're wondering? see item 2). 2) The programmer does not know what the valid parameters are unless they are defined in the formal parameter list. 3) Without a formal parameter list automatic documentation generators cannot produce API documentation (see item 2) 4) The Python interpreter cannot validate the parameters being passed because there is no formal parameter list. Note, Python does not validate the type of parameters, but it does validate the correct number of postitional parameters are passed and only defined keyword parameters are passed. Bypassing the language support facilities leads to programming errors. 5) Without a formal parameter list program checkers such as pylint cannot validate the program which leads to progamming errors. 6) Without a formal parameter list which includes default keyword parameters it's not possible to use keyword arguments nor to know what their default values are (see item 2). One is forced to pass a keyword argument as a positional argument, plus you must then pass every keyword argument between the end of the positional argument list and keyword arg of interest even of the other keyword arguments are not of interest. This also demands you know what the default value of the intermediate keyword arguments are (see item 2) and hope they don't change. Also the *args anonymous tuple get passed into the error handling code so it can report what the called values were. But because the tuple is anonymous the error handler cannot not describe what it was passed. In addition the error handling code makes assumptions about the possible contents of the anonymous tuple based on current practice instead of actual defined values. Things like "if the number of items in the tuple is 2 or less then the first tuple item must be a dn (Distinguished Name)" or "if the number of items in the tuple is greater than 2 then the 3rd item must be an ldap search filter". These are constructs which are not robust and will fail at some point in the future. This patch also fixes the use of IPAdmin.addEntry(). It was sometimes being called with (dn, modlist), sometimes a Entry object, or sometimes a Entity object. Now it's always called with either a Entry or Entity object and IPAdmin.addEntry() validates the type of the parameter passed. --- ipaserver/install/ldapupdate.py | 4 +- ipaserver/ipaldap.py | 94 ++++++++++++++++++--------------------- 2 files changed, 45 insertions(+), 53 deletions(-) diff --git a/ipaserver/install/ldapupdate.py b/ipaserver/install/ldapupdate.py index e1f6b1f43929ed77e1896e5fb37515e7ad70aabb..1b6a6a5f4d572100504c28ebc628569c6bce85a8 100644 --- a/ipaserver/install/ldapupdate.py +++ b/ipaserver/install/ldapupdate.py @@ -357,7 +357,7 @@ class LDAPUpdate: logging.debug("Task id: %s", dn) if self.live_run: - self.conn.addEntry(e.dn, e.toTupleList()) + self.conn.addEntry(e) return dn @@ -657,7 +657,7 @@ class LDAPUpdate: # addifexist may result in an entry with only a # dn defined. In that case there is nothing to do. # It means the entry doesn't exist, so skip it. - self.conn.addEntry(entry.dn, entry.toTupleList()) + self.conn.addEntry(entry) self.modified = True except Exception, e: logging.error("Add failure %s", e) diff --git a/ipaserver/ipaldap.py b/ipaserver/ipaldap.py index 74cfbfda911facbf6f3bddf5972b3f035a9cfde0..fdcbe62495805f76397e7588fe24041ce540b8c0 100644 --- a/ipaserver/ipaldap.py +++ b/ipaserver/ipaldap.py @@ -1,5 +1,6 @@ # Authors: Rich Megginson -# Rob Crittenden +# John Dennis # # Copyright (C) 2007 Red Hat # see file 'COPYING' for use and warranty information @@ -35,6 +36,7 @@ from ldap.ldapobject import SimpleLDAPObject from ipaserver import ipautil from ipalib import errors from ipapython.ipautil import format_netloc +from ipapython.entity import Entity # Global variable to define SASL auth SASL_AUTH = ldap.sasl.sasl({},'GSSAPI') @@ -263,7 +265,7 @@ class IPAdmin(SimpleLDAPObject): self.dbdir = os.path.dirname(ent.getValue('nsslapd-directory')) except ldap.LDAPError, e: - self.__handle_errors(e, **{}) + self.__handle_errors(e) def __str__(self): return self.host + ":" + str(self.port) @@ -304,8 +306,8 @@ class IPAdmin(SimpleLDAPObject): # re-raise the error so we can handle it raise e except ldap.NO_SUCH_OBJECT, e: - args = kw.get('args', ["entry not found"]) - raise errors.NotFound(reason=notfound(args)) + arg_desc = kw.get('arg_desc', "entry not found") + raise errors.NotFound(reason=arg_desc) except ldap.ALREADY_EXISTS, e: raise errors.DuplicateEntry() except ldap.CONSTRAINT_VIOLATION, e: @@ -344,7 +346,7 @@ class IPAdmin(SimpleLDAPObject): self.principal = principal self.proxydn = None except ldap.LDAPError, e: - self.__handle_errors(e, **{}) + self.__handle_errors(e) def do_simple_bind(self, binddn="cn=directory manager", bindpw=""): self.binddn = binddn @@ -361,7 +363,7 @@ class IPAdmin(SimpleLDAPObject): self.sasl_interactive_bind_s("", auth_tokens) self.__lateinit() - def getEntry(self,*args): + def getEntry(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0): """This wraps the search function. It is common to just get one entry""" sctrl = self.__get_server_controls() @@ -370,21 +372,22 @@ class IPAdmin(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) try: - res = self.search(*args) + res = self.search(base, scope, filterstr, attrlist, attrsonly) objtype, obj = self.result(res) except ldap.LDAPError, e: - kw = {'args': args} - self.__handle_errors(e, **kw) + arg_desc = 'base="%s", scope=%s, filterstr="%s"' % (base, scope, filterstr) + self.__handle_errors(e, arg_desc=arg_desc) if not obj: - raise errors.NotFound(reason=notfound(args)) + arg_desc = 'base="%s", scope=%s, filterstr="%s"' % (base, scope, filterstr) + raise errors.NotFound(reason=arg_desc) elif isinstance(obj,Entry): return obj else: # assume list/tuple return obj[0] - def getList(self,*args): + def getList(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0): """This wraps the search function to find multiple entries.""" sctrl = self.__get_server_controls() @@ -392,14 +395,15 @@ class IPAdmin(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) try: - res = self.search(*args) + res = self.search(base, scope, filterstr, attrlist, attrsonly) objtype, obj = self.result(res) except ldap.LDAPError, e: - kw = {'args': args} - self.__handle_errors(e, **kw) + arg_desc = 'base="%s", scope=%s, filterstr="%s"' % (base, scope, filterstr) + self.__handle_errors(e, arg_desc=arg_desc) if not obj: - raise errors.NotFound(reason=notfound(args)) + arg_desc = 'base="%s", scope=%s, filterstr="%s"' % (base, scope, filterstr) + raise errors.NotFound(reason=arg_desc) entries = [] for s in obj: @@ -407,7 +411,8 @@ class IPAdmin(SimpleLDAPObject): return entries - def getListAsync(self,*args): + def getListAsync(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0, + serverctrls=None, clientctrls=None, timeout=-1, sizelimit=0): """This version performs an asynchronous search, to allow results even if we hit a limit. @@ -423,7 +428,8 @@ class IPAdmin(SimpleLDAPObject): partial = 0 try: - msgid = self.search_ext(*args) + msgid = self.search_ext(base, scope, filterstr, attrlist, attrsonly, + serverctrls, clientctrls, timeout, sizelimit) objtype, result_list = self.result(msgid, 0) while result_list: for result in result_list: @@ -433,11 +439,13 @@ class IPAdmin(SimpleLDAPObject): ldap.TIMELIMIT_EXCEEDED), e: partial = 1 except ldap.LDAPError, e: - kw = {'args': args} - self.__handle_errors(e, **kw) + arg_desc = 'base="%s", scope=%s, filterstr="%s", timeout=%s, sizelimit=%s' % \ + (base, scope, filterstr, timeout, sizelimit) + self.__handle_errors(e, arg_desc=arg_desc) if not entries: - raise errors.NotFound(reason=notfound(args)) + arg_desc = 'base="%s", scope=%s, filterstr="%s"' % (base, scope, filterstr) + raise errors.NotFound(reason=arg_desc) if partial == 1: counter = -1 @@ -446,19 +454,22 @@ class IPAdmin(SimpleLDAPObject): return [counter] + entries - def addEntry(self,*args): + def addEntry(self, entry): """This wraps the add function. It assumes that the entry is already populated with all of the desired objectclasses and attributes""" + if not isinstance(entry, (Entry, Entity)): + raise TypeError('addEntry expected an Entry or Entity object, got %s instead' % entry.__class__) + sctrl = self.__get_server_controls() try: if sctrl is not None: self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) - self.add_s(*args) + self.add_s(entry.dn, entry.toTupleList()) except ldap.LDAPError, e: - kw = {'args': args} - self.__handle_errors(e, **kw) + arg_desc = 'entry=%s, modlist=%s' % (entry) + self.__handle_errors(e, arg_desc=arg_desc) return True def updateRDN(self, dn, newrdn): @@ -475,7 +486,7 @@ class IPAdmin(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) self.modrdn_s(dn, newrdn, delold=1) except ldap.LDAPError, e: - self.__handle_errors(e, **{}) + self.__handle_errors(e) return True def updateEntry(self,dn,oldentry,newentry): @@ -494,7 +505,7 @@ class IPAdmin(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) self.modify_s(dn, modlist) except ldap.LDAPError, e: - self.__handle_errors(e, **{}) + self.__handle_errors(e) return True def generateModList(self, old_entry, new_entry): @@ -575,10 +586,10 @@ class IPAdmin(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) self.modify_s(dn, modlist) except ldap.LDAPError, e: - self.__handle_errors(e, **{}) + self.__handle_errors(e) return True - def deleteEntry(self,*args): + def deleteEntry(self, dn): """This wraps the delete function. Use with caution.""" sctrl = self.__get_server_controls() @@ -586,10 +597,10 @@ class IPAdmin(SimpleLDAPObject): try: if sctrl is not None: self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) - self.delete_s(*args) + self.delete_s(dn) except ldap.LDAPError, e: - kw = {'args': args} - self.__handle_errors(e, **kw) + arg_desc = 'dn=%s' % (dn) + self.__handle_errors(e, arg_desc=arg_desc) return True def modifyPassword(self,dn,oldpass,newpass): @@ -607,7 +618,7 @@ class IPAdmin(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) self.passwd_s(dn, oldpass, newpass) except ldap.LDAPError, e: - self.__handle_errors(e, **{}) + self.__handle_errors(e) return True def __wrapmethods(self): @@ -737,22 +748,3 @@ class IPAdmin(SimpleLDAPObject): keys.sort(reverse=reverse) return map(res.get, keys) - - -def notfound(args): - """Return a string suitable for displaying as an error when a - search returns no results. - - This just returns whatever is after the equals sign""" - if len(args) > 2: - searchfilter = args[2] - try: - # Python re doesn't do paren counting so the string could - # have a trailing paren "foo)" - target = re.match(r'\(.*=(.*)\)', searchfilter).group(1) - target = target.replace(")","") - except: - target = searchfilter - return "%s not found" % str(target) - else: - return args[0] -- 1.7.7.3 -------------- next part -------------- >From ef26fcd8c0909e6fd0b57a43575f4ea0006ac3f5 Mon Sep 17 00:00:00 2001 From: John Dennis Date: Mon, 26 Sep 2011 10:39:15 -0400 Subject: [PATCH 1/2] Ticket #1879 - IPAdmin undefined anonymous parameter lists The IPAdmin class in ipaserver/ipaldap.py has methods with anonymous undefined parameter lists. For example: def getList(self,*args): In Python syntax this means you can call getList with any positional parameter list you want. This is bad because: 1) It's not true, *args gets passed to an ldap function with a well defined parameter list, so you really do have to call it with a defined parameter list. *args will let you pass anything, but once it gets passed to the ldap function it will blow up if the parameters do not match (what parameters are those you're wondering? see item 2). 2) The programmer does not know what the valid parameters are unless they are defined in the formal parameter list. 3) Without a formal parameter list automatic documentation generators cannot produce API documentation (see item 2) 4) The Python interpreter cannot validate the parameters being passed because there is no formal parameter list. Note, Python does not validate the type of parameters, but it does validate the correct number of postitional parameters are passed and only defined keyword parameters are passed. Bypassing the language support facilities leads to programming errors. 5) Without a formal parameter list program checkers such as pylint cannot validate the program which leads to progamming errors. 6) Without a formal parameter list which includes default keyword parameters it's not possible to use keyword arguments nor to know what their default values are (see item 2). One is forced to pass a keyword argument as a positional argument, plus you must then pass every keyword argument between the end of the positional argument list and keyword arg of interest even of the other keyword arguments are not of interest. This also demands you know what the default value of the intermediate keyword arguments are (see item 2) and hope they don't change. Also the *args anonymous tuple get passed into the error handling code so it can report what the called values were. But because the tuple is anonymous the error handler cannot not describe what it was passed. In addition the error handling code makes assumptions about the possible contents of the anonymous tuple based on current practice instead of actual defined values. Things like "if the number of items in the tuple is 2 or less then the first tuple item must be a dn (Distinguished Name)" or "if the number of items in the tuple is greater than 2 then the 3rd item must be an ldap search filter". These are constructs which are not robust and will fail at some point in the future. This patch also fixes the use of IPAdmin.addEntry(). It was sometimes being called with (dn, modlist), sometimes a Entry object, or sometimes a Entity object. Now it's always called with either a Entry or Entity object and IPAdmin.addEntry() validates the type of the parameter passed. --- ipaserver/install/ldapupdate.py | 4 +- ipaserver/ipaldap.py | 94 ++++++++++++++++++--------------------- 2 files changed, 45 insertions(+), 53 deletions(-) diff --git a/ipaserver/install/ldapupdate.py b/ipaserver/install/ldapupdate.py index 575f22421b6e310d35a0ac52dc85b8a559f54e93..91d3d83cffd9e53073dc53aad63f9f973e8933e1 100644 --- a/ipaserver/install/ldapupdate.py +++ b/ipaserver/install/ldapupdate.py @@ -344,7 +344,7 @@ class LDAPUpdate: root_logger.debug("Task id: %s", dn) if self.live_run: - self.conn.addEntry(e.dn, e.toTupleList()) + self.conn.addEntry(e) return dn @@ -644,7 +644,7 @@ class LDAPUpdate: # addifexist may result in an entry with only a # dn defined. In that case there is nothing to do. # It means the entry doesn't exist, so skip it. - self.conn.addEntry(entry.dn, entry.toTupleList()) + self.conn.addEntry(entry) self.modified = True except Exception, e: root_logger.error("Add failure %s", e) diff --git a/ipaserver/ipaldap.py b/ipaserver/ipaldap.py index 74cfbfda911facbf6f3bddf5972b3f035a9cfde0..fdcbe62495805f76397e7588fe24041ce540b8c0 100644 --- a/ipaserver/ipaldap.py +++ b/ipaserver/ipaldap.py @@ -1,5 +1,6 @@ # Authors: Rich Megginson -# Rob Crittenden +# John Dennis # # Copyright (C) 2007 Red Hat # see file 'COPYING' for use and warranty information @@ -35,6 +36,7 @@ from ldap.ldapobject import SimpleLDAPObject from ipaserver import ipautil from ipalib import errors from ipapython.ipautil import format_netloc +from ipapython.entity import Entity # Global variable to define SASL auth SASL_AUTH = ldap.sasl.sasl({},'GSSAPI') @@ -263,7 +265,7 @@ class IPAdmin(SimpleLDAPObject): self.dbdir = os.path.dirname(ent.getValue('nsslapd-directory')) except ldap.LDAPError, e: - self.__handle_errors(e, **{}) + self.__handle_errors(e) def __str__(self): return self.host + ":" + str(self.port) @@ -304,8 +306,8 @@ class IPAdmin(SimpleLDAPObject): # re-raise the error so we can handle it raise e except ldap.NO_SUCH_OBJECT, e: - args = kw.get('args', ["entry not found"]) - raise errors.NotFound(reason=notfound(args)) + arg_desc = kw.get('arg_desc', "entry not found") + raise errors.NotFound(reason=arg_desc) except ldap.ALREADY_EXISTS, e: raise errors.DuplicateEntry() except ldap.CONSTRAINT_VIOLATION, e: @@ -344,7 +346,7 @@ class IPAdmin(SimpleLDAPObject): self.principal = principal self.proxydn = None except ldap.LDAPError, e: - self.__handle_errors(e, **{}) + self.__handle_errors(e) def do_simple_bind(self, binddn="cn=directory manager", bindpw=""): self.binddn = binddn @@ -361,7 +363,7 @@ class IPAdmin(SimpleLDAPObject): self.sasl_interactive_bind_s("", auth_tokens) self.__lateinit() - def getEntry(self,*args): + def getEntry(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0): """This wraps the search function. It is common to just get one entry""" sctrl = self.__get_server_controls() @@ -370,21 +372,22 @@ class IPAdmin(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) try: - res = self.search(*args) + res = self.search(base, scope, filterstr, attrlist, attrsonly) objtype, obj = self.result(res) except ldap.LDAPError, e: - kw = {'args': args} - self.__handle_errors(e, **kw) + arg_desc = 'base="%s", scope=%s, filterstr="%s"' % (base, scope, filterstr) + self.__handle_errors(e, arg_desc=arg_desc) if not obj: - raise errors.NotFound(reason=notfound(args)) + arg_desc = 'base="%s", scope=%s, filterstr="%s"' % (base, scope, filterstr) + raise errors.NotFound(reason=arg_desc) elif isinstance(obj,Entry): return obj else: # assume list/tuple return obj[0] - def getList(self,*args): + def getList(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0): """This wraps the search function to find multiple entries.""" sctrl = self.__get_server_controls() @@ -392,14 +395,15 @@ class IPAdmin(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) try: - res = self.search(*args) + res = self.search(base, scope, filterstr, attrlist, attrsonly) objtype, obj = self.result(res) except ldap.LDAPError, e: - kw = {'args': args} - self.__handle_errors(e, **kw) + arg_desc = 'base="%s", scope=%s, filterstr="%s"' % (base, scope, filterstr) + self.__handle_errors(e, arg_desc=arg_desc) if not obj: - raise errors.NotFound(reason=notfound(args)) + arg_desc = 'base="%s", scope=%s, filterstr="%s"' % (base, scope, filterstr) + raise errors.NotFound(reason=arg_desc) entries = [] for s in obj: @@ -407,7 +411,8 @@ class IPAdmin(SimpleLDAPObject): return entries - def getListAsync(self,*args): + def getListAsync(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0, + serverctrls=None, clientctrls=None, timeout=-1, sizelimit=0): """This version performs an asynchronous search, to allow results even if we hit a limit. @@ -423,7 +428,8 @@ class IPAdmin(SimpleLDAPObject): partial = 0 try: - msgid = self.search_ext(*args) + msgid = self.search_ext(base, scope, filterstr, attrlist, attrsonly, + serverctrls, clientctrls, timeout, sizelimit) objtype, result_list = self.result(msgid, 0) while result_list: for result in result_list: @@ -433,11 +439,13 @@ class IPAdmin(SimpleLDAPObject): ldap.TIMELIMIT_EXCEEDED), e: partial = 1 except ldap.LDAPError, e: - kw = {'args': args} - self.__handle_errors(e, **kw) + arg_desc = 'base="%s", scope=%s, filterstr="%s", timeout=%s, sizelimit=%s' % \ + (base, scope, filterstr, timeout, sizelimit) + self.__handle_errors(e, arg_desc=arg_desc) if not entries: - raise errors.NotFound(reason=notfound(args)) + arg_desc = 'base="%s", scope=%s, filterstr="%s"' % (base, scope, filterstr) + raise errors.NotFound(reason=arg_desc) if partial == 1: counter = -1 @@ -446,19 +454,22 @@ class IPAdmin(SimpleLDAPObject): return [counter] + entries - def addEntry(self,*args): + def addEntry(self, entry): """This wraps the add function. It assumes that the entry is already populated with all of the desired objectclasses and attributes""" + if not isinstance(entry, (Entry, Entity)): + raise TypeError('addEntry expected an Entry or Entity object, got %s instead' % entry.__class__) + sctrl = self.__get_server_controls() try: if sctrl is not None: self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) - self.add_s(*args) + self.add_s(entry.dn, entry.toTupleList()) except ldap.LDAPError, e: - kw = {'args': args} - self.__handle_errors(e, **kw) + arg_desc = 'entry=%s, modlist=%s' % (entry) + self.__handle_errors(e, arg_desc=arg_desc) return True def updateRDN(self, dn, newrdn): @@ -475,7 +486,7 @@ class IPAdmin(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) self.modrdn_s(dn, newrdn, delold=1) except ldap.LDAPError, e: - self.__handle_errors(e, **{}) + self.__handle_errors(e) return True def updateEntry(self,dn,oldentry,newentry): @@ -494,7 +505,7 @@ class IPAdmin(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) self.modify_s(dn, modlist) except ldap.LDAPError, e: - self.__handle_errors(e, **{}) + self.__handle_errors(e) return True def generateModList(self, old_entry, new_entry): @@ -575,10 +586,10 @@ class IPAdmin(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) self.modify_s(dn, modlist) except ldap.LDAPError, e: - self.__handle_errors(e, **{}) + self.__handle_errors(e) return True - def deleteEntry(self,*args): + def deleteEntry(self, dn): """This wraps the delete function. Use with caution.""" sctrl = self.__get_server_controls() @@ -586,10 +597,10 @@ class IPAdmin(SimpleLDAPObject): try: if sctrl is not None: self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) - self.delete_s(*args) + self.delete_s(dn) except ldap.LDAPError, e: - kw = {'args': args} - self.__handle_errors(e, **kw) + arg_desc = 'dn=%s' % (dn) + self.__handle_errors(e, arg_desc=arg_desc) return True def modifyPassword(self,dn,oldpass,newpass): @@ -607,7 +618,7 @@ class IPAdmin(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) self.passwd_s(dn, oldpass, newpass) except ldap.LDAPError, e: - self.__handle_errors(e, **{}) + self.__handle_errors(e) return True def __wrapmethods(self): @@ -737,22 +748,3 @@ class IPAdmin(SimpleLDAPObject): keys.sort(reverse=reverse) return map(res.get, keys) - - -def notfound(args): - """Return a string suitable for displaying as an error when a - search returns no results. - - This just returns whatever is after the equals sign""" - if len(args) > 2: - searchfilter = args[2] - try: - # Python re doesn't do paren counting so the string could - # have a trailing paren "foo)" - target = re.match(r'\(.*=(.*)\)', searchfilter).group(1) - target = target.replace(")","") - except: - target = searchfilter - return "%s not found" % str(target) - else: - return args[0] -- 1.7.7.3 -------------- next part -------------- >From 66c3473b0b7231dc58e84f1715640e665b7580b8 Mon Sep 17 00:00:00 2001 From: John Dennis Date: Mon, 26 Sep 2011 17:33:32 -0400 Subject: [PATCH 2/3] ticket #1870 - subclass SimpleLDAPObject We use convenience types (classes) in IPA which make working with LDAP easier and more robust. It would be really nice if the basic python-ldap library understood our utility types and could accept them as parameters to the basic ldap functions and/or the basic ldap functions returned our utility types. Normally such a requirement would trivially be handled in an object- oriented language (which Python is) by subclassing to extend and modify the functionality. For some reason we didn't do this with the python-ldap classes. python-ldap objects are primarily used in two different places in our code, ipaserver.ipaldap.py for the IPAdmin class and in ipaserver/plugins/ldap2.py for the ldap2 class's .conn member. In IPAdmin we use a IPA utility class called Entry to make it easier to use the results returned by LDAP. The IPAdmin class is derived from python-ldap.SimpleLDAPObject. But for some reason when we added the support for the use of the Entry class in SimpleLDAPObject we didn't subclass SimpleLDAPObject and extend it for use with the Entry class as would be the normal expected methodology in an object-oriented language, rather we used an obscure feature of the Python language to override all methods of the SimpleLDAPObject class by wrapping those class methods in another function call. The reason why this isn't a good approach is: * It violates object-oriented methodology. * Other classes cannot be derived and inherit the customization (because the method wrapping occurs in a class instance, not within the class type). * It's non-obvious and obscure * It's inefficient. Here is a summary of what the code was doing: It iterated over every member of the SimpleLDAPObject class and if it was callable it wrapped the method. The wrapper function tested the name of the method being wrapped, if it was one of a handful of methods we wanted to customize we modified a parameter and called the original method. If the method wasn't of interest to use we still wrapped the method. It was inefficient because every non-customized method (the majority) executed a function call for the wrapper, the wrapper during run-time used logic to determine if the method was being overridden and then called the original method. So every call to ldap was doing extra function calls and logic processing which for the majority of cases produced nothing useful (and was non-obvious from brief code reading some methods were being overridden). Object-orientated languages have support built in for calling the right method for a given class object that do not involve extra function call overhead to realize customized class behaviour. Also when programmers look for customized class behaviour they look for derived classes. They might also want to utilize the customized class as the base class for their use. Also the wrapper logic was fragile, it did things like: if the method name begins with "add" I'll unconditionally modify the first and second argument. It would be some much cleaner if the "add", "add_s", etc. methods were overridden in a subclass where the logic could be seen and where it would apply to only the explicit functions and parameters being overridden. Also we would really benefit if there were classes which could be used as a base class which had specific ldap customization. At the moment our ldap customization needs are: 1) Support DN objects being passed to ldap operations 2) Support Entry & Entity objects being passed into and returned from ldap operations. We want to subclass the ldap SimpleLDAPObject class, that is the base ldap class with all the ldap methods we're using. IPASimpleLDAPObject class would subclass SimpleLDAPObject class which knows about DN objects (and possilby other IPA specific types that are universally used in IPA). Then IPAEntrySimpleLDAPObject would subclass IPASimpleLDAPObject which knows about Entry objects. The reason for the suggested class hierarchy is because DN objects will be used whenever we talk to LDAP (in the future we may want to add other IPA specific classes which will always be used). We don't add Entry support to the the IPASimpleLDAPObject class because Entry objects are (currently) only used in IPAdmin. What this patch does is: * Introduce IPASimpleLDAPObject derived from SimpleLDAPObject. IPASimpleLDAPObject is DN object aware. * Introduce IPAEntryLDAPObject derived from IPASimpleLDAPObject. IPAEntryLDAPObject is Entry object aware. * Derive IPAdmin from IPAEntryLDAPObject and remove the funky method wrapping from IPAdmin. * Code which called add_s() with an Entry or Entity object now calls addEntry(). addEntry() always existed, it just wasn't always used. add_s() had been modified to accept Entry or Entity object (why didn't we just call addEntry()?). The add*() ldap routine in IPAEntryLDAPObject have been subclassed to accept Entry and Entity objects, but that should proably be removed in the future and just use addEntry(). * Replace the call to ldap.initialize() in ldap2.create_connection() with a class constructor for IPASimpleLDAPObject. The ldap.initialize() is a convenience function in python-ldap, but it always returns a SimpleLDAPObject created via the SimpleLDAPObject constructor, thus ldap.initialize() did not allow subclassing, yet has no particular ease-of-use advantage thus we better off using the obvious class constructor mechanism. * Fix the use of _handle_errors(), it's not necessary to construct an empty dict to pass to it. If we follow the standard class derivation pattern for ldap we can make us of our own ldap utilities in a far easier, cleaner and more efficient manner. --- ipaserver/install/krbinstance.py | 4 +- ipaserver/install/service.py | 2 +- ipaserver/ipaldap.py | 105 ++++++++++++++------------------ ipaserver/plugins/ldap2.py | 124 ++++++++++++++++++++++++++++++++++--- 4 files changed, 164 insertions(+), 71 deletions(-) diff --git a/ipaserver/install/krbinstance.py b/ipaserver/install/krbinstance.py index ce70c231dfb7e7b6b59c0496721cced0d09f1604..df6fc5a6ea6fbc4d9c207122dbb3c1ce1f5b4f50 100644 --- a/ipaserver/install/krbinstance.py +++ b/ipaserver/install/krbinstance.py @@ -284,7 +284,7 @@ class KrbInstance(service.Service): entry.setValues("nsSaslMapFilterTemplate", '(krbPrincipalName=\\1@\\2)') try: - self.admin_conn.add_s(entry) + self.admin_conn.addEntry(entry) except ldap.ALREADY_EXISTS: logging.critical("failed to add Full Principal Sasl mapping") raise e @@ -297,7 +297,7 @@ class KrbInstance(service.Service): entry.setValues("nsSaslMapFilterTemplate", '(krbPrincipalName=&@%s)' % self.realm) try: - self.admin_conn.add_s(entry) + self.admin_conn.addEntry(entry) except ldap.ALREADY_EXISTS: logging.critical("failed to add Name Only Sasl mapping") raise e diff --git a/ipaserver/install/service.py b/ipaserver/install/service.py index 2fd15d8f8010114914549871fc5d0a228561fe1c..9fcc095b64f1abc121f1960d7c7ec15dbe53821f 100644 --- a/ipaserver/install/service.py +++ b/ipaserver/install/service.py @@ -287,7 +287,7 @@ class Service(object): "enabledService", "startOrder " + str(order)) try: - conn.add_s(entry) + conn.addEntry(entry) except ldap.ALREADY_EXISTS, e: logging.critical("failed to add %s Service startup entry" % name) raise e diff --git a/ipaserver/ipaldap.py b/ipaserver/ipaldap.py index fdcbe62495805f76397e7588fe24041ce540b8c0..82feacbc1c69d543be1ede9a5dc6ffdba45b50e5 100644 --- a/ipaserver/ipaldap.py +++ b/ipaserver/ipaldap.py @@ -37,10 +37,52 @@ from ipaserver import ipautil from ipalib import errors from ipapython.ipautil import format_netloc from ipapython.entity import Entity +from ipaserver.plugins.ldap2 import IPASimpleLDAPObject # Global variable to define SASL auth SASL_AUTH = ldap.sasl.sasl({},'GSSAPI') +class IPAEntryLDAPObject(IPASimpleLDAPObject): + def __init__(self, *args, **kwds): + IPASimpleLDAPObject.__init__(self, *args, **kwds) + + def result(self, msgid=ldap.RES_ANY, all=1, timeout=None): + objtype, data = IPASimpleLDAPObject.result(self, msgid, all, timeout) + # data is either a 2-tuple or a list of 2-tuples + if data: + if isinstance(data, tuple): + return objtype, Entry(data) + elif isinstance(data, list): + return objtype, [Entry(x) for x in data] + else: + raise TypeError, "unknown data type %s returned by result" % type(data) + else: + return objtype, data + + def add(self, dn, modlist): + if isinstance(dn, Entry): + return IPASimpleLDAPObject.add(self, dn.dn, dn.toTupleList()) + else: + return IPASimpleLDAPObject.add(self, dn, modlist) + + def add_s(self, dn, modlist): + if isinstance(dn, Entry): + return IPASimpleLDAPObject.add_s(self, dn.dn, dn.toTupleList()) + else: + return IPASimpleLDAPObject.add_s(self, dn, modlist) + + def add_ext(self, dn, modlist, serverctrls=None, clientctrls=None): + if isinstance(dn, Entry): + return IPASimpleLDAPObject.add_ext(self, dn.dn, dn.toTupleList(), serverctrls, clientctrls) + else: + return IPASimpleLDAPObject.add_ext(self, dn, modlist, serverctrls, clientctrls) + + def add_ext_s(self, dn, modlist, serverctrls=None, clientctrls=None): + if isinstance(dn, Entry): + return IPASimpleLDAPObject.add_ext_s(self, dn.dn, dn.toTupleList(), serverctrls, clientctrls) + else: + return IPASimpleLDAPObject.add_ext_s(self, dn, modlist, serverctrls, clientctrls) + class Entry: """ This class represents an LDAP Entry object. An LDAP entry consists of @@ -165,49 +207,7 @@ class Entry: ldif.LDIFWriter(sio,Entry.base64_attrs,1000).unparse(self.dn,newdata) return sio.getvalue() -def wrapper(f,name): - """This is the method that wraps all of the methods of the superclass. - This seems to need to be an unbound method, that's why it's outside - of IPAdmin. Perhaps there is some way to do this with the new - classmethod or staticmethod of 2.4. Basically, we replace every call - to a method in SimpleLDAPObject (the superclass of IPAdmin) with a - call to inner. The f argument to wrapper is the bound method of - IPAdmin (which is inherited from the superclass). Bound means that it - will implicitly be called with the self argument, it is not in the - args list. name is the name of the method to call. If name is a - method that returns entry objects (e.g. result), we wrap the data - returned by an Entry class. If name is a method that takes an entry - argument, we extract the raw data from the entry object to pass in. - """ - def inner(*args, **kargs): - if name == 'result': - objtype, data = f(*args, **kargs) - # data is either a 2-tuple or a list of 2-tuples - # print data - if data: - if isinstance(data,tuple): - return objtype, Entry(data) - elif isinstance(data,list): - return objtype, [Entry(x) for x in data] - else: - raise TypeError, "unknown data type %s returned by result" % type(data) - else: - return objtype, data - elif name.startswith('add'): - # the first arg is self - # the second and third arg are the dn and the data to send - # We need to convert the Entry into the format used by - # python-ldap - ent = args[0] - if isinstance(ent,Entry): - return f(ent.dn, ent.toTupleList(), *args[2:]) - else: - return f(*args, **kargs) - else: - return f(*args, **kargs) - return inner - -class IPAdmin(SimpleLDAPObject): +class IPAdmin(IPAEntryLDAPObject): def __localinit(self): """If a CA certificate is provided then it is assumed that we are @@ -218,12 +218,12 @@ class IPAdmin(SimpleLDAPObject): its own encryption. """ if self.cacert is not None: - SimpleLDAPObject.__init__(self,'ldaps://%s' % format_netloc(self.host, self.port)) + IPAEntryLDAPObject.__init__(self,'ldaps://%s' % format_netloc(self.host, self.port)) else: if self.ldapi: - SimpleLDAPObject.__init__(self,'ldapi://%%2fvar%%2frun%%2fslapd-%s.socket' % "-".join(self.realm.split("."))) + IPAEntryLDAPObject.__init__(self,'ldapi://%%2fvar%%2frun%%2fslapd-%s.socket' % "-".join(self.realm.split("."))) else: - SimpleLDAPObject.__init__(self,'ldap://%s' % format_netloc(self.host, self.port)) + IPAEntryLDAPObject.__init__(self,'ldap://%s' % format_netloc(self.host, self.port)) def __init__(self,host='',port=389,cacert=None,bindcert=None,bindkey=None,proxydn=None,debug=None,ldapi=False,realm=None): """We just set our instance variables and wrap the methods - the real @@ -240,7 +240,6 @@ class IPAdmin(SimpleLDAPObject): if bindkey is not None: ldap.set_option(ldap.OPT_X_TLS_KEYFILE,bindkey) - self.__wrapmethods() self.port = port self.host = host self.cacert = cacert @@ -468,7 +467,7 @@ class IPAdmin(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) self.add_s(entry.dn, entry.toTupleList()) except ldap.LDAPError, e: - arg_desc = 'entry=%s, modlist=%s' % (entry) + arg_desc = 'entry=%s' % (entry) self.__handle_errors(e, arg_desc=arg_desc) return True @@ -621,16 +620,6 @@ class IPAdmin(SimpleLDAPObject): self.__handle_errors(e) return True - def __wrapmethods(self): - """This wraps all methods of SimpleLDAPObject, so that we can intercept - the methods that deal with entries. Instead of using a raw list of tuples - of lists of hashes of arrays as the entry object, we want to wrap entries - in an Entry class that provides some useful methods""" - for name in dir(self.__class__.__bases__[0]): - attr = getattr(self, name) - if callable(attr): - setattr(self, name, wrapper(attr, name)) - def waitForEntry(self, dn, timeout=7200, attr='', quiet=True): scope = ldap.SCOPE_BASE filter = "(objectclass=*)" diff --git a/ipaserver/plugins/ldap2.py b/ipaserver/plugins/ldap2.py index 5c40182933143021abf817bff4ed12287697e4ea..d014c3f67f5b6c87fc7fec2443994cc5d59e6b5b 100644 --- a/ipaserver/plugins/ldap2.py +++ b/ipaserver/plugins/ldap2.py @@ -38,6 +38,7 @@ import re import krbV import logging import ldap as _ldap +from ldap.ldapobject import SimpleLDAPObject import ldap.filter as _ldap_filter import ldap.sasl as _ldap_sasl from ldap.controls import LDAPControl @@ -53,7 +54,6 @@ from ipalib.crud import CrudBackend from ipalib.encoder import Encoder, encode_args, decode_retval from ipalib.request import context - # Group Member types MEMBERS_ALL = 0 MEMBERS_DIRECT = 1 @@ -62,6 +62,110 @@ MEMBERS_INDIRECT = 2 # SASL authentication mechanism SASL_AUTH = _ldap_sasl.sasl({}, 'GSSAPI') +class IPASimpleLDAPObject(SimpleLDAPObject): + ''' + This is a thin layer over SimpleLDAPObject which allows us to utilize + IPA specific types with the python-ldap API without the IPA caller needing + to perform the type translation, consider this a convenience layer for the + IPA programmer. + + This subclass performs the following translations: + + * DN objects may be passed into any ldap function expecting a dn. The DN + object will be converted to a string before being passed to the python-ldap + function. This allows us to maintain DN objects as DN objects in the rest + of the code (useful for DN manipulation and DN information) and not have + to worry about conversion to a string prior to passing it ldap. + + ''' + def __init__(self, *args, **kwds): + SimpleLDAPObject.__init__(self, *args, **kwds) + + def add(self, dn, modlist): + return SimpleLDAPObject.add(self, str(dn), modlist) + + def add_ext(self, dn, modlist, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.add_ext(self, str(dn), modlist, serverctrls, clientctrls) + + def add_ext_s(self, dn, modlist, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.add_ext_s(self, str(dn), modlist, serverctrls, clientctrls) + + def add_s(self, dn, modlist): + return SimpleLDAPObject.add_s(self, str(dn), modlist) + + def compare(self, dn, attr, value): + return SimpleLDAPObject.compare(self, str(dn), attr, value) + + def compare_ext(self, dn, attr, value, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.compare_ext(self, str(dn), attr, value, serverctrls, clientctrls) + + def compare_ext_s(self, dn, attr, value, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.compare_ext_s(self, str(dn), attr, value, serverctrls, clientctrls) + + def compare_s(self, dn, attr, value): + return SimpleLDAPObject.compare_s(self, str(dn), attr, value) + + def delete(self, dn): + return SimpleLDAPObject.delete(self, str(dn)) + + def delete_ext(self, dn, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.delete_ext(self, str(dn), serverctrls, clientctrls) + + def delete_ext_s(self, dn, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.delete_ext_s(self, str(dn), serverctrls, clientctrls) + + def delete_s(self, dn): + return SimpleLDAPObject.delete_s(self, str(dn)) + + def modify(self, dn, modlist): + return SimpleLDAPObject.modify(self, str(dn), modlist) + + def modify_ext(self, dn, modlist, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.modify_ext(self, str(dn), modlist, serverctrls, clientctrls) + + def modify_ext_s(self, dn, modlist, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.modify_ext_s(self, str(dn), modlist, serverctrls, clientctrls) + + def modify_s(self, dn, modlist): + return SimpleLDAPObject.modify_s(self, str(dn), modlist) + + def modrdn(self, dn, newrdn, delold=1): + return SimpleLDAPObject.modrdn(self, str(dn), str(newrdn), delold) + + def modrdn_s(self, dn, newrdn, delold=1): + return SimpleLDAPObject.modrdn_s(self, str(dn), str(newrdn), delold) + + def read_subschemasubentry_s(self, subschemasubentry_dn, attrs=None): + return SimpleLDAPObject.read_subschemasubentry_s(self, str(subschemasubentry_dn), attrs) + + def rename(self, dn, newrdn, newsuperior=None, delold=1, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.rename(self, str(dn), str(newrdn), newsuperior, delold, serverctrls, clientctrls) + + def rename_s(self, dn, newrdn, newsuperior=None, delold=1, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.rename_s(self, str(dn), str(newrdn), newsuperior, delold, serverctrls, clientctrls) + + def search(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0): + return SimpleLDAPObject.search(self, str(base), scope, filterstr, attrlist, attrsonly) + + def search_ext(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0, + serverctrls=None, clientctrls=None, timeout=-1, sizelimit=0): + return SimpleLDAPObject.search_ext(self, str(base), scope, filterstr, attrlist, attrsonly, + serverctrls, clientctrls, timeout, sizelimit) + + def search_ext_s(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0, + serverctrls=None, clientctrls=None, timeout=-1, sizelimit=0): + return SimpleLDAPObject.search_ext_s(self, str(base), scope, filterstr, attrlist, attrsonly, + serverctrls, clientctrls, timeout, sizelimit) + + def search_s(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0): + return SimpleLDAPObject.search_s(self, str(base), scope, filterstr, attrlist, attrsonly) + + def search_st(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0, timeout=-1): + return SimpleLDAPObject.search_st(self, str(base), scope, filterstr, attrlist, attrsonly, timeout) + + def search_subschemasubentry_s(self, dn=''): + return SimpleLDAPObject.search_subschemasubentry_s(self, str(dn)) + # universal LDAPError handler def _handle_errors(e, **kw): """ @@ -162,7 +266,7 @@ def get_schema(url, conn=None): raise StandardError('Unable to retrieve LDAP schema. Error initializing principal %s in %s: %s' % (principal.name, '/etc/httpd/conf/ipa.keytab', str(e))) if conn is None: - conn = _ldap.initialize(url) + conn = IPASimpleLDAPObject(url) if url.startswith('ldapi://'): conn.set_option(_ldap.OPT_HOST_NAME, api.env.host) conn.sasl_interactive_bind_s('', SASL_AUTH) @@ -337,7 +441,7 @@ class ldap2(CrudBackend, Encoder): # no kerberos ccache, use simple bind conn.simple_bind_s(bind_dn, bind_pw) except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) if _schema: object.__setattr__(self, 'schema', _schema) @@ -425,7 +529,7 @@ class ldap2(CrudBackend, Encoder): try: self.conn.add_s(dn, list(entry_attrs.iteritems())) except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) # generating filters for find_entry # some examples: @@ -599,7 +703,7 @@ class ldap2(CrudBackend, Encoder): _ldap.SIZELIMIT_EXCEEDED), e: truncated = True except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) if not res: raise errors.NotFound(reason='no such entry') @@ -818,7 +922,7 @@ class ldap2(CrudBackend, Encoder): self.conn.rename_s(dn, new_rdn, delold=int(del_old)) time.sleep(.3) # Give memberOf plugin a chance to work except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) def _generate_modlist(self, dn, entry_attrs, normalize): # get original entry @@ -886,7 +990,7 @@ class ldap2(CrudBackend, Encoder): try: self.conn.modify_s(dn, modlist) except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) @encode_args(1) def delete_entry(self, dn, normalize=True): @@ -896,7 +1000,7 @@ class ldap2(CrudBackend, Encoder): try: self.conn.delete_s(dn) except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) @encode_args(1, 2, 3) def modify_password(self, dn, new_pass, old_pass=''): @@ -916,7 +1020,7 @@ class ldap2(CrudBackend, Encoder): try: self.conn.passwd_s(dn, old_pass, new_pass) except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) def add_entry_to_group(self, dn, group_dn, member_attr='member', allow_same=False): """ @@ -1132,7 +1236,7 @@ class ldap2(CrudBackend, Encoder): try: self.conn.modify_s(dn, mod) except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) # CrudBackend methods -- 1.7.7.3 -------------- next part -------------- >From b75d08773d848fab585d43c7f023d5c9af658707 Mon Sep 17 00:00:00 2001 From: John Dennis Date: Mon, 26 Sep 2011 17:33:32 -0400 Subject: [PATCH 2/2] ticket #1870 - subclass SimpleLDAPObject We use convenience types (classes) in IPA which make working with LDAP easier and more robust. It would be really nice if the basic python-ldap library understood our utility types and could accept them as parameters to the basic ldap functions and/or the basic ldap functions returned our utility types. Normally such a requirement would trivially be handled in an object- oriented language (which Python is) by subclassing to extend and modify the functionality. For some reason we didn't do this with the python-ldap classes. python-ldap objects are primarily used in two different places in our code, ipaserver.ipaldap.py for the IPAdmin class and in ipaserver/plugins/ldap2.py for the ldap2 class's .conn member. In IPAdmin we use a IPA utility class called Entry to make it easier to use the results returned by LDAP. The IPAdmin class is derived from python-ldap.SimpleLDAPObject. But for some reason when we added the support for the use of the Entry class in SimpleLDAPObject we didn't subclass SimpleLDAPObject and extend it for use with the Entry class as would be the normal expected methodology in an object-oriented language, rather we used an obscure feature of the Python language to override all methods of the SimpleLDAPObject class by wrapping those class methods in another function call. The reason why this isn't a good approach is: * It violates object-oriented methodology. * Other classes cannot be derived and inherit the customization (because the method wrapping occurs in a class instance, not within the class type). * It's non-obvious and obscure * It's inefficient. Here is a summary of what the code was doing: It iterated over every member of the SimpleLDAPObject class and if it was callable it wrapped the method. The wrapper function tested the name of the method being wrapped, if it was one of a handful of methods we wanted to customize we modified a parameter and called the original method. If the method wasn't of interest to use we still wrapped the method. It was inefficient because every non-customized method (the majority) executed a function call for the wrapper, the wrapper during run-time used logic to determine if the method was being overridden and then called the original method. So every call to ldap was doing extra function calls and logic processing which for the majority of cases produced nothing useful (and was non-obvious from brief code reading some methods were being overridden). Object-orientated languages have support built in for calling the right method for a given class object that do not involve extra function call overhead to realize customized class behaviour. Also when programmers look for customized class behaviour they look for derived classes. They might also want to utilize the customized class as the base class for their use. Also the wrapper logic was fragile, it did things like: if the method name begins with "add" I'll unconditionally modify the first and second argument. It would be some much cleaner if the "add", "add_s", etc. methods were overridden in a subclass where the logic could be seen and where it would apply to only the explicit functions and parameters being overridden. Also we would really benefit if there were classes which could be used as a base class which had specific ldap customization. At the moment our ldap customization needs are: 1) Support DN objects being passed to ldap operations 2) Support Entry & Entity objects being passed into and returned from ldap operations. We want to subclass the ldap SimpleLDAPObject class, that is the base ldap class with all the ldap methods we're using. IPASimpleLDAPObject class would subclass SimpleLDAPObject class which knows about DN objects (and possilby other IPA specific types that are universally used in IPA). Then IPAEntrySimpleLDAPObject would subclass IPASimpleLDAPObject which knows about Entry objects. The reason for the suggested class hierarchy is because DN objects will be used whenever we talk to LDAP (in the future we may want to add other IPA specific classes which will always be used). We don't add Entry support to the the IPASimpleLDAPObject class because Entry objects are (currently) only used in IPAdmin. What this patch does is: * Introduce IPASimpleLDAPObject derived from SimpleLDAPObject. IPASimpleLDAPObject is DN object aware. * Introduce IPAEntryLDAPObject derived from IPASimpleLDAPObject. IPAEntryLDAPObject is Entry object aware. * Derive IPAdmin from IPAEntryLDAPObject and remove the funky method wrapping from IPAdmin. * Code which called add_s() with an Entry or Entity object now calls addEntry(). addEntry() always existed, it just wasn't always used. add_s() had been modified to accept Entry or Entity object (why didn't we just call addEntry()?). The add*() ldap routine in IPAEntryLDAPObject have been subclassed to accept Entry and Entity objects, but that should proably be removed in the future and just use addEntry(). * Replace the call to ldap.initialize() in ldap2.create_connection() with a class constructor for IPASimpleLDAPObject. The ldap.initialize() is a convenience function in python-ldap, but it always returns a SimpleLDAPObject created via the SimpleLDAPObject constructor, thus ldap.initialize() did not allow subclassing, yet has no particular ease-of-use advantage thus we better off using the obvious class constructor mechanism. * Fix the use of _handle_errors(), it's not necessary to construct an empty dict to pass to it. If we follow the standard class derivation pattern for ldap we can make us of our own ldap utilities in a far easier, cleaner and more efficient manner. --- ipaserver/install/krbinstance.py | 4 +- ipaserver/install/service.py | 2 +- ipaserver/ipaldap.py | 105 ++++++++++++++------------------ ipaserver/plugins/ldap2.py | 124 ++++++++++++++++++++++++++++++++++--- 4 files changed, 164 insertions(+), 71 deletions(-) diff --git a/ipaserver/install/krbinstance.py b/ipaserver/install/krbinstance.py index 6ed38516223a56e723bcc9cf643ac2b362200da4..6566d8a3a4fb0743818b887f3d995a7ff3da8a65 100644 --- a/ipaserver/install/krbinstance.py +++ b/ipaserver/install/krbinstance.py @@ -259,7 +259,7 @@ class KrbInstance(service.Service): entry.setValues("nsSaslMapFilterTemplate", '(krbPrincipalName=\\1@\\2)') try: - self.admin_conn.add_s(entry) + self.admin_conn.addEntry(entry) except ldap.ALREADY_EXISTS: root_logger.critical("failed to add Full Principal Sasl mapping") raise e @@ -272,7 +272,7 @@ class KrbInstance(service.Service): entry.setValues("nsSaslMapFilterTemplate", '(krbPrincipalName=&@%s)' % self.realm) try: - self.admin_conn.add_s(entry) + self.admin_conn.addEntry(entry) except ldap.ALREADY_EXISTS: root_logger.critical("failed to add Name Only Sasl mapping") raise e diff --git a/ipaserver/install/service.py b/ipaserver/install/service.py index 249727b15b6f3c879336c36d43410fc7234d9d9d..d9e6def370567f550a3dbe2c90637a7bbc8129ff 100644 --- a/ipaserver/install/service.py +++ b/ipaserver/install/service.py @@ -289,7 +289,7 @@ class Service(object): "enabledService", "startOrder " + str(order)) try: - conn.add_s(entry) + conn.addEntry(entry) except ldap.ALREADY_EXISTS, e: root_logger.critical("failed to add %s Service startup entry" % name) raise e diff --git a/ipaserver/ipaldap.py b/ipaserver/ipaldap.py index fdcbe62495805f76397e7588fe24041ce540b8c0..82feacbc1c69d543be1ede9a5dc6ffdba45b50e5 100644 --- a/ipaserver/ipaldap.py +++ b/ipaserver/ipaldap.py @@ -37,10 +37,52 @@ from ipaserver import ipautil from ipalib import errors from ipapython.ipautil import format_netloc from ipapython.entity import Entity +from ipaserver.plugins.ldap2 import IPASimpleLDAPObject # Global variable to define SASL auth SASL_AUTH = ldap.sasl.sasl({},'GSSAPI') +class IPAEntryLDAPObject(IPASimpleLDAPObject): + def __init__(self, *args, **kwds): + IPASimpleLDAPObject.__init__(self, *args, **kwds) + + def result(self, msgid=ldap.RES_ANY, all=1, timeout=None): + objtype, data = IPASimpleLDAPObject.result(self, msgid, all, timeout) + # data is either a 2-tuple or a list of 2-tuples + if data: + if isinstance(data, tuple): + return objtype, Entry(data) + elif isinstance(data, list): + return objtype, [Entry(x) for x in data] + else: + raise TypeError, "unknown data type %s returned by result" % type(data) + else: + return objtype, data + + def add(self, dn, modlist): + if isinstance(dn, Entry): + return IPASimpleLDAPObject.add(self, dn.dn, dn.toTupleList()) + else: + return IPASimpleLDAPObject.add(self, dn, modlist) + + def add_s(self, dn, modlist): + if isinstance(dn, Entry): + return IPASimpleLDAPObject.add_s(self, dn.dn, dn.toTupleList()) + else: + return IPASimpleLDAPObject.add_s(self, dn, modlist) + + def add_ext(self, dn, modlist, serverctrls=None, clientctrls=None): + if isinstance(dn, Entry): + return IPASimpleLDAPObject.add_ext(self, dn.dn, dn.toTupleList(), serverctrls, clientctrls) + else: + return IPASimpleLDAPObject.add_ext(self, dn, modlist, serverctrls, clientctrls) + + def add_ext_s(self, dn, modlist, serverctrls=None, clientctrls=None): + if isinstance(dn, Entry): + return IPASimpleLDAPObject.add_ext_s(self, dn.dn, dn.toTupleList(), serverctrls, clientctrls) + else: + return IPASimpleLDAPObject.add_ext_s(self, dn, modlist, serverctrls, clientctrls) + class Entry: """ This class represents an LDAP Entry object. An LDAP entry consists of @@ -165,49 +207,7 @@ class Entry: ldif.LDIFWriter(sio,Entry.base64_attrs,1000).unparse(self.dn,newdata) return sio.getvalue() -def wrapper(f,name): - """This is the method that wraps all of the methods of the superclass. - This seems to need to be an unbound method, that's why it's outside - of IPAdmin. Perhaps there is some way to do this with the new - classmethod or staticmethod of 2.4. Basically, we replace every call - to a method in SimpleLDAPObject (the superclass of IPAdmin) with a - call to inner. The f argument to wrapper is the bound method of - IPAdmin (which is inherited from the superclass). Bound means that it - will implicitly be called with the self argument, it is not in the - args list. name is the name of the method to call. If name is a - method that returns entry objects (e.g. result), we wrap the data - returned by an Entry class. If name is a method that takes an entry - argument, we extract the raw data from the entry object to pass in. - """ - def inner(*args, **kargs): - if name == 'result': - objtype, data = f(*args, **kargs) - # data is either a 2-tuple or a list of 2-tuples - # print data - if data: - if isinstance(data,tuple): - return objtype, Entry(data) - elif isinstance(data,list): - return objtype, [Entry(x) for x in data] - else: - raise TypeError, "unknown data type %s returned by result" % type(data) - else: - return objtype, data - elif name.startswith('add'): - # the first arg is self - # the second and third arg are the dn and the data to send - # We need to convert the Entry into the format used by - # python-ldap - ent = args[0] - if isinstance(ent,Entry): - return f(ent.dn, ent.toTupleList(), *args[2:]) - else: - return f(*args, **kargs) - else: - return f(*args, **kargs) - return inner - -class IPAdmin(SimpleLDAPObject): +class IPAdmin(IPAEntryLDAPObject): def __localinit(self): """If a CA certificate is provided then it is assumed that we are @@ -218,12 +218,12 @@ class IPAdmin(SimpleLDAPObject): its own encryption. """ if self.cacert is not None: - SimpleLDAPObject.__init__(self,'ldaps://%s' % format_netloc(self.host, self.port)) + IPAEntryLDAPObject.__init__(self,'ldaps://%s' % format_netloc(self.host, self.port)) else: if self.ldapi: - SimpleLDAPObject.__init__(self,'ldapi://%%2fvar%%2frun%%2fslapd-%s.socket' % "-".join(self.realm.split("."))) + IPAEntryLDAPObject.__init__(self,'ldapi://%%2fvar%%2frun%%2fslapd-%s.socket' % "-".join(self.realm.split("."))) else: - SimpleLDAPObject.__init__(self,'ldap://%s' % format_netloc(self.host, self.port)) + IPAEntryLDAPObject.__init__(self,'ldap://%s' % format_netloc(self.host, self.port)) def __init__(self,host='',port=389,cacert=None,bindcert=None,bindkey=None,proxydn=None,debug=None,ldapi=False,realm=None): """We just set our instance variables and wrap the methods - the real @@ -240,7 +240,6 @@ class IPAdmin(SimpleLDAPObject): if bindkey is not None: ldap.set_option(ldap.OPT_X_TLS_KEYFILE,bindkey) - self.__wrapmethods() self.port = port self.host = host self.cacert = cacert @@ -468,7 +467,7 @@ class IPAdmin(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) self.add_s(entry.dn, entry.toTupleList()) except ldap.LDAPError, e: - arg_desc = 'entry=%s, modlist=%s' % (entry) + arg_desc = 'entry=%s' % (entry) self.__handle_errors(e, arg_desc=arg_desc) return True @@ -621,16 +620,6 @@ class IPAdmin(SimpleLDAPObject): self.__handle_errors(e) return True - def __wrapmethods(self): - """This wraps all methods of SimpleLDAPObject, so that we can intercept - the methods that deal with entries. Instead of using a raw list of tuples - of lists of hashes of arrays as the entry object, we want to wrap entries - in an Entry class that provides some useful methods""" - for name in dir(self.__class__.__bases__[0]): - attr = getattr(self, name) - if callable(attr): - setattr(self, name, wrapper(attr, name)) - def waitForEntry(self, dn, timeout=7200, attr='', quiet=True): scope = ldap.SCOPE_BASE filter = "(objectclass=*)" diff --git a/ipaserver/plugins/ldap2.py b/ipaserver/plugins/ldap2.py index 1229e5bbcca0cd9ca8b06338b0e52da97d14611b..4bfc849d82cd4384afe9bad7b244184265315f7d 100644 --- a/ipaserver/plugins/ldap2.py +++ b/ipaserver/plugins/ldap2.py @@ -39,6 +39,7 @@ import pwd import krbV from ipapython.ipa_log_manager import * import ldap as _ldap +from ldap.ldapobject import SimpleLDAPObject import ldap.filter as _ldap_filter import ldap.sasl as _ldap_sasl from ldap.controls import LDAPControl @@ -55,7 +56,6 @@ from ipalib.crud import CrudBackend from ipalib.encoder import Encoder, encode_args, decode_retval from ipalib.request import context - # Group Member types MEMBERS_ALL = 0 MEMBERS_DIRECT = 1 @@ -80,6 +80,110 @@ def _encode_bool(self, value): # set own Bool parameter encoder Bool._encode = _encode_bool +class IPASimpleLDAPObject(SimpleLDAPObject): + ''' + This is a thin layer over SimpleLDAPObject which allows us to utilize + IPA specific types with the python-ldap API without the IPA caller needing + to perform the type translation, consider this a convenience layer for the + IPA programmer. + + This subclass performs the following translations: + + * DN objects may be passed into any ldap function expecting a dn. The DN + object will be converted to a string before being passed to the python-ldap + function. This allows us to maintain DN objects as DN objects in the rest + of the code (useful for DN manipulation and DN information) and not have + to worry about conversion to a string prior to passing it ldap. + + ''' + def __init__(self, *args, **kwds): + SimpleLDAPObject.__init__(self, *args, **kwds) + + def add(self, dn, modlist): + return SimpleLDAPObject.add(self, str(dn), modlist) + + def add_ext(self, dn, modlist, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.add_ext(self, str(dn), modlist, serverctrls, clientctrls) + + def add_ext_s(self, dn, modlist, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.add_ext_s(self, str(dn), modlist, serverctrls, clientctrls) + + def add_s(self, dn, modlist): + return SimpleLDAPObject.add_s(self, str(dn), modlist) + + def compare(self, dn, attr, value): + return SimpleLDAPObject.compare(self, str(dn), attr, value) + + def compare_ext(self, dn, attr, value, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.compare_ext(self, str(dn), attr, value, serverctrls, clientctrls) + + def compare_ext_s(self, dn, attr, value, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.compare_ext_s(self, str(dn), attr, value, serverctrls, clientctrls) + + def compare_s(self, dn, attr, value): + return SimpleLDAPObject.compare_s(self, str(dn), attr, value) + + def delete(self, dn): + return SimpleLDAPObject.delete(self, str(dn)) + + def delete_ext(self, dn, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.delete_ext(self, str(dn), serverctrls, clientctrls) + + def delete_ext_s(self, dn, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.delete_ext_s(self, str(dn), serverctrls, clientctrls) + + def delete_s(self, dn): + return SimpleLDAPObject.delete_s(self, str(dn)) + + def modify(self, dn, modlist): + return SimpleLDAPObject.modify(self, str(dn), modlist) + + def modify_ext(self, dn, modlist, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.modify_ext(self, str(dn), modlist, serverctrls, clientctrls) + + def modify_ext_s(self, dn, modlist, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.modify_ext_s(self, str(dn), modlist, serverctrls, clientctrls) + + def modify_s(self, dn, modlist): + return SimpleLDAPObject.modify_s(self, str(dn), modlist) + + def modrdn(self, dn, newrdn, delold=1): + return SimpleLDAPObject.modrdn(self, str(dn), str(newrdn), delold) + + def modrdn_s(self, dn, newrdn, delold=1): + return SimpleLDAPObject.modrdn_s(self, str(dn), str(newrdn), delold) + + def read_subschemasubentry_s(self, subschemasubentry_dn, attrs=None): + return SimpleLDAPObject.read_subschemasubentry_s(self, str(subschemasubentry_dn), attrs) + + def rename(self, dn, newrdn, newsuperior=None, delold=1, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.rename(self, str(dn), str(newrdn), newsuperior, delold, serverctrls, clientctrls) + + def rename_s(self, dn, newrdn, newsuperior=None, delold=1, serverctrls=None, clientctrls=None): + return SimpleLDAPObject.rename_s(self, str(dn), str(newrdn), newsuperior, delold, serverctrls, clientctrls) + + def search(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0): + return SimpleLDAPObject.search(self, str(base), scope, filterstr, attrlist, attrsonly) + + def search_ext(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0, + serverctrls=None, clientctrls=None, timeout=-1, sizelimit=0): + return SimpleLDAPObject.search_ext(self, str(base), scope, filterstr, attrlist, attrsonly, + serverctrls, clientctrls, timeout, sizelimit) + + def search_ext_s(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0, + serverctrls=None, clientctrls=None, timeout=-1, sizelimit=0): + return SimpleLDAPObject.search_ext_s(self, str(base), scope, filterstr, attrlist, attrsonly, + serverctrls, clientctrls, timeout, sizelimit) + + def search_s(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0): + return SimpleLDAPObject.search_s(self, str(base), scope, filterstr, attrlist, attrsonly) + + def search_st(self, base, scope, filterstr='(objectClass=*)', attrlist=None, attrsonly=0, timeout=-1): + return SimpleLDAPObject.search_st(self, str(base), scope, filterstr, attrlist, attrsonly, timeout) + + def search_subschemasubentry_s(self, dn=''): + return SimpleLDAPObject.search_subschemasubentry_s(self, str(dn)) + # universal LDAPError handler def _handle_errors(e, **kw): """ @@ -180,7 +284,7 @@ def get_schema(url, conn=None): raise StandardError('Unable to retrieve LDAP schema. Error initializing principal %s in %s: %s' % (principal.name, '/etc/httpd/conf/ipa.keytab', str(e))) if conn is None: - conn = _ldap.initialize(url) + conn = IPASimpleLDAPObject(url) if url.startswith('ldapi://'): conn.set_option(_ldap.OPT_HOST_NAME, api.env.host) conn.sasl_interactive_bind_s('', SASL_AUTH) @@ -362,7 +466,7 @@ class ldap2(CrudBackend, Encoder): conn.simple_bind_s(bind_dn, bind_pw) except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) if _schema: object.__setattr__(self, 'schema', _schema) @@ -450,7 +554,7 @@ class ldap2(CrudBackend, Encoder): try: self.conn.add_s(dn, list(entry_attrs.iteritems())) except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) # generating filters for find_entry # some examples: @@ -624,7 +728,7 @@ class ldap2(CrudBackend, Encoder): _ldap.SIZELIMIT_EXCEEDED), e: truncated = True except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) if not res: raise errors.NotFound(reason='no such entry') @@ -843,7 +947,7 @@ class ldap2(CrudBackend, Encoder): self.conn.rename_s(dn, new_rdn, delold=int(del_old)) time.sleep(.3) # Give memberOf plugin a chance to work except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) def _generate_modlist(self, dn, entry_attrs, normalize): # get original entry @@ -911,7 +1015,7 @@ class ldap2(CrudBackend, Encoder): try: self.conn.modify_s(dn, modlist) except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) @encode_args(1) def delete_entry(self, dn, normalize=True): @@ -921,7 +1025,7 @@ class ldap2(CrudBackend, Encoder): try: self.conn.delete_s(dn) except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) @encode_args(1, 2, 3) def modify_password(self, dn, new_pass, old_pass=''): @@ -941,7 +1045,7 @@ class ldap2(CrudBackend, Encoder): try: self.conn.passwd_s(dn, old_pass, new_pass) except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) def add_entry_to_group(self, dn, group_dn, member_attr='member', allow_same=False): """ @@ -1156,7 +1260,7 @@ class ldap2(CrudBackend, Encoder): try: self.conn.modify_s(dn, mod) except _ldap.LDAPError, e: - _handle_errors(e, **{}) + _handle_errors(e) # CrudBackend methods -- 1.7.7.3 From mkosek at redhat.com Tue Nov 29 12:38:34 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 Nov 2011 13:38:34 +0100 Subject: [Freeipa-devel] [PATCH] 0037 make-lint fails on Fedora 16/Rawhide In-Reply-To: <20111129115439.GB5725@redhat.com> References: <20111128122851.GB4773@redhat.com> <20111128152609.GA18195@redhat.com> <1322559880.32073.17.camel@balmora.brq.redhat.com> <20111129095224.GA5725@redhat.com> <20111129115439.GB5725@redhat.com> Message-ID: <1322570314.8332.3.camel@balmora.brq.redhat.com> On Tue, 2011-11-29 at 13:54 +0200, Alexander Bokovoy wrote: > On Tue, 29 Nov 2011, Alexander Bokovoy wrote: > > On Tue, 29 Nov 2011, Martin Kosek wrote: > > > > Conservative patch is attached. > > > > > > > > > > What about other add_s(entry) calls? I see we call it this way on more > > > places, especially in replication.py: > > > > > > $ git grep "add_s(e" > > > ipaserver/install/cainstance.py: ld.add_s(entry_dn, entry) > > > ipaserver/install/krbinstance.py: self.admin_conn.add_s(entry) #pylint: disable=E1120 > > > ipaserver/install/krbinstance.py: self.admin_conn.add_s(entry) #pylint: disable=E1120 > > > ipaserver/install/replication.py: conn.add_s(ent) > > > ipaserver/install/replication.py: conn.add_s(entry) > > > ipaserver/install/replication.py: conn.add_s(entry) > > > ipaserver/install/replication.py: self.conn.add_s(entry) > > > ipaserver/install/replication.py: conn.add_s(entry) > > > ipaserver/install/replication.py: a_conn.add_s(entry) > > > ipaserver/install/replication.py: self.conn.add_s(entry) > > > ipaserver/install/service.py: conn.add_s(entry) #pylint: disable=E1120 > > > > > > Should we patch ipa-2-1 branch as well? If we do another release for > > > F-16 we want to have pylint check clean. We would need a rebased patch > > > for ipa-2-1 branch in this case. > > I think this patch also can go away, we found out with Rob yesterday > > night that John has addressed these issues in his patches 48 and 49 in > > September. The patches were ACKed but not merged. I'm doing their > > testing now and will push them after that. > > > > Look at John's 48/49 patches in meanwhile. > Ok, I've tested patches 48 and 49 and also made versions for ipa-2-1 > branch. All attached. > Both pushed to master. As discussed on IRC, we are not sure that these pylint errors would manifest in ipa-2-1 branch, so for the sake of stability I decided to push them to master branch only. Martin From mkosek at redhat.com Tue Nov 29 12:40:10 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 Nov 2011 13:40:10 +0100 Subject: [Freeipa-devel] [PATCH 49/49] ticket #1870 - subclass SimpleLDAPObject In-Reply-To: <4E84DDF0.4050003@redhat.com> References: <201109281927.p8SJRTZV020282@int-mx01.intmail.prod.int.phx2.redhat.com> <4E84DDF0.4050003@redhat.com> Message-ID: <1322570410.8332.4.camel@balmora.brq.redhat.com> On Thu, 2011-09-29 at 15:06 -0600, Rich Megginson wrote: > On 09/28/2011 01:27 PM, John Dennis wrote: > > We use convenience types (classes) in IPA which make working with LDAP > > easier and more robust. It would be really nice if the basic python-ldap > > library understood our utility types and could accept them as parameters > > to the basic ldap functions and/or the basic ldap functions returned our > > utility types. > > > > Normally such a requirement would trivially be handled in an object- > > oriented language (which Python is) by subclassing to extend and modify > > the functionality. For some reason we didn't do this with the python-ldap > > classes. > > > > python-ldap objects are primarily used in two different places in our > > code, ipaserver.ipaldap.py for the IPAdmin class and in > > ipaserver/plugins/ldap2.py for the ldap2 class's .conn member. > > > > In IPAdmin we use a IPA utility class called Entry to make it easier to > > use the results returned by LDAP. The IPAdmin class is derived from > > python-ldap.SimpleLDAPObject. But for some reason when we added the > > support for the use of the Entry class in SimpleLDAPObject we didn't > > subclass SimpleLDAPObject and extend it for use with the Entry class as > > would be the normal expected methodology in an object-oriented language, > > rather we used an obscure feature of the Python language to override all > > methods of the SimpleLDAPObject class by wrapping those class methods in > > another function call. The reason why this isn't a good approach is: > > > > * It violates object-oriented methodology. > > > > * Other classes cannot be derived and inherit the customization (because > > the method wrapping occurs in a class instance, not within the class > > type). > > > > * It's non-obvious and obscure > > > > * It's inefficient. > > > > Here is a summary of what the code was doing: > > > > It iterated over every member of the SimpleLDAPObject class and if it was > > callable it wrapped the method. The wrapper function tested the name of > > the method being wrapped, if it was one of a handful of methods we wanted > > to customize we modified a parameter and called the original method. If > > the method wasn't of interest to use we still wrapped the method. > > > > It was inefficient because every non-customized method (the majority) > > executed a function call for the wrapper, the wrapper during run-time used > > logic to determine if the method was being overridden and then called the > > original method. So every call to ldap was doing extra function calls and > > logic processing which for the majority of cases produced nothing useful > > (and was non-obvious from brief code reading some methods were being > > overridden). > > > > Object-orientated languages have support built in for calling the right > > method for a given class object that do not involve extra function call > > overhead to realize customized class behaviour. Also when programmers look > > for customized class behaviour they look for derived classes. They might > > also want to utilize the customized class as the base class for their use. > > > > Also the wrapper logic was fragile, it did things like: if the method name > > begins with "add" I'll unconditionally modify the first and second > > argument. It would be some much cleaner if the "add", "add_s", etc. > > methods were overridden in a subclass where the logic could be seen and > > where it would apply to only the explicit functions and parameters being > > overridden. > > > > Also we would really benefit if there were classes which could be used as > > a base class which had specific ldap customization. > > > > At the moment our ldap customization needs are: > > > > 1) Support DN objects being passed to ldap operations > > > > 2) Support Entry & Entity objects being passed into and returned from > > ldap operations. > > > > We want to subclass the ldap SimpleLDAPObject class, that is the base > > ldap class with all the ldap methods we're using. IPASimpleLDAPObject > > class would subclass SimpleLDAPObject class which knows about DN > > objects (and possilby other IPA specific types that are universally > > used in IPA). Then IPAEntrySimpleLDAPObject would subclass > > IPASimpleLDAPObject which knows about Entry objects. > > > > The reason for the suggested class hierarchy is because DN objects will be > > used whenever we talk to LDAP (in the future we may want to add other IPA > > specific classes which will always be used). We don't add Entry support to > > the the IPASimpleLDAPObject class because Entry objects are (currently) > > only used in IPAdmin. > > > > What this patch does is: > > > > * Introduce IPASimpleLDAPObject derived from > > SimpleLDAPObject. IPASimpleLDAPObject is DN object aware. > > > > * Introduce IPAEntryLDAPObject derived from > > IPASimpleLDAPObject. IPAEntryLDAPObject is Entry object aware. > > > > * Derive IPAdmin from IPAEntryLDAPObject and remove the funky method > > wrapping from IPAdmin. > > > > * Code which called add_s() with an Entry or Entity object now calls > > addEntry(). addEntry() always existed, it just wasn't always > > used. add_s() had been modified to accept Entry or Entity object > > (why didn't we just call addEntry()?). The add*() ldap routine in > > IPAEntryLDAPObject have been subclassed to accept Entry and Entity > > objects, but that should proably be removed in the future and just > > use addEntry(). > > > > * Replace the call to ldap.initialize() in ldap2.create_connection() > > with a class constructor for IPASimpleLDAPObject. The > > ldap.initialize() is a convenience function in python-ldap, but it > > always returns a SimpleLDAPObject created via the SimpleLDAPObject > > constructor, thus ldap.initialize() did not allow subclassing, yet > > has no particular ease-of-use advantage thus we better off using the > > obvious class constructor mechanism. > > > > * Fix the use of _handle_errors(), it's not necessary to construct an > > empty dict to pass to it. > > > > If we follow the standard class derivation pattern for ldap we can make us > > of our own ldap utilities in a far easier, cleaner and more efficient > > manner. > ack Pushed to master (additional information in a thread for Alexander's patch 0037). Martin From mkosek at redhat.com Tue Nov 29 12:40:47 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 Nov 2011 13:40:47 +0100 Subject: [Freeipa-devel] [PATCH 48/48] Ticket #1879 - IPAdmin undefined anonymous parameter lists In-Reply-To: <4E8AFB5F.6080502@redhat.com> References: <201109261952.p8QJqjVo016911@int-mx09.intmail.prod.int.phx2.redhat.com> <4E8AFB5F.6080502@redhat.com> Message-ID: <1322570447.8332.5.camel@balmora.brq.redhat.com> On Tue, 2011-10-04 at 14:26 +0200, Jan Cholasta wrote: > On 26.9.2011 21:52, John Dennis wrote: > > The IPAdmin class in ipaserver/ipaldap.py has methods with anonymous > > undefined parameter lists. > > > > For example: > > > > def getList(self,*args): > > > > In Python syntax this means you can call getList with any positional > > parameter list you want. > > > > This is bad because: > > > > 1) It's not true, *args gets passed to an ldap function with a well > > defined parameter list, so you really do have to call it with a > > defined parameter list. *args will let you pass anything, but once it > > gets passed to the ldap function it will blow up if the parameters do > > not match (what parameters are those you're wondering? see item 2). > > > > 2) The programmer does not know what the valid parameters are unless > > they are defined in the formal parameter list. > > > > 3) Without a formal parameter list automatic documentation generators > > cannot produce API documentation (see item 2) > > > > 4) The Python interpreter cannot validate the parameters being passed > > because there is no formal parameter list. Note, Python does not > > validate the type of parameters, but it does validate the correct > > number of postitional parameters are passed and only defined keyword > > parameters are passed. Bypassing the language support facilities leads > > to programming errors. > > > > 5) Without a formal parameter list program checkers such as pylint > > cannot validate the program which leads to progamming errors. > > > > 6) Without a formal parameter list which includes default keyword > > parameters it's not possible to use keyword arguments nor to know what > > their default values are (see item 2). One is forced to pass a keyword > > argument as a positional argument, plus you must then pass every > > keyword argument between the end of the positional argument list and > > keyword arg of interest even of the other keyword arguments are not of > > interest. This also demands you know what the default value of the > > intermediate keyword arguments are (see item 2) and hope they don't > > change. > > > > Also the *args anonymous tuple get passed into the error handling code > > so it can report what the called values were. But because the tuple is > > anonymous the error handler cannot not describe what it was passed. In > > addition the error handling code makes assumptions about the possible > > contents of the anonymous tuple based on current practice instead of > > actual defined values. Things like "if the number of items in the > > tuple is 2 or less then the first tuple item must be a dn > > (Distinguished Name)" or "if the number of items in the tuple is > > greater than 2 then the 3rd item must be an ldap search filter". These > > are constructs which are not robust and will fail at some point in the > > future. > > > > This patch also fixes the use of IPAdmin.addEntry(). It was sometimes > > being called with (dn, modlist), sometimes a Entry object, or > > sometimes a Entity object. Now it's always called with either a Entry > > or Entity object and IPAdmin.addEntry() validates the type of the > > parameter passed. > > > > -- > > John Dennis > > > > Looking to carve out IT costs? > > www.redhat.com/carveoutcosts/ > > > > ACK. > > Honza > Pushed to master (additional information in a thread for Alexander's patch 0037). Martin From mkosek at redhat.com Tue Nov 29 13:16:12 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 Nov 2011 14:16:12 +0100 Subject: [Freeipa-devel] [PATCH] 8 Add DNS service records for Windows In-Reply-To: <20111128171634.GD2246@localhost.localdomain> References: <20111014101557.GC4622@localhost.localdomain> <20111014151202.GD4622@localhost.localdomain> <20111014172150.GA17515@redhat.com> <20111014191648.GE4622@localhost.localdomain> <20111121083211.GD13007@redhat.com> <4ECD74C6.5080602@redhat.com> <20111125171231.GA2246@localhost.localdomain> <20111128122600.GA4773@redhat.com> <20111128171634.GD2246@localhost.localdomain> Message-ID: <1322572572.8332.8.camel@balmora.brq.redhat.com> On Mon, 2011-11-28 at 18:16 +0100, Sumit Bose wrote: > On Mon, Nov 28, 2011 at 02:26:00PM +0200, Alexander Bokovoy wrote: > > On Fri, 25 Nov 2011, Sumit Bose wrote: > > > On Wed, Nov 23, 2011 at 05:33:42PM -0500, Rob Crittenden wrote: > > > > Alexander Bokovoy wrote: > > > > >Hi Sumit, > > > > > > > > > >On Fri, 14 Oct 2011, Sumit Bose wrote: > > > > >>>It would make more clear what is the default and that it is really > > > > >>>optional setting -- I'm thinking from the perspective of maintenance > > > > >>>of the code in future. > > > > >> > > > > >>Thank you for your comments, new version attached. > > > > >Finally got to test it. ACK. > > > > > > > > > > > > > pushed to master. > > > > > > Sorry, I think you pushed the first version and not -3- which was ACKed > > > by Alexander. > > Hm. Sumit, could you please rebase -3- on top of current master > > HEAD+(1)? > > > > I tried that briefly myself and failed. > > > > (1) Right now I'm also getting make-lint failing due to more strict > > PyLint in F16/Rawhide and those seems to be corect and also affect > > adtrustinstance. > > > > I'm sending the patch shortly, so please rebase on top of it. > > ok, rebased version is attached. To push this upstream you still have to > revert the wrong commit. > > bye, > Sumit I reverted the wrong version of patch so that Sumit can provide correct rebased version: master: ac45a5eee8382ab62b7c8b7c78c2bca36e5bf8a6 Martin From rcritten at redhat.com Tue Nov 29 14:09:45 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 29 Nov 2011 09:09:45 -0500 Subject: [Freeipa-devel] [PATCH] 908 make some fields required In-Reply-To: <4ED410C2.7000905@redhat.com> References: <4ED3CE6A.9000804@redhat.com> <4ED410C2.7000905@redhat.com> Message-ID: <4ED4E7A9.7060705@redhat.com> Endi Sukma Dewata wrote: > On 11/28/2011 12:09 PM, Rob Crittenden wrote: >> Some attributes in the framework were not marked as required even though >> they are in the schema. These are typically computed values and I think >> the intention was to not prompt for them. This has the side-effect of >> them showing as not required in the UI even though they are. >> >> Since they all have default values set they won't be prompted for on the >> CLI so there won't be any practical changes. > > This patch fixes the problem with required attributes in DNS Zones and > cn, uidNumber, and gidNumber in Users. The UI now shows the required > indicators for these attributes. So this patch is ACKed. > > Some problems mentioned in ticket #2015 are still present: > > 1. Removing the homeDirectory from a user fails because it's required by > posixAccount. Ok, I can probably fix this one. > 2. Removing the gidNumber from a group fails because it's required by > posixGroup. Well, its hard to make gidNumber required since you can have non-posix groups. I think we'll have to just live with this one. > > 3. Removing config attributes listed in the ticket generates internal > error. I think at least the server should return a proper error message. > The required indicator can be hard-coded in the UI if necessary. > The config problems are a separate issue. To make them required would mean that the user would need to provide a value with a -mod request. rob From simo at redhat.com Tue Nov 29 14:32:30 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 Nov 2011 09:32:30 -0500 Subject: [Freeipa-devel] [PATCH] #2122 Fix PAC re-signing In-Reply-To: <20111129091311.GG2246@localhost.localdomain> References: <1322007054.6750.76.camel@willson.li.ssimo.org> <20111123105311.GG2205@localhost.localdomain> <20111124125419.GK2205@localhost.localdomain> <1322527437.2613.42.camel@willson.li.ssimo.org> <20111129091311.GG2246@localhost.localdomain> Message-ID: <1322577150.2613.44.camel@willson.li.ssimo.org> On Tue, 2011-11-29 at 10:13 +0100, Sumit Bose wrote: > On Mon, Nov 28, 2011 at 07:43:57PM -0500, Simo Sorce wrote: > > On Thu, 2011-11-24 at 13:54 +0100, Sumit Bose wrote: > > > I think I found two issues which should be fixed by the following > > > patch: > > > - krb5_pac_add_buffer() expects krb5_pac and not krb5_pac * as a > > > second > > > argument > > > > good catch > > > > > - your patch copies all buffers, including the checksums, which you > > > wanted to remove from the new pac > > > > also good catch > > > > > With this patch applied I do not see any errors in the krb5kdc.log and > > > ssh from AD to IPA server works. > > > > I still haven't solved my ssh issue from an AD client to IPA, but I get > > a ticket on the client now, so it must be unrelated stuff. > > > > I have prepared a patch which have a slightly different version of your > > fixes. > > The patch looks fine and works for me. > > ACK Thanks, pushed to master. Simo. -- Simo Sorce * Red Hat, Inc * New York From ohamada at redhat.com Tue Nov 29 14:46:09 2011 From: ohamada at redhat.com (Ondrej Hamada) Date: Tue, 29 Nov 2011 15:46:09 +0100 Subject: [Freeipa-devel] [PATCH] 6 Sort password policy by priority Message-ID: <4ED4F031.9030408@redhat.com> https://fedorahosted.org/freeipa/ticket/2045 'ipa pwpolicy-find' output is now sorted by priority of the policies. Lower position means lower priority. Global policy is then at the bottom. The changes has also affected LDAPSearch class in baseldap.py: LDAPSearch class sorts the search results by primary key be default (which is usually 'cn'). Therefor a function pointer entries_sortfn was added. If no sorting function exists, default sorting by primary key is used. Sorting function had to be introduced due to the fact that pwpolicy's primary key is also it's 'cn' and global policy is not allowed to have any priority. -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada From simo at redhat.com Tue Nov 29 14:46:47 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 Nov 2011 09:46:47 -0500 Subject: [Freeipa-devel] [PATCH] spec file fix Message-ID: <1322578007.2613.55.camel@willson.li.ssimo.org> Pushed the following patch under one-liner. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-spec-We-do-not-need-krb5-server-ldap-anymore.patch Type: text/x-patch Size: 731 bytes Desc: not available URL: From ohamada at redhat.com Tue Nov 29 15:05:37 2011 From: ohamada at redhat.com (Ondrej Hamada) Date: Tue, 29 Nov 2011 16:05:37 +0100 Subject: [Freeipa-devel] [PATCH] 6 Sort password policy by priority In-Reply-To: <4ED4F031.9030408@redhat.com> References: <4ED4F031.9030408@redhat.com> Message-ID: <4ED4F4C1.90806@redhat.com> On 11/29/2011 03:46 PM, Ondrej Hamada wrote: > https://fedorahosted.org/freeipa/ticket/2045 > > 'ipa pwpolicy-find' output is now sorted by priority of the policies. > Lower position means lower priority. Global policy is then at the bottom. > > The changes has also affected LDAPSearch class in baseldap.py: > LDAPSearch class sorts the search results by primary key be default > (which is usually 'cn'). Therefor a function pointer entries_sortfn > was added. If no sorting function exists, default sorting by primary key > is used. > > Sorting function had to be introduced due to the fact that pwpolicy's > primary > key is also it's 'cn' and global policy is not allowed to have any > priority. > forget to attach the patch, sorry :-[ -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ohamada-6-Sort-password-policy-by-priority.patch Type: text/x-patch Size: 3395 bytes Desc: not available URL: From simo at redhat.com Tue Nov 29 15:10:57 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 Nov 2011 10:10:57 -0500 Subject: [Freeipa-devel] [PATCH] 265 Fixed layout problem in permission adder dialog. In-Reply-To: <4E69CC01.504@redhat.com> References: <4E67F305.9060001@redhat.com> <4E68E9B0.9080500@redhat.com> <4E68F2A5.7010901@redhat.com> <4E69CC01.504@redhat.com> Message-ID: <1322579457.2613.57.camel@willson.li.ssimo.org> Apparently this patch was pushed. On Fri, 2011-09-09 at 10:19 +0200, Petr Vobornik wrote: > On 09/08/2011 06:51 PM, Endi Sukma Dewata wrote: > > On 9/8/2011 11:13 AM, Petr Vobornik wrote: > >> In IPA.details_table_section: > >> 1)not renamed list_section_create method > > > > Fixed. > > > >> Code clean-up in aci.js: > >> 2) IPA.rights_section can be deleted and replaced by spec object usage. > >> It doesn't add any functionality. > > > > Fixed. > > > >> 3) IPA.permission_details_facet can be deleted - it isn't used anywhere. > > > > Fixed. > > > > ACK > -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Tue Nov 29 15:11:09 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 Nov 2011 10:11:09 -0500 Subject: [Freeipa-devel] [PATCH] 266 Fixed sudo rule association dialogs. In-Reply-To: <4E69F0A6.3000501@redhat.com> References: <4E68DF22.6010805@redhat.com> <4E68F31A.7060809@redhat.com> <4E69F0A6.3000501@redhat.com> Message-ID: <1322579469.2613.58.camel@willson.li.ssimo.org> This patch too seem to have been pushed. On Fri, 2011-09-09 at 12:55 +0200, Petr Vobornik wrote: > On 09/08/2011 06:53 PM, Endi Sukma Dewata wrote: > > On 9/8/2011 10:28 AM, Endi Sukma Dewata wrote: > >> The adder dialog for the user and host tables in sudo rule details > >> page have been fixed to use --not-in-sudorules to avoid showing > >> entries that are already added into the rule either directly or > >> indirectly via groups. > >> > >> This does not apply to the command and run-as tables because they > >> do not support such option. > >> > >> Ticket #1768 > > > > Wrong email title. It should be patch #266. > > > ACK > -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Tue Nov 29 15:12:37 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 Nov 2011 10:12:37 -0500 Subject: [Freeipa-devel] [PATCH] 286 Fixed tab and dialog widths. In-Reply-To: <4E84387A.1040405@redhat.com> References: <4E8255EE.6040700@redhat.com> <4E84387A.1040405@redhat.com> Message-ID: <1322579557.2613.59.camel@willson.li.ssimo.org> This patch was pushed too. (I am closing loose ends) On Thu, 2011-09-29 at 11:20 +0200, Petr Vobornik wrote: > On 09/28/2011 01:02 AM, Endi Sukma Dewata wrote: > > The width of the 1st level tab has been modified to expand according > > to the size of the tab label. > > > > The width of the adder dialogs have been increased to allow longer > > button labels. > > > > Ticket #1825 > ACK > -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Tue Nov 29 15:14:04 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 Nov 2011 10:14:04 -0500 Subject: [Freeipa-devel] [PATCH] 275 Use editable combobox for service type. In-Reply-To: <4E843BE6.4010100@redhat.com> References: <4E738466.3060705@redhat.com> <4E825558.6060509@redhat.com> <4E843BE6.4010100@redhat.com> Message-ID: <1322579644.2613.60.camel@willson.li.ssimo.org> This patch was also pushed. On Thu, 2011-09-29 at 11:35 +0200, Petr Vobornik wrote: > On 09/28/2011 12:59 AM, Endi Sukma Dewata wrote: > > On 9/16/2011 12:16 PM, Endi Sukma Dewata wrote: > >> The service type field in the service adder dialog has been modified > >> to use an editable combobox. > >> > >> Ticket #1633. > > > > Rebased (removed undo parameter). > > ACK -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Tue Nov 29 15:14:55 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 Nov 2011 10:14:55 -0500 Subject: [Freeipa-devel] [PATCH] 288 Disable enroll button if nothing selected. In-Reply-To: <4E845DAC.8040802@redhat.com> References: <4E83A4AD.6060300@redhat.com> <4E83C30A.8040007@redhat.com> <4E83F931.5020606@redhat.com> <4E845DAC.8040802@redhat.com> Message-ID: <1322579695.2613.61.camel@willson.li.ssimo.org> This patch was also pushed. On Thu, 2011-09-29 at 13:59 +0200, Petr Vobornik wrote: > On 09/29/2011 06:50 AM, Endi Sukma Dewata wrote: > > On 9/28/2011 7:59 PM, Adam Young wrote: > >> On 09/28/2011 06:50 PM, Endi Sukma Dewata wrote: > >>> A new IPA.dialog_button class has been added to encapsulate the > >>> buttons in the dialog box so they can be managed more easily. > >>> > >>> The adder dialog has been modified to disable the enroll button if > >>> there is no entries selected. > >>> > >>> Ticket #1856 > > > > The solution is simple, but it requires refactoring which probably > > should have been done much earlier. So this patch is like a combination > > of several patches. > > > > > As a further enhancement, the IPA.dialog_button class probably can be > > combined with the IPA.action_button/IPA.button class so we can use icons > > or enable/disable buttons in a more consistent way. This is one step in > > that direction. > I Agree. This would be nice especially in association table widget. > > ACK > > -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Tue Nov 29 15:17:07 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 Nov 2011 10:17:07 -0500 Subject: [Freeipa-devel] [PATCH] 016 Fixed: Some widgets do not have space for validation error message In-Reply-To: <4E846A1A.3080100@redhat.com> References: <4E709A20.8070301@redhat.com> <4E709CBD.7040009@redhat.com> <4E713E0E.80609@redhat.com> <4E81C67A.2000201@redhat.com> <4E823818.5050209@redhat.com> <4E846A1A.3080100@redhat.com> Message-ID: <1322579827.2613.62.camel@willson.li.ssimo.org> I couldn't find a clear ACK for this patch but it has been pushed as well. Guys please try to better report on the list when patches are pushed and close the thread. Simo. On Thu, 2011-09-29 at 14:52 +0200, Petr Vobornik wrote: > On 09/27/2011 10:54 PM, Endi Sukma Dewata wrote: > > On 9/27/2011 7:50 AM, Petr Vobornik wrote: > >> I've added padding and unified font-weight in error-message style. The > >> padding is added because text of the message was right on the border > >> which wasn't much readable. Font-weight is added because sometimes it > >> inherits font-weight from other style and so it is inconsistent with > >> other appearances. I'm not sure about the 'bold', though. > > > > This is good but can we use a new CSS class for the error message? I'd > > rather not change the jquery-ui.css (removing white spaces is ok) > > because it will be difficult to keep track the changes if we need to > > upgrade it. > Moved this change to ipa.css (no need to add extra class, it will > overwrite it as long ipa.css is linked after jquery-ui.css). > > > > There's another minor issue, but this one can be fixed separately if you > > want. Try editing a self-service permission, uncheck all attributes, > > then click Update. The undo link and the error message appear in 2 > > separate lines. > Kept them on 2 lines. > > I think it's better to show them in a single line. When > > we fix the attributes widget to inherit from table, they will fit into > > the footer. > I agree. > > The padding should be consistent as well, right now the undo > > box is smaller than the error message. > Added padding to undo. Added 1px margin-bottom to undo to separate undo > and error message borders when they are on two lines. > > Changed display of undo in attributes_widget from inline to inline-block > - looks better with added padding. > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Tue Nov 29 15:18:52 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 Nov 2011 10:18:52 -0500 Subject: [Freeipa-devel] [PATCH] 888 always verify hostname In-Reply-To: <1318345671.2965.1.camel@balmora.brq.redhat.com> References: <4E8E6AF4.8060302@redhat.com> <1317979530.8399.26.camel@dhcp-25-52.brq.redhat.com> <4E8EF766.2070706@redhat.com> <1317992800.8399.43.camel@dhcp-25-52.brq.redhat.com> <4E8EFC09.9030301@redhat.com> <1318345671.2965.1.camel@balmora.brq.redhat.com> Message-ID: <1322579932.2613.63.camel@willson.li.ssimo.org> On Tue, 2011-10-11 at 17:07 +0200, Martin Kosek wrote: > On Fri, 2011-10-07 at 09:18 -0400, Rob Crittenden wrote: > > Martin Kosek wrote: > > >> > > >> Yes but the entry is added /etc/hosts at the very END of installation, > > >> apparently too late for some things. We can alternately add this prior > > >> to configuring anything else. > > > > > > But we add the entry to /etc/hosts right in the beginning. After the > > > line marked with<<<<<< is printed. I double-checked it right now. > > > > Ok, this is totally freaky then. See ticket > > https://fedorahosted.org/freeipa/ticket/1931 > > > > > I think it is worth mentioning there that the /etc/hosts entry is added > in the beginning only if the hostname is not resolvable and IP address > is passed by the user, i.e. only when the following line printed: > > # ipa-server-install --setup-dns (or --no-host-dns) > ... > Please provide the IP address to be used for this host name: 10.16.78.50 > Adding [10.16.78.50 ipa.example.com] to your /etc/hosts file > ... > > I saw that 1931 should be solved by a new custom hostname parameter > passed to bind-dyndb-ldap plugin. > > > I did some additional testing of my proposed patch 140 and it behaved > fine. It is able to catch misconfigured /etc/hosts in both following ways: > > 1) invalid hostname for given IP address > > 1.2.3.4 foo > > or short name first: > > 1.2.3.4 foo foo.example.com > > > To sum this up - I think the patch is ready for review. What's the status of this patch ? Simo. -- Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Tue Nov 29 15:38:36 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 Nov 2011 16:38:36 +0100 Subject: [Freeipa-devel] [PATCH] 888 always verify hostname In-Reply-To: <1322579932.2613.63.camel@willson.li.ssimo.org> References: <4E8E6AF4.8060302@redhat.com> <1317979530.8399.26.camel@dhcp-25-52.brq.redhat.com> <4E8EF766.2070706@redhat.com> <1317992800.8399.43.camel@dhcp-25-52.brq.redhat.com> <4E8EFC09.9030301@redhat.com> <1318345671.2965.1.camel@balmora.brq.redhat.com> <1322579932.2613.63.camel@willson.li.ssimo.org> Message-ID: <1322581116.8332.30.camel@balmora.brq.redhat.com> On Tue, 2011-11-29 at 10:18 -0500, Simo Sorce wrote: > On Tue, 2011-10-11 at 17:07 +0200, Martin Kosek wrote: > > On Fri, 2011-10-07 at 09:18 -0400, Rob Crittenden wrote: > > > Martin Kosek wrote: > > > >> > > > >> Yes but the entry is added /etc/hosts at the very END of installation, > > > >> apparently too late for some things. We can alternately add this prior > > > >> to configuring anything else. > > > > > > > > But we add the entry to /etc/hosts right in the beginning. After the > > > > line marked with<<<<<< is printed. I double-checked it right now. > > > > > > Ok, this is totally freaky then. See ticket > > > https://fedorahosted.org/freeipa/ticket/1931 > > > > > > > > I think it is worth mentioning there that the /etc/hosts entry is added > > in the beginning only if the hostname is not resolvable and IP address > > is passed by the user, i.e. only when the following line printed: > > > > # ipa-server-install --setup-dns (or --no-host-dns) > > ... > > Please provide the IP address to be used for this host name: 10.16.78.50 > > Adding [10.16.78.50 ipa.example.com] to your /etc/hosts file > > ... > > > > I saw that 1931 should be solved by a new custom hostname parameter > > passed to bind-dyndb-ldap plugin. > > > > > > I did some additional testing of my proposed patch 140 and it behaved > > fine. It is able to catch misconfigured /etc/hosts in both following ways: > > > > 1) invalid hostname for given IP address > > > > 1.2.3.4 foo > > > > or short name first: > > > > 1.2.3.4 foo foo.example.com > > > > > > To sum this up - I think the patch is ready for review. > > What's the status of this patch ? > > Simo. > All patches related to this topic has been reviewed, acked and pushed in different thread named "[PATCH] 140 + 148 + 147 Hostname fixes". Both relevant tickets has been fixed with these 3 patches: https://fedorahosted.org/freeipa/ticket/1923 https://fedorahosted.org/freeipa/ticket/1931 But you are right that no information that the discussion continues somewhere else was given here. Martin From rcritten at redhat.com Tue Nov 29 15:39:47 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 29 Nov 2011 10:39:47 -0500 Subject: [Freeipa-devel] [PATCH] 166+167 Zonemgr improvements In-Reply-To: <1322061461.27504.20.camel@balmora.brq.redhat.com> References: <1321021336.4215.15.camel@balmora.brq.redhat.com> <4EBD3DA0.6050908@redhat.com> <1321263429.4805.7.camel@balmora.brq.redhat.com> <1322061461.27504.20.camel@balmora.brq.redhat.com> Message-ID: <4ED4FCC3.2010509@redhat.com> Martin Kosek wrote: > On Mon, 2011-11-14 at 10:37 +0100, Martin Kosek wrote: >> On Fri, 2011-11-11 at 10:22 -0500, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> There are 2 patches improving our zone zonemgr: >>>> >>>> 1) ipa-server-install --zonemgr option validation and normalization + >>>> the same thing in ipa dnszone-add/mod --admin-email. They now allow and >>>> correctly process '.' in a local-part of the zonemgr e-mail (it is >>>> encoded as '\.'. >>>> >>>> How to test: >>>> >>>> ipa-server-install -p secret123 -a secret123 --setup-dns --zonemgr=foo.bar at example.com >>>> OR if e-mail is passed in SOA format: >>>> ipa-server-install -p secret123 -a secret123 --setup-dns --zonemgr='foo\.bar.example.com' >>>> >>>> In both cases, the zonemgr e-mail will be set to correct format: 'foo >>>> \.bar.example.com'. >>>> >>>> 2) Our default zonemgr is changed to follow RFC 2142 recommendation - >>>> hostmaster@ >>>> >>>> hostmaster is an alias to root anyway (see /etc/aliases). >>>> >>>> Martin >>> >>> NACK 166, this illegal address is allowed: foo\.bar\.baz\.com >>> >>> ACK 167 when 166 is ready. >>> >>> rob >> >> Are you sure that you quoted the string in shell properly? It likes to >> eat backslashes when one is not cautious. The zonemgr value, including >> backslashes, should then show up in `ipa dnszone-show ZONE'. >> >> This is my output: >> >> # ipa dnszone-mod example.com --admin-email='foo\.bar\.baz\.com' >> ipa: ERROR: invalid 'admin_email': address domain is not fully qualified >> ("example.com" instead of just "example") >> >> Anyway, attaching a rebased patch (it collided with my patch 120). >> >> Martin > > I rebased both patches. I also fixed a bug when ipa-replica-prepare > $HOST --ip-address=$IP was failing because None was passed as zonemgr. > > Martin $ ./make-lint ipaserver/install/bindinstance.py:392: [E0602, BindInstance.setup] Undefined variable 'normalize_zonemgr' Fixed with: --- a/ipaserver/install/bindinstance.py +++ b/ipaserver/install/bindinstance.py @@ -32,7 +32,7 @@ from ipapython import sysrestore from ipapython import ipautil from ipalib.constants import DNS_ZONE_REFRESH from ipalib.parameters import IA5Str -from ipalib.util import validate_zonemgr +from ipalib.util import validate_zonemgr, normalize_zonemgr from ipapython.ipa_log_manager import * import ipalib ACK for both with this import fixed. rob From edewata at redhat.com Tue Nov 29 16:17:10 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 29 Nov 2011 10:17:10 -0600 Subject: [Freeipa-devel] [PATCH] 016 Fixed: Some widgets do not have space for validation error message In-Reply-To: <1322579827.2613.62.camel@willson.li.ssimo.org> References: <4E709A20.8070301@redhat.com> <4E709CBD.7040009@redhat.com> <4E713E0E.80609@redhat.com> <4E81C67A.2000201@redhat.com> <4E823818.5050209@redhat.com> <4E846A1A.3080100@redhat.com> <1322579827.2613.62.camel@willson.li.ssimo.org> Message-ID: <4ED50586.5090907@redhat.com> On 11/29/2011 9:17 AM, Simo Sorce wrote: > I couldn't find a clear ACK for this patch but it has been pushed as > well. Correct. > Guys please try to better report on the list when patches are pushed and > close the thread. Yes, sorry, sometimes I missed a few, my fault. -- Endi S. Dewata From mkosek at redhat.com Tue Nov 29 16:22:00 2011 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 Nov 2011 17:22:00 +0100 Subject: [Freeipa-devel] [PATCH] 166+167 Zonemgr improvements In-Reply-To: <4ED4FCC3.2010509@redhat.com> References: <1321021336.4215.15.camel@balmora.brq.redhat.com> <4EBD3DA0.6050908@redhat.com> <1321263429.4805.7.camel@balmora.brq.redhat.com> <1322061461.27504.20.camel@balmora.brq.redhat.com> <4ED4FCC3.2010509@redhat.com> Message-ID: <1322583720.8332.36.camel@balmora.brq.redhat.com> On Tue, 2011-11-29 at 10:39 -0500, Rob Crittenden wrote: > Martin Kosek wrote: > > On Mon, 2011-11-14 at 10:37 +0100, Martin Kosek wrote: > >> On Fri, 2011-11-11 at 10:22 -0500, Rob Crittenden wrote: > >>> Martin Kosek wrote: > >>>> There are 2 patches improving our zone zonemgr: > >>>> > >>>> 1) ipa-server-install --zonemgr option validation and normalization + > >>>> the same thing in ipa dnszone-add/mod --admin-email. They now allow and > >>>> correctly process '.' in a local-part of the zonemgr e-mail (it is > >>>> encoded as '\.'. > >>>> > >>>> How to test: > >>>> > >>>> ipa-server-install -p secret123 -a secret123 --setup-dns --zonemgr=foo.bar at example.com > >>>> OR if e-mail is passed in SOA format: > >>>> ipa-server-install -p secret123 -a secret123 --setup-dns --zonemgr='foo\.bar.example.com' > >>>> > >>>> In both cases, the zonemgr e-mail will be set to correct format: 'foo > >>>> \.bar.example.com'. > >>>> > >>>> 2) Our default zonemgr is changed to follow RFC 2142 recommendation - > >>>> hostmaster@ > >>>> > >>>> hostmaster is an alias to root anyway (see /etc/aliases). > >>>> > >>>> Martin > >>> > >>> NACK 166, this illegal address is allowed: foo\.bar\.baz\.com > >>> > >>> ACK 167 when 166 is ready. > >>> > >>> rob > >> > >> Are you sure that you quoted the string in shell properly? It likes to > >> eat backslashes when one is not cautious. The zonemgr value, including > >> backslashes, should then show up in `ipa dnszone-show ZONE'. > >> > >> This is my output: > >> > >> # ipa dnszone-mod example.com --admin-email='foo\.bar\.baz\.com' > >> ipa: ERROR: invalid 'admin_email': address domain is not fully qualified > >> ("example.com" instead of just "example") > >> > >> Anyway, attaching a rebased patch (it collided with my patch 120). > >> > >> Martin > > > > I rebased both patches. I also fixed a bug when ipa-replica-prepare > > $HOST --ip-address=$IP was failing because None was passed as zonemgr. > > > > Martin > > > $ ./make-lint > ipaserver/install/bindinstance.py:392: [E0602, BindInstance.setup] > Undefined variable 'normalize_zonemgr' > > Fixed with: > > --- a/ipaserver/install/bindinstance.py > +++ b/ipaserver/install/bindinstance.py > @@ -32,7 +32,7 @@ from ipapython import sysrestore > from ipapython import ipautil > from ipalib.constants import DNS_ZONE_REFRESH > from ipalib.parameters import IA5Str > -from ipalib.util import validate_zonemgr > +from ipalib.util import validate_zonemgr, normalize_zonemgr > from ipapython.ipa_log_manager import * > > import ipalib > > ACK for both with this import fixed. > > rob Good catch. This must have been some late fix again. Fixed. Both patches pushed to master. Martin From rcritten at redhat.com Tue Nov 29 19:43:09 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 29 Nov 2011 14:43:09 -0500 Subject: [Freeipa-devel] [PATCH] 6 Sort password policy by priority In-Reply-To: <4ED4F4C1.90806@redhat.com> References: <4ED4F031.9030408@redhat.com> <4ED4F4C1.90806@redhat.com> Message-ID: <4ED535CD.2090709@redhat.com> Ondrej Hamada wrote: > On 11/29/2011 03:46 PM, Ondrej Hamada wrote: >> https://fedorahosted.org/freeipa/ticket/2045 >> >> 'ipa pwpolicy-find' output is now sorted by priority of the policies. >> Lower position means lower priority. Global policy is then at the bottom. >> >> The changes has also affected LDAPSearch class in baseldap.py: >> LDAPSearch class sorts the search results by primary key be default >> (which is usually 'cn'). Therefor a function pointer entries_sortfn >> was added. If no sorting function exists, default sorting by primary key >> is used. >> >> Sorting function had to be introduced due to the fact that pwpolicy's >> primary >> key is also it's 'cn' and global policy is not allowed to have any >> priority. >> > forget to attach the patch, sorry :-[ > Doesn't work with the lite-server due to changing a ReadOnly value. I think you have the right idea you just need to assign the sorting statically instead of dynamically (it won't change after finalization). $ ./ipa pwpolicy-add editors --minlife=2 --priority=9 ipa: DEBUG: Created connection context.ldap2 ipa: DEBUG: raw: pwpolicy_add(u'editors', krbminpwdlife=2, cospriority=9, all=False, raw=False, version=u'2.15') ipa: DEBUG: pwpolicy_add(u'editors', krbminpwdlife=2, cospriority=9, all=False, raw=False, version=u'2.15') ipa: DEBUG: raw: cosentry_add(u'editors', krbpwdpolicyreference=u'cn=editors,cn=GREYOAK.COM,cn=kerberos,dc=greyoak,dc=com', cospriority=9) ipa: DEBUG: cosentry_add(u'editors', krbpwdpolicyreference=u'cn=editors,cn=GREYOAK.COM,cn=kerberos,dc=greyoak,dc=com', cospriority=9, all=False, raw=False) ipa: DEBUG: raw: group_show(u'editors', all=True) ipa: DEBUG: group_show(u'editors', rights=False, all=True, raw=False) ipa: DEBUG: raw: cosentry_find(None, cospriority=9) ipa: DEBUG: cosentry_find(None, cospriority=9, all=False, raw=False, pkey_only=False) ipa: ERROR: non-public: AttributeError: locked: cannot set cosentry_find.entries_sortfn to at 0x7f7abc33ec80> Traceback (most recent call last): File "/home/rcrit/redhat/freeipa-review/ipaserver/rpcserver.py", line 223, in wsgi_execute result = self.Command[name](*args, **options) File "/home/rcrit/redhat/freeipa-review/ipalib/frontend.py", line 438, in __call__ ret = self.run(*args, **options) File "/home/rcrit/redhat/freeipa-review/ipalib/frontend.py", line 756, in run return self.execute(*args, **options) File "/home/rcrit/redhat/freeipa-review/ipalib/plugins/baseldap.py", line 700, in execute ldap, dn, entry_attrs, attrs_list, *keys, **options File "/home/rcrit/redhat/freeipa-review/ipalib/plugins/pwpolicy.py", line 346, in pre_callback cospriority=options.get('cospriority') File "/home/rcrit/redhat/freeipa-review/ipalib/frontend.py", line 438, in __call__ ret = self.run(*args, **options) File "/home/rcrit/redhat/freeipa-review/ipalib/frontend.py", line 756, in run return self.execute(*args, **options) File "/home/rcrit/redhat/freeipa-review/ipalib/plugins/baseldap.py", line 700, in execute ldap, dn, entry_attrs, attrs_list, *keys, **options File "/home/rcrit/redhat/freeipa-review/ipalib/plugins/pwpolicy.py", line 127, in pre_callback self.obj.check_priority_uniqueness(*keys, **options) File "/home/rcrit/redhat/freeipa-review/ipalib/plugins/pwpolicy.py", line 101, in check_priority_uniqueness cospriority=options['cospriority'] File "/home/rcrit/redhat/freeipa-review/ipalib/frontend.py", line 438, in __call__ ret = self.run(*args, **options) File "/home/rcrit/redhat/freeipa-review/ipalib/frontend.py", line 756, in run return self.execute(*args, **options) File "/home/rcrit/redhat/freeipa-review/ipalib/plugins/baseldap.py", line 1620, in execute self.entries_sortfn=lambda x,y: cmp(x[1][self.obj.primary_key.name][0].lower(), y[1][self.obj.primary_key.name][0].lower()) File "/home/rcrit/redhat/freeipa-review/ipalib/base.py", line 131, in __setattr__ SET_ERROR % (self.__class__.__name__, name, value) AttributeError: locked: cannot set cosentry_find.entries_sortfn to at 0x7f7abc33ec80> ipa: INFO: admin at GREYOAK.COM: pwpolicy_add(u'editors', krbminpwdlife=2, cospriority=9, all=False, raw=False, version=u'2.15'): AttributeError ipa: DEBUG: response: InternalError: an internal error has occurred ipa: DEBUG: Destroyed connection context.ldap2 From sbose at redhat.com Tue Nov 29 21:09:26 2011 From: sbose at redhat.com (Sumit Bose) Date: Tue, 29 Nov 2011 22:09:26 +0100 Subject: [Freeipa-devel] [PATCH] Add ipasam samba passdb backend In-Reply-To: <20111128172227.GE2246@localhost.localdomain> References: <20111123154802.GI2205@localhost.localdomain> <20111128172227.GE2246@localhost.localdomain> Message-ID: <20111129210926.GK2246@localhost.localdomain> On Mon, Nov 28, 2011 at 06:22:27PM +0100, Sumit Bose wrote: > On Wed, Nov 23, 2011 at 04:48:02PM +0100, Sumit Bose wrote: > > Hi, > > > > this set of patches basically adds a samba passwd backend for IPA with > > can be build in the freeipa tree, plus the needed new objectclasses and > > attributes and enables the CLDAP service from Simo which is already > > committed. > > > > I compressed "Add-ipasam-samba-passdb-backend" to save some bandwidth. > > The backend is based on the old IPA passdb backend form the samba tree > > and various modified parts from the samba LDAP backend to make it work. > > As the result there are parts of the code which are not very pretty, > > but will work as planned. I will start refactoring the code together > > with fixing the first Coverity findings. > > > > bye, > > Sumit > > Please find attached a rebased version on top of Alexander's latest > patch. > and now rebased on top of the current master. bye, Sumit -------------- next part -------------- From 3be1e8ffbc785099d1679771a1e7291d24b4f924 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Mon, 7 Nov 2011 11:56:57 +0100 Subject: [PATCH 1/6] Move our own domain info into cn=etc https://fedorahosted.org/freeipa/ticket/2001 --- ipaserver/install/adtrustinstance.py | 26 +++++++++++++++++--------- 1 files changed, 17 insertions(+), 9 deletions(-) diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index ee50a43061e76bd9e8c6744bc66b13ce10802521..7142d79aba6412e336229e07e531654d3b3b578b 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -183,17 +183,24 @@ class ADTRUSTInstance(service.Service): except errors.NotFound: pass - try: - self.admin_conn.getEntry(self.trust_dn, ldap.SCOPE_BASE) - except errors.NotFound: - entry = ipaldap.Entry(self.trust_dn) - entry.setValues("objectclass", ["nsContainer"]) - entry.setValues("cn", "trusts") - self.admin_conn.add_s(entry) + for new_dn in (self.trust_dn, \ + "cn=ad,"+self.trust_dn, \ + "cn=ad,cn=etc,"+self.suffix): + try: + self.admin_conn.getEntry(dn, ldap.SCOPE_BASE) + except errors.NotFound: + entry = ipaldap.Entry(dn) + entry.setValues("objectclass", ["nsContainer"]) + name = dn.split('=')[1].split(',')[0] + if not name: + print "Cannot extract RDN attribute value from [%s]" % dn + return + entry.setValues("cn", name) + self.admin_conn.addEntry(entry) entry = ipaldap.Entry(self.smb_dom_dn) entry.setValues("objectclass", ["sambaDomain", "nsContainer"]) - entry.setValues("cn", "ad") + entry.setValues("cn", self.domain_name) entry.setValues("sambaDomainName", self.netbios_name) entry.setValues("sambaSID", self.__gen_sid_string()) #TODO: which MAY attributes do we want to set ? @@ -346,7 +353,8 @@ class ADTRUSTInstance(service.Service): self.smb_dn_pwd = ipautil.ipa_generate_password() self.trust_dn = "cn=trusts,%s" % self.suffix - self.smb_dom_dn = "cn=ad,%s" % self.trust_dn + self.smb_dom_dn = "cn=%s,cn=ad,cn=etc,%s" % (self.domain_name, \ + self.suffix) self.__setup_sub_dict() -- 1.7.6 -------------- next part -------------- From 4e3568c920d730cecc54630bf577eb85a5f6c076 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Wed, 9 Nov 2011 16:38:10 +0100 Subject: [PATCH 2/6] Add trust objectclass and attributes to v3 schema --- install/share/60basev3.ldif | 11 +++++++++++ 1 files changed, 11 insertions(+), 0 deletions(-) diff --git a/install/share/60basev3.ldif b/install/share/60basev3.ldif index f518541586b2df9ed08718098a7f170563aa4e1d..6db644addf298216e2b85dc68b616e8351457cf5 100644 --- a/install/share/60basev3.ldif +++ b/install/share/60basev3.ldif @@ -14,7 +14,18 @@ attributeTypes: (2.16.840.1.113730.3.8.11.7 NAME 'ipaNTProfilePath' DESC 'User P attributeTypes: (2.16.840.1.113730.3.8.11.8 NAME 'ipaNTHomeDirectory' DESC 'User Home Directory Path' EQUALITY caseIgnoreMatch OREDRING caseIgnoreOrderingMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'IPA v3' ) attributeTypes: (2.16.840.1.113730.3.8.11.9 NAME 'ipaNTHomeDirectoryDrive' DESC 'User Home Drive Letter' EQUALITY caseIgnoreMatch OREDRING caseIgnoreOrderingMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'IPA v3' ) attributeTypes: (2.16.840.1.113730.3.8.11.10 NAME 'ipaNTDomainGUID' DESC 'NT Domain GUID' EQUALITY caseIgnoreIA5Match OREDRING caseIgnoreIA5OrderingMatch SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VALUE X-ORIGIN 'IPA v3' ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.11 NAME 'ipaNTTrustType' DESC 'Type of trust' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.12 NAME 'ipaNTTrustAttributes' DESC 'Trust attributes for a trusted domain' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.13 NAME 'ipaNTTrustDirection' DESC 'Direction of a trust' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.14 NAME 'ipaNTTrustPartner' DESC 'Fully qualified name of the domain with which a trust exists' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{128} ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.15 NAME 'ipaNTTrustAuthOutgoing' DESC 'Authentication information for the outgoing portion of a trust' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.16 NAME 'ipaNTTrustAuthIncoming' DESC 'Authentication information for the incoming portion of a trust' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.17 NAME 'ipaNTTrustForestTrustInfo' DESC 'Forest trust information for a trusted domain object' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.18 NAME 'ipaNTTrustPosixOffset' DESC 'POSIX offset of a trust' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) +attributeTypes: ( 2.16.840.1.113730.3.8.11.19 NAME 'ipaNTSupportedEncryptionTypes' DESC 'Supported encryption types of a trust' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) objectClasses: (2.16.840.1.113730.3.8.12.1 NAME 'ipaExternalGroup' SUP top STRUCTURAL MUST ( cn ) MAY ( ipaExternalMember $ memberOf $ description $ owner) X-ORIGIN 'IPA v3' ) objectClasses: (2.16.840.1.113730.3.8.12.2 NAME 'ipaNTUserAttrs' SUP top AUXILIARY MUST ( ipaNTSecurityIdentifier ) MAY ( ipaNTHash $ ipaNTLogonScript $ ipaNTProfilePath $ ipaNTHomeDirectory $ ipaNTHomeDirectoryDrive ) X-ORIGIN 'IPA v3' ) objectClasses: (2.16.840.1.113730.3.8.12.3 NAME 'ipaNTGroupAttrs' SUP top AUXILIARY MUST ( ipaNTSecurityIdentifier ) X-ORIGIN 'IPA v3' ) objectClasses: (2.16.840.1.113730.3.8.12.4 NAME 'ipaNTDomainAttrs' SUP top AUXILIARY MUST ( ipaNTSecurityIdentifier $ ipaNTFlatName $ ipaNTDomainGUID ) MAY ( ipaNTFallbackPrimaryGroup ) X-ORIGIN 'IPA v3' ) +objectClasses: (2.16.840.1.113730.3.8.12.5 NAME 'ipaNTTrustedDomain' SUP top STRUCTURAL DESC 'Trusted Domain Object' MUST ( cn ) MAY ( ipaNTTrustType $ ipaNTTrustAttributes $ ipaNTTrustDirection $ ipaNTTrustPartner $ ipaNTFlatName $ ipaNTTrustAuthOutgoing $ ipaNTTrustAuthIncoming $ ipaNTSecurityIdentifier $ ipaNTTrustForestTrustInfo $ ipaNTTrustPosixOffset $ ipaNTSupportedEncryptionTypes) ) + -- 1.7.6 -------------- next part -------------- From 861ef6683df8ae7f2c8643840770aae2c26231a3 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Mon, 7 Nov 2011 12:48:10 +0100 Subject: [PATCH 3/6] Use new objectclasses and attributes for trust --- ipaserver/install/adtrustinstance.py | 46 +++++++++++++++++++++++---------- 1 files changed, 32 insertions(+), 14 deletions(-) diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 7142d79aba6412e336229e07e531654d3b3b578b..0bdedfd2b42df38ec98a3bd3532c9ba05fd3e0df 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -22,7 +22,7 @@ import errno import ldap import service import tempfile -import installutils +import uuid from ipaserver import ipaldap from ipaserver.install.dsinstance import realm_to_serverid from ipaserver.install.bindinstance import get_rr, add_rr, del_rr, \ @@ -75,6 +75,14 @@ def make_netbios_name(s): return ''.join([c for c in s.split('.')[0].upper() if c in allowed_netbios_chars])[:15] class ADTRUSTInstance(service.Service): + + ATTR_SID = "ipaNTSecurityIdentifier" + ATTR_FLAT_NAME = "ipaNTFlatName" + ATTR_GUID = "ipaNTDomainGUID" + OBJC_USER = "ipaNTUserAttrs" + OBJC_GROUP = "ipaNTGroupAttrs" + OBJC_DOMAIN = "ipaNTDomainAttrs" + def __init__(self, fstore=None, dm_password=None): service.Service.__init__(self, "smb", dm_password=dm_password) @@ -107,13 +115,22 @@ class ADTRUSTInstance(service.Service): # Also the premission to create trusted domain objects below the # domain object is granted. mod = [(ldap.MOD_ADD, 'aci', - str('(targetattr = "sambaNTPassword")' \ + str('(targetattr = "ipaNTHash")' \ '(version 3.0; acl "Samba user can read NT passwords";' \ 'allow (read) userdn="ldap:///%s";)' % self.smb_dn)), (ldap.MOD_ADD, 'aci', str('(target = "ldap:///cn=ad,cn=trusts,%s")' \ - '(targetattr = "sambaTrustType || sambaTrustAttributes || sambaTrustDirection || sambaTrustPartner || sambaFlatName || sambaTrustAuthOutgoing || sambaTrustAuthIncoming || sambaSecurityIdentifier || sambaTrustForestTrustInfo || sambaTrustPosixOffset || sambaSupportedEncryptionTypes")' \ - '(version 3.0;acl "Allow samba user to create and delete trust accounts";' \ + '(targetattr = "ipaNTTrustType || ipaNTTrustAttributes || ' \ + 'ipaNTTrustDirection || ' \ + 'ipaNTTrustPartner || ipaNTFlatName || ' \ + 'ipaNTTrustAuthOutgoing || ' \ + 'ipaNTTrustAuthIncoming || ' \ + 'ipaNTSecurityIdentifier || ' \ + 'ipaNTTrustForestTrustInfo || ' \ + 'ipaNTTrustPosixOffset || ' \ + 'ipaNTSupportedEncryptionTypes")' \ + '(version 3.0;acl "Allow samba user to create and delete ' \ + 'trust accounts";' \ 'allow (write,add,delete) userdn = "ldap:///%s";)' % \ (self.suffix, self.smb_dn)))] @@ -137,7 +154,7 @@ class ADTRUSTInstance(service.Service): print "Samba domain object not found" return - dom_sid = dom_entry.getValue("sambaSID") + dom_sid = dom_entry.getValue(self.ATTR_SID) if not dom_sid: print "Samba domain object does not have a SID" return @@ -155,22 +172,22 @@ class ADTRUSTInstance(service.Service): print "IPA admin group object not found" return - if admin_entry.getValue("sambaSID") or \ - admin_group_entry.getValue("sambaSID"): + if admin_entry.getValue(self.ATTR_SID) or \ + admin_group_entry.getValue(self.ATTR_SID): print "Admin SID already set, nothing to do" return try: self.admin_conn.modify_s(admin_dn, \ - [(ldap.MOD_ADD, "objectclass", "sambaSamAccount"), \ - (ldap.MOD_ADD, "sambaSID", dom_sid + "-500")]) + [(ldap.MOD_ADD, "objectclass", self.OBJC_USER), \ + (ldap.MOD_ADD, self.ATTR_SID, dom_sid + "-500")]) except: print "Failed to modify IPA admin object" try: self.admin_conn.modify_s(admin_group_dn, \ - [(ldap.MOD_ADD, "objectclass", "sambaSidEntry"), \ - (ldap.MOD_ADD, "sambaSID", dom_sid + "-512")]) + [(ldap.MOD_ADD, "objectclass", self.OBJC_GROUP), \ + (ldap.MOD_ADD, self.ATTR_SID, dom_sid + "-512")]) except: print "Failed to modify IPA admin group object" @@ -199,10 +216,11 @@ class ADTRUSTInstance(service.Service): self.admin_conn.addEntry(entry) entry = ipaldap.Entry(self.smb_dom_dn) - entry.setValues("objectclass", ["sambaDomain", "nsContainer"]) + entry.setValues("objectclass", [self.OBJC_DOMAIN, "nsContainer"]) entry.setValues("cn", self.domain_name) - entry.setValues("sambaDomainName", self.netbios_name) - entry.setValues("sambaSID", self.__gen_sid_string()) + entry.setValues(self.ATTR_FLAT_NAME, self.netbios_name) + entry.setValues(self.ATTR_SID, self.__gen_sid_string()) + entry.setValues(self.ATTR_GUID, str(uuid.uuid4())) #TODO: which MAY attributes do we want to set ? self.admin_conn.add_s(entry) -- 1.7.6 -------------- next part -------------- From 4632c3ea945af3fab47ed67741a373ec6ec04ade Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Mon, 7 Nov 2011 12:59:20 +0100 Subject: [PATCH 4/6] Fix some pylint warnings --- install/tools/ipa-adtrust-install | 2 +- ipaserver/install/adtrustinstance.py | 107 +++++++++++++++++++++------------- 2 files changed, 68 insertions(+), 41 deletions(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index 87fecbfb4834d65fdccc3f8536a5665ba75e48a5..c6fd3478a28697301cac317dff1bbf25c6d865ce 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -111,7 +111,7 @@ def main(): print "" # Check if samba packages are installed - if not adtrustinstance.check_inst(options.unattended): + if not adtrustinstance.check_inst(): sys.exit("Aborting installation.") # Initialize the ipalib api diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 0bdedfd2b42df38ec98a3bd3532c9ba05fd3e0df..7808b3dea809d237299527ed107bc161b565c834 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -20,10 +20,11 @@ import os import errno import ldap -import service import tempfile import uuid from ipaserver import ipaldap +from ipaserver.install import installutils +from ipaserver.install import service from ipaserver.install.dsinstance import realm_to_serverid from ipaserver.install.bindinstance import get_rr, add_rr, del_rr, \ dns_zone_exists @@ -32,17 +33,17 @@ from ipapython import sysrestore from ipapython import ipautil from ipapython.ipa_log_manager import * -import random import string import struct -allowed_netbios_chars = string.ascii_uppercase + string.digits +ALLOWED_NETBIOS_CHARS = string.ascii_uppercase + string.digits -def check_inst(unattended): - for f in ['/usr/sbin/smbd', '/usr/bin/net', '/usr/bin/smbpasswd']: - if not os.path.exists(f): - print "%s was not found on this system" % f - print "Please install the 'samba' packages and start the installation again" +def check_inst(): + for smbfile in ['/usr/sbin/smbd', '/usr/bin/net', '/usr/bin/smbpasswd']: + if not os.path.exists(smbfile): + print "%s was not found on this system" % file + print "Please install the 'samba' packages and " \ + "start the installation again" return False #TODO: Add check for needed samba4 libraries @@ -51,13 +52,13 @@ def check_inst(unattended): def ipa_smb_conf_exists(): try: - fd = open('/etc/samba/smb.conf', 'r') - except IOError, e: - if e.errno == errno.ENOENT: + conf_fd = open('/etc/samba/smb.conf', 'r') + except IOError, err: + if err.errno == errno.ENOENT: return False - lines = fd.readlines() - fd.close() + lines = conf_fd.readlines() + conf_fd.close() for line in lines: if line.startswith('### Added by IPA Installer ###'): return True @@ -66,13 +67,15 @@ def ipa_smb_conf_exists(): def check_netbios_name(s): # NetBIOS names may not be longer than 15 allowed characters - if not s or len(s) > 15 or ''.join([c for c in s if c not in allowed_netbios_chars]): + if not s or len(s) > 15 or \ + ''.join([c for c in s if c not in ALLOWED_NETBIOS_CHARS]): return False return True def make_netbios_name(s): - return ''.join([c for c in s.split('.')[0].upper() if c in allowed_netbios_chars])[:15] + return ''.join([c for c in s.split('.')[0].upper() \ + if c in ALLOWED_NETBIOS_CHARS])[:15] class ADTRUSTInstance(service.Service): @@ -84,6 +87,22 @@ class ADTRUSTInstance(service.Service): OBJC_DOMAIN = "ipaNTDomainAttrs" def __init__(self, fstore=None, dm_password=None): + self.fqdn = None + self.ip_address = None + self.realm_name = None + self.domain_name = None + self.netbios_name = None + self.no_msdcs = None + self.smbd_user = None + self.suffix = None + self.ldapi_socket = None + self.smb_conf = None + self.smb_dn = None + self.smb_dn_pwd = None + self.trust_dn = None + self.smb_dom_dn = None + self.sub_dict = None + service.Service.__init__(self, "smb", dm_password=dm_password) if fstore: @@ -97,7 +116,8 @@ class ADTRUSTInstance(service.Service): self.admin_conn.getEntry(self.smb_dn, ldap.SCOPE_BASE) print "Samba user entry exists, resetting password" - self.admin_conn.modify_s(self.smb_dn, [(ldap.MOD_REPLACE, "userPassword", self.smb_dn_pwd)]) + self.admin_conn.modify_s(self.smb_dn, \ + [(ldap.MOD_REPLACE, "userPassword", self.smb_dn_pwd)]) return except errors.NotFound: @@ -108,7 +128,7 @@ class ADTRUSTInstance(service.Service): entry.setValues("objectclass", ["account", "simplesecurityobject"]) entry.setValues("uid", "samba") entry.setValues("userPassword", self.smb_dn_pwd) - self.admin_conn.add_s(entry) + self.admin_conn.addEntry(entry) # And finally grant it permission to read NT passwords, we do not want # to support LM passwords so there is no need to allow access to them. @@ -204,13 +224,14 @@ class ADTRUSTInstance(service.Service): "cn=ad,"+self.trust_dn, \ "cn=ad,cn=etc,"+self.suffix): try: - self.admin_conn.getEntry(dn, ldap.SCOPE_BASE) + self.admin_conn.getEntry(new_dn, ldap.SCOPE_BASE) except errors.NotFound: - entry = ipaldap.Entry(dn) + entry = ipaldap.Entry(new_dn) entry.setValues("objectclass", ["nsContainer"]) - name = dn.split('=')[1].split(',')[0] + name = new_dn.split('=')[1].split(',')[0] if not name: - print "Cannot extract RDN attribute value from [%s]" % dn + print "Cannot extract RDN attribute value from [%s]" % \ + new_dn return entry.setValues("cn", name) self.admin_conn.addEntry(entry) @@ -222,23 +243,23 @@ class ADTRUSTInstance(service.Service): entry.setValues(self.ATTR_SID, self.__gen_sid_string()) entry.setValues(self.ATTR_GUID, str(uuid.uuid4())) #TODO: which MAY attributes do we want to set ? - self.admin_conn.add_s(entry) + self.admin_conn.addEntry(entry) def __write_smb_conf(self): self.fstore.backup_file(self.smb_conf) - fd = open(self.smb_conf, "w") - fd.write('### Added by IPA Installer ###\n') - fd.write('[global]\n') - fd.write('config backend = registry\n') - fd.close() + conf_fd = open(self.smb_conf, "w") + conf_fd.write('### Added by IPA Installer ###\n') + conf_fd.write('[global]\n') + conf_fd.write('config backend = registry\n') + conf_fd.close() def __write_smb_registry(self): template = os.path.join(ipautil.SHARE_DIR, "smb.conf.template") conf = ipautil.template_file(template, self.sub_dict) - [fd, tmp_name] = tempfile.mkstemp() - os.write(fd, conf) - os.close(fd) + [tmp_fd, tmp_name] = tempfile.mkstemp() + os.write(tmp_fd, conf) + os.close(tmp_fd) args = ["/usr/bin/net", "conf", "import", tmp_name] @@ -250,7 +271,8 @@ class ADTRUSTInstance(service.Service): def __set_smb_ldap_password(self): args = ["/usr/bin/smbpasswd", "-c", self.smb_conf, "-s", "-W" ] - ipautil.run(args, stdin = self.smb_dn_pwd + "\n" + self.smb_dn_pwd + "\n" ) + ipautil.run(args, stdin = self.smb_dn_pwd + "\n" + \ + self.smb_dn_pwd + "\n" ) def __setup_principal(self): cifs_principal = "cifs/" + self.fqdn + "@" + self.realm_name @@ -291,7 +313,7 @@ class ADTRUSTInstance(service.Service): ".dc._msdcs") err_msg = None - ret = api.Command.dns_is_enabled() + ret = api.Command['dns_is_enabled']() if not ret['result']: err_msg = "DNS management was not enabled at install time." else: @@ -341,7 +363,8 @@ class ADTRUSTInstance(service.Service): # Instead we reply on the IPA init script to start only enabled # components as found in our LDAP configuration tree try: - self.ldap_enable('ADTRUST', self.fqdn, self.dm_password, self.suffix) + self.ldap_enable('ADTRUST', self.fqdn, self.dm_password, \ + self.suffix) except ldap.ALREADY_EXISTS: root_logger.critical("ADTRUST Service startup entry already exists.") pass @@ -355,7 +378,7 @@ class ADTRUSTInstance(service.Service): def setup(self, fqdn, ip_address, realm_name, domain_name, netbios_name, no_msdcs=False, smbd_user="samba"): - self.fqdn =fqdn + self.fqdn = fqdn self.ip_address = ip_address self.realm_name = realm_name self.domain_name = domain_name @@ -363,7 +386,8 @@ class ADTRUSTInstance(service.Service): self.no_msdcs = no_msdcs self.smbd_user = smbd_user self.suffix = ipautil.realm_to_suffix(self.realm_name) - self.ldapi_socket = "%%2fvar%%2frun%%2fslapd-%s.socket" % realm_to_serverid(self.realm_name) + self.ldapi_socket = "%%2fvar%%2frun%%2fslapd-%s.socket" % \ + realm_to_serverid(self.realm_name) self.smb_conf = "/etc/samba/smb.conf" @@ -383,15 +407,18 @@ class ADTRUSTInstance(service.Service): self.step("stopping smbd", self.__stop) self.step("create samba user", self.__create_samba_user) - self.step("create samba domain object", self.__create_samba_domain_object) + self.step("create samba domain object", \ + self.__create_samba_domain_object) self.step("create samba config registry", self.__write_smb_registry) self.step("writing samba config file", self.__write_smb_conf) - self.step("setting password for the samba user", self.__set_smb_ldap_password) + self.step("setting password for the samba user", \ + self.__set_smb_ldap_password) self.step("Adding cifs Kerberos principal", self.__setup_principal) self.step("Adding admin(group) SIDs", self.__add_admin_sids) self.step("configuring smbd to start on boot", self.__enable) if not self.no_msdcs: - self.step("adding special DNS service records", self.__add_dns_service_records) + self.step("adding special DNS service records", \ + self.__add_dns_service_records) self.step("starting smbd", self.__start) self.start_creation("Configuring smbd:") @@ -408,9 +435,9 @@ class ADTRUSTInstance(service.Service): except: pass - for f in [self.smb_conf]: + for r_file in [self.smb_conf]: try: - self.fstore.restore_file(f) + self.fstore.restore_file(r_file) except ValueError, error: root_logger.debug(error) pass -- 1.7.6 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-sbose-0013-3-Add-ipasam-samba-passdb-backend.patch.bz2 Type: application/x-bzip2 Size: 17026 bytes Desc: not available URL: -------------- next part -------------- From 7c5ef0443c0ded32a467d78ca950c4667d26163e Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Fri, 18 Nov 2011 14:04:09 +0100 Subject: [PATCH 6/6] activate CLDAP --- install/tools/ipa-adtrust-install | 3 +-- ipaserver/install/adtrustinstance.py | 4 ++++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index c6fd3478a28697301cac317dff1bbf25c6d865ce..248ea35eaa86dd59ebbc871b86df780cfd71ccf6 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -214,6 +214,7 @@ def main(): print "\t\tUDP Ports:" print "\t\t * 138: netbios-dgm" print "\t\t * 139: netbios-ssn" + print "\t\t * 389: (C)LDAP" print "\t\t * 445: microsoft-ds" print "" print "\tAdditionally you have to make sure the FreeIPA LDAP server cannot reached" @@ -221,8 +222,6 @@ def main(): print "\tfollowing ports for these servers:" print "\t\tTCP Ports:" print "\t\t * 389, 636: LDAP/LDAPS" - print "\t\tUDP Ports:" - print "\t\t * 389: (C)LDAP" print "\tYou may want to choose to REJECT the network packets instead of DROPing them" print "\tto avoid timeouts on the AD domain controllers." diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 7808b3dea809d237299527ed107bc161b565c834..f4379019dfe2c61c9b3a4b13d71cf148112a4d85 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -254,6 +254,9 @@ class ADTRUSTInstance(service.Service): conf_fd.write('config backend = registry\n') conf_fd.close() + def __add_cldap_module(self): + self._ldap_mod("ipa-cldap-conf.ldif", self.sub_dict) + def __write_smb_registry(self): template = os.path.join(ipautil.SHARE_DIR, "smb.conf.template") conf = ipautil.template_file(template, self.sub_dict) @@ -415,6 +418,7 @@ class ADTRUSTInstance(service.Service): self.__set_smb_ldap_password) self.step("Adding cifs Kerberos principal", self.__setup_principal) self.step("Adding admin(group) SIDs", self.__add_admin_sids) + self.step("Activation CLDAP plugin", self.__add_cldap_module) self.step("configuring smbd to start on boot", self.__enable) if not self.no_msdcs: self.step("adding special DNS service records", \ -- 1.7.6 From simo at redhat.com Tue Nov 29 21:13:31 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 Nov 2011 16:13:31 -0500 Subject: [Freeipa-devel] [PATCH] fix uninitialized free() [Fwd: daemons/ipa-kdb] Message-ID: <1322601211.2613.77.camel@willson.li.ssimo.org> I pushed the attached patch under the oneliner rule -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- An embedded message was scrubbed... From: Simo Sorce Subject: daemons/ipa-kdb Date: Tue, 29 Nov 2011 21:09:06 +0000 (UTC) Size: 3228 URL: From simo at redhat.com Tue Nov 29 21:14:08 2011 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 Nov 2011 16:14:08 -0500 Subject: [Freeipa-devel] [PATCH] fix memleaks Message-ID: <1322601248.2613.78.camel@willson.li.ssimo.org> Found a couple of memleaks while reviewing code. Attached. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-ipa-kdb-fix-memleaks-in-ipa_kdb_mspac.c.patch Type: text/x-patch Size: 1789 bytes Desc: not available URL: From abokovoy at redhat.com Tue Nov 29 21:25:41 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 29 Nov 2011 23:25:41 +0200 Subject: [Freeipa-devel] [PATCH] Add ipasam samba passdb backend In-Reply-To: <20111129210926.GK2246@localhost.localdomain> References: <20111123154802.GI2205@localhost.localdomain> <20111128172227.GE2246@localhost.localdomain> <20111129210926.GK2246@localhost.localdomain> Message-ID: <20111129212541.GA24514@redhat.com> On Tue, 29 Nov 2011, Sumit Bose wrote: > @@ -199,10 +216,11 @@ class ADTRUSTInstance(service.Service): > self.admin_conn.addEntry(entry) > > entry = ipaldap.Entry(self.smb_dom_dn) > - entry.setValues("objectclass", ["sambaDomain", "nsContainer"]) > + entry.setValues("objectclass", [self.OBJC_DOMAIN, "nsContainer"]) > entry.setValues("cn", self.domain_name) > - entry.setValues("sambaDomainName", self.netbios_name) > - entry.setValues("sambaSID", self.__gen_sid_string()) > + entry.setValues(self.ATTR_FLAT_NAME, self.netbios_name) > + entry.setValues(self.ATTR_SID, self.__gen_sid_string()) > + entry.setValues(self.ATTR_GUID, str(uuid.uuid4())) > #TODO: which MAY attributes do we want to set ? > self.admin_conn.add_s(entry) Could you please also convert this one to .addEntry(entry)? I think it is the last one left... -- / Alexander Bokovoy From rcritten at redhat.com Tue Nov 29 21:33:12 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 29 Nov 2011 16:33:12 -0500 Subject: [Freeipa-devel] [PATCH] ipa-client-install with --no-sssd option should check for nss_ldap In-Reply-To: <4EC13FD2.9000203@redhat.com> References: <4EBD294A.20200@redhat.com> <4EC13FD2.9000203@redhat.com> Message-ID: <4ED54F98.4070700@redhat.com> Ondrej Hamada wrote: > On 11/11/2011 02:55 PM, Ondrej Hamada wrote: >> https://fedorahosted.org/freeipa/ticket/2063 >> >> In order to check presence of nss_ldap when installing client with >> '--no-sssd' option there was added code into ipa-client-install. Check >> is base on existence of nss_ldap configuration files. This >> configuration could be in 'etc/ldap.conf', '/etc/nss_ldap.conf' or >> '/etc/libnss_ldap.conf'. Presence of any of these files is considered >> as success otherwise failure. >> >> >> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > I've rewritten it. Additionally it checks for existence of nss-pam-ldapd > and makes the results reusable by configure_{ldap|nslcd}_conf() functions. > > https://fedorahosted.org/freeipa/ticket/2063 > > In order to check presence of nss_ldap or nss-pam-ldapd when installing > client > with '--no-sssd' option there was added code into ipa-client-install. > Checking is based on existence of nss_ldap configuration files. This > configuration could be in 'etc/ldap.conf', '/etc/nss_ldap.conf' or > '/etc/libnss_ldap.conf'. Optionaly the nss_ldap could cooperate with > pam_ldap > module and hence the presence of it is checked by looking for > 'pam_ldap.conf' file. > Existence of nss-pam-ldapd is checked against existence of 'nslcd.conf' > file. > All this checking is done by function nssldap_exists(). > Because both main modules are maintained by two different functions, the > function > returns tuple containing return code and dictionary structure - its key > is name > of target function and value is list of existing configuration files. > Files to check are specified inside the nssldap_exists() function. > > In order to fit the returned values, the functions > configure_{ldap|nslcd}_conf() > were slightly modified. They accept one more parameter which is list of > existing files. > They are not checking existence of above mentioned files anymore. The patch looks good, just a couple of issues. 1. In the nslcd configurator you add ''.join(files). Did you mean ','.join(files)? 2. The commit message lines wrap making it difficult to read. Can you limit the lines to ~70 chars per line? 3. I think the message printed when neither package is available can be simplified to: One of these packages must be installed: nss_ldap or nss-pam-ldapd It needs a rebase too. rob From rcritten at redhat.com Tue Nov 29 21:47:42 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 29 Nov 2011 16:47:42 -0500 Subject: [Freeipa-devel] make-lint failures Message-ID: <4ED552FE.8020000@redhat.com> Seeing some new make-lint failures after the reworking of ipaldap function signatures; $ ./make-lint ipaserver/install/adtrustinstance.py:101: [E1120, ADTRUSTInstance.__create_samba_user] No value passed for parameter 'modlist' in function call ipaserver/install/adtrustinstance.py:190: [E1120, ADTRUSTInstance.__create_samba_domain_object] No value passed for parameter 'modlist' in function call ipaserver/install/adtrustinstance.py:198: [E1120, ADTRUSTInstance.__create_samba_domain_object] No value passed for parameter 'modlist' in function call I wonder if the signature needs to be: def add_s(self, dn, modlist=None): For the case were dn isn't an Entry we probably need to raise an exception if modlist is None (or test to see what python-ldap add_s does). rob From abokovoy at redhat.com Tue Nov 29 22:12:24 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 30 Nov 2011 00:12:24 +0200 Subject: [Freeipa-devel] make-lint failures In-Reply-To: <4ED552FE.8020000@redhat.com> References: <4ED552FE.8020000@redhat.com> Message-ID: <20111129221222.GB24514@redhat.com> On Tue, 29 Nov 2011, Rob Crittenden wrote: > Seeing some new make-lint failures after the reworking of ipaldap > function signatures; > > $ ./make-lint > ipaserver/install/adtrustinstance.py:101: [E1120, > ADTRUSTInstance.__create_samba_user] No value passed for parameter > 'modlist' in function call > ipaserver/install/adtrustinstance.py:190: [E1120, > ADTRUSTInstance.__create_samba_domain_object] No value passed for > parameter 'modlist' in function call > ipaserver/install/adtrustinstance.py:198: [E1120, > ADTRUSTInstance.__create_samba_domain_object] No value passed for > parameter 'modlist' in function call > > I wonder if the signature needs to be: > > def add_s(self, dn, modlist=None): > > For the case were dn isn't an Entry we probably need to raise an > exception if modlist is None (or test to see what python-ldap add_s > does). The original LDAPObject.add_s() has modlist as non-optional argument: def add_s(self,dn,modlist): I don't think it is wise to break that API assumption. In all the cases above it should get .add_s(entry) replaced by .addEntry(entry). The reason these failures are shown is because Martin reverted the earlier version of Sumit's patch that you mistakenly committed. Sumit has produced new patch already but there is one minor issue in it (another .add_s() -> .addEntry() replacement needs to be done). -- / Alexander Bokovoy From edewata at redhat.com Tue Nov 29 22:35:12 2011 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 29 Nov 2011 16:35:12 -0600 Subject: [Freeipa-devel] Activation and password reset webapp UI In-Reply-To: <4ED3F8F5.8070402@pet.ubc.ca> References: <4ED3F8F5.8070402@pet.ubc.ca> Message-ID: <4ED55E20.8010202@redhat.com> On 11/28/2011 3:11 PM, Ryan Thomson wrote: > Hello, > > Attached is my amateur attempt at contributing to FreeIPA. > > The patch implements an account "activation" and password reset webapp > UI to cover use cases where FreeIPA may be acting only as a backend to > services such as Samba or other web application that do no expose a > method for changing expired kerberos credentials. > > This code is based on the "migration" webapp UI. > > See ticket 1907. > > https://fedorahosted.org/freeipa/ticket/1907 Hi, thanks for the patch! I found some issues: 1. Build failed due to missing variables: install/activation/activation.py:94: [E0602, activate] Undefined variable 'e' install/activation/activation.py:97: [E0602, activate] Undefined variable 'e' 2. After fixing #1, I found that the activation folder is not installed into /usr/share/ipa. I think this needs to be fixed in install/Makefile.am and freeipa.spec.in. 3. After copying the folder manually I can access the activation page, but when test it I get this error in /var/log/httpd/errors_log: [Tue Nov 29 16:13:30 2011] [error] ERROR:root:migration context search failed: { 'info': 'TLS error -8172:Unknown code ___f 20', 'desc': "Can't contact LDAP serv er"} It probably should use ldapi instead of ldaps. See migration.py. 4. In index.html:33 I think the "Password" label should say "Current Password" for better clarity. Also in line 41 the "Confirm" label should say "Verify Password" for consistency with the rest of the UI. 5. In index.html:44 it should be instead of
    . This is an existing problem in migration/index.html. 6. There are broken image links in ipa_activation.css. The images have been slightly renamed and moved into install/ui/images recently. 7. There's a broken link to the CSS file in invalid.html:7. 8. The intall/activation/jquery-ui.css doesn't seem to be used anywhere. This is also an existing problem in migration. 9. In activation.py:72 the error message still says 'migration'. 10. In activation.py:121 the return statement in the 'finally' clause is redundant if there's an exception because it will return another page. This return statement should be moved inside the 'try' clause. Also we might need another return statement in the 'except' clause to handle the case where the errno doesn't match EPERM or EIO. 11. The file actually contains 3 patches. It's better to put them in separate files because in case one of them needs revision you won't have to resend everything. Another option is to squash them into a single patch. 12. The second patch contains the first patch file. 13. There are some warnings about extra whitespaces when applying the patch. Try applying the patch with --whitespace=fix then reexporting it again. -- Endi S. Dewata From rcritten at redhat.com Tue Nov 29 22:45:57 2011 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 29 Nov 2011 17:45:57 -0500 Subject: [Freeipa-devel] [PATCH] 906 Add SELinux user mapping framework. In-Reply-To: <20111128171034.GA26611@redhat.com> References: <4ECD6D4F.600@redhat.com> <20111123230047.GB14772@redhat.com> <20111128171034.GA26611@redhat.com> Message-ID: <4ED560A5.7000804@redhat.com> Alexander Bokovoy wrote: > On Thu, 24 Nov 2011, Alexander Bokovoy wrote: >> On Wed, 23 Nov 2011, Rob Crittenden wrote: >>> This will allow one to define what SELinux context a given user gets >>> on a given machine. A rule can contain a set of users and hosts or it >>> can point to an existing HBAC rule that defines them. >>> >>> https://fedorahosted.org/freeipa/ticket/755 >> I read through the patch, will need to test it later this week. I >> basically have two minor points: >> >> 1. Split charachter in the SE Linux user map order. >>> + >>> + Define SELinux user map order: >>> + ipa config-mod --ipaselinuxusermaporder='guest_u:s0$xguest_u:s0$user_u:s0-s0:c0.c1023$staff_u:s0-s0:c0.c1023$unconfined_u:s0-s0:c0.c1023' >>> """) >> $ can be considered 'active' character in all shells in a sense it >> changes treatment of following characters from the shell perspective >> and therefore will always require shielding from the shell's >> influence. This increases likelyhood of error from a user side. >> >> Maybe / would be more neutral character? >> >> As you said on IRC, people might have religious feeling about >> separators but tricking users into always thinking about >> escaping/single quoting is equally bad. >> >> 2. We have two possible ways to address named properties in MagicDict >> and NameSpace objects -- through explicit attribute use and through >> the dictionary key. I guess for the cases when we know the attribute >> name in advance, it would perhaps be preferrable to use the former >> style: >> >>> + def pre_callback(self, ldap, dn, *keys, **options): >>> + kw = dict(seealso=dn) >>> + _entries = api.Command['selinuxusermap_find'](None, **kw) >> this would be >> _entries = api.Command.selinuxusermap_find(None, **kw) >> >> Other than those two minor points, the patch looks very good. I'm >> going to give it a run on Friday. > I tested the patch and it works for me on a new install. On upgrade of > existing installation I've got few errors during run of > ipa-ldap-updater for SELinux schema changes. Unfortunately, didn't > save the log as it was 2.1 -> 2.99 upgrade as well. > It turns out that other characters are just as troublesome and require escaping (space and \). I"m going to leave it as $ unless someone comes up with something better that the shell isn't going to whine about. I fixed some other minor issues and rebased. Upgrading isn't really testable at this point yet, other things in 3.0 need to be addressed as well. We have a separate ticket to look into the schema updates so I've removed the update file for now. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-906-2-selinux.patch Type: text/x-patch Size: 65407 bytes Desc: not available URL: From jdennis at redhat.com Tue Nov 29 23:35:05 2011 From: jdennis at redhat.com (John Dennis) Date: Tue, 29 Nov 2011 18:35:05 -0500 Subject: [Freeipa-devel] make-lint failures In-Reply-To: <20111129221222.GB24514@redhat.com> References: <4ED552FE.8020000@redhat.com> <20111129221222.GB24514@redhat.com> Message-ID: <4ED56C29.90002@redhat.com> On 11/29/2011 05:12 PM, Alexander Bokovoy wrote: > On Tue, 29 Nov 2011, Rob Crittenden wrote: >> Seeing some new make-lint failures after the reworking of ipaldap >> function signatures; >> >> $ ./make-lint >> ipaserver/install/adtrustinstance.py:101: [E1120, >> ADTRUSTInstance.__create_samba_user] No value passed for parameter >> 'modlist' in function call >> ipaserver/install/adtrustinstance.py:190: [E1120, >> ADTRUSTInstance.__create_samba_domain_object] No value passed for >> parameter 'modlist' in function call >> ipaserver/install/adtrustinstance.py:198: [E1120, >> ADTRUSTInstance.__create_samba_domain_object] No value passed for >> parameter 'modlist' in function call >> >> I wonder if the signature needs to be: >> >> def add_s(self, dn, modlist=None): >> >> For the case were dn isn't an Entry we probably need to raise an >> exception if modlist is None (or test to see what python-ldap add_s >> does). > The original LDAPObject.add_s() has modlist as non-optional argument: > def add_s(self,dn,modlist): > > I don't think it is wise to break that API assumption. > > In all the cases above it should get .add_s(entry) replaced by > .addEntry(entry). > > The reason these failures are shown is because Martin reverted the > earlier version of Sumit's patch that you mistakenly committed. Sumit > has produced new patch already but there is one minor issue in it > (another .add_s() -> .addEntry() replacement needs to be done). > FWIW this is a perfect example of why not using ldap2 in the installer code is problematic. When the installer code is directly invoking a native ldap method though inheritance it becomes difficult to pass different object types to the methods. In IPA there are many places where we would benefit from using IPA custom classes (e.g. Entry). For the DN work I would also like to pass DN objects. I had to add an entire class subclassed from LDAPObject for no purpose other than to support object conversion on native LDAP methods and then instantiate that class in the installer conn objects. It would be so much nicer if we had an abstraction layer that knew about our preferred internal objects exposing an LDAP interface suited to our needs. We have such an abstraction, it's called ldap2, but alas the installer code doesn't use it :-( -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From ryan at pet.ubc.ca Wed Nov 30 01:57:21 2011 From: ryan at pet.ubc.ca (Ryan Thomson) Date: Tue, 29 Nov 2011 17:57:21 -0800 Subject: [Freeipa-devel] Activation and password reset webapp UI In-Reply-To: <4ED55E20.8010202@redhat.com> References: <4ED3F8F5.8070402@pet.ubc.ca> <4ED55E20.8010202@redhat.com> Message-ID: <4ED58D81.2040203@pet.ubc.ca> Hi Endi, Thanks for reviewing the patch. Looks like I have some work to do. 1-2) I have to admit I didn't even try building with these patches. I was pretty sure install/Makefile.am would need modification to install it but I didn't know if submitting patches to install/Makefile.am and the spec file was frowned upon or not and I assumed wrong. I'll test by building from git next time and making sure it's ready to go from there on. 3) Strange. I suppose originally coding against IPA as packaged by RHEL then "porting" to and testing against FreeIPA as packaged in F16 could why I didn't catch that. Gotta do everything against the latest in git. Got it. 4) Box width will have to be increased in CSS to compensate unless we want "Current Password" and "Verify Password" to get wrapped but no big deal. 5-9) Noted. 10) Besides "finally" being redundant, error handling isn't great overall, let me see if I can improve that. 11-13) Git newbie mistakes. I really don't know what I'm doing with regards to git. I assumed sending multiple patches where it's just me fixing earlier mistakes would be worse than jamming all the commits into a single file. I kind of just wanted to send one patch to implement everything but I have no idea how to do that besides get everything "right" in a single commit. Is that how I'm supposed to do it? Might be a few weeks before I get another patch submitted to fix all these amateur mistakes. Thanks, --Ryan On 11/29/2011 02:35 PM, Endi Sukma Dewata wrote: > On 11/28/2011 3:11 PM, Ryan Thomson wrote: >> Hello, >> >> Attached is my amateur attempt at contributing to FreeIPA. >> >> The patch implements an account "activation" and password reset webapp >> UI to cover use cases where FreeIPA may be acting only as a backend to >> services such as Samba or other web application that do no expose a >> method for changing expired kerberos credentials. >> >> This code is based on the "migration" webapp UI. >> >> See ticket 1907. >> >> https://fedorahosted.org/freeipa/ticket/1907 > > Hi, thanks for the patch! I found some issues: > > 1. Build failed due to missing variables: > > install/activation/activation.py:94: [E0602, activate] Undefined > variable 'e' > install/activation/activation.py:97: [E0602, activate] Undefined > variable 'e' > > 2. After fixing #1, I found that the activation folder is not installed > into /usr/share/ipa. I think this needs to be fixed in > install/Makefile.am and freeipa.spec.in. > > 3. After copying the folder manually I can access the activation page, > but when test it I get this error in /var/log/httpd/errors_log: > > [Tue Nov 29 16:13:30 2011] [error] ERROR:root:migration context search > failed: { > 'info': 'TLS error -8172:Unknown code ___f 20', 'desc': "Can't contact > LDAP serv > er"} > > It probably should use ldapi instead of ldaps. See migration.py. > > 4. In index.html:33 I think the "Password" label should say "Current > Password" for better clarity. Also in line 41 the "Confirm" label should > say "Verify Password" for consistency with the rest of the UI. > > 5. In index.html:44 it should be
instead of
    . This is an > existing problem in migration/index.html. > > 6. There are broken image links in ipa_activation.css. The images have > been slightly renamed and moved into install/ui/images recently. > > 7. There's a broken link to the CSS file in invalid.html:7. > > 8. The intall/activation/jquery-ui.css doesn't seem to be used anywhere. > This is also an existing problem in migration. > > 9. In activation.py:72 the error message still says 'migration'. > > 10. In activation.py:121 the return statement in the 'finally' clause is > redundant if there's an exception because it will return another page. > This return statement should be moved inside the 'try' clause. Also we > might need another return statement in the 'except' clause to handle the > case where the errno doesn't match EPERM or EIO. > > 11. The file actually contains 3 patches. It's better to put them in > separate files because in case one of them needs revision you won't have > to resend everything. Another option is to squash them into a single patch. > > 12. The second patch contains the first patch file. > > 13. There are some warnings about extra whitespaces when applying the > patch. Try applying the patch with --whitespace=fix then reexporting it > again. > From ayoung at redhat.com Wed Nov 30 02:44:46 2011 From: ayoung at redhat.com (Adam Young) Date: Tue, 29 Nov 2011 21:44:46 -0500 Subject: [Freeipa-devel] Activation and password reset webapp UI In-Reply-To: <4ED58D81.2040203@pet.ubc.ca> References: <4ED3F8F5.8070402@pet.ubc.ca> <4ED55E20.8010202@redhat.com> <4ED58D81.2040203@pet.ubc.ca> Message-ID: <4ED5989E.9080306@redhat.com> On 11/29/2011 08:57 PM, Ryan Thomson wrote: > Hi Endi, > > Thanks for reviewing the patch. Looks like I have some work to do. > > 1-2) I have to admit I didn't even try building with these patches. I > was pretty sure install/Makefile.am would need modification to install > it but I didn't know if submitting patches to install/Makefile.am and > the spec file was frowned upon or not and I assumed wrong. I'll test > by building from git next time and making sure it's ready to go from > there on. > > 3) Strange. I suppose originally coding against IPA as packaged by > RHEL then "porting" to and testing against FreeIPA as packaged in F16 > could why I didn't catch that. Gotta do everything against the latest > in git. Got it. > > 4) Box width will have to be increased in CSS to compensate unless we > want "Current Password" and "Verify Password" to get wrapped but no > big deal. > > 5-9) Noted. > > 10) Besides "finally" being redundant, error handling isn't great > overall, let me see if I can improve that. > > 11-13) Git newbie mistakes. I really don't know what I'm doing with > regards to git. I assumed sending multiple patches where it's just me > fixing earlier mistakes would be worse than jamming all the commits > into a single file. I kind of just wanted to send one patch to > implement everything but I have no idea how to do that besides get > everything "right" in a single commit. Is that how I'm supposed to do it? I'd submit it all as one patch, as it isn't that big, and the work is fairly atomic. Once you have it working as you want, squash it. We have a convention where we number the patches using 4 digits. So yours would be 0001. And updated verson would be 0001-1 and the next fixed version would be 0001-2. Git is very forgiving. Here's what I'd do. Once you get it working right make two branches. One will continue to have all of your changes in it in separate patches, and you'll use the other one to squash them. I'm going to assume that you did all of this in you home tree in your master branch. So git checkout -b activation_password will create a branch with that name from the current point. Then, git checkout -b activation_password_squashed will do the same. You can see your branches with git branch. You can see which commit they refere to with git log . Then run git rebase -i HEAD~4 (assuming you have three commits. Always one more than the number of commits) and then use the vi commands to edit the commits into a single commit. Once you have it all in a single commit, run the git patchformat command and submit that. https://fedorahosted.org/freeipa/wiki/PatchFormat > > Might be a few weeks before I get another patch submitted to fix all > these amateur mistakes. No problem. Just make sure that your run git fetch ; git rebase origin/master to bring your code in line with any changes made between now and then. > > Thanks, > > --Ryan > > On 11/29/2011 02:35 PM, Endi Sukma Dewata wrote: >> On 11/28/2011 3:11 PM, Ryan Thomson wrote: >>> Hello, >>> >>> Attached is my amateur attempt at contributing to FreeIPA. >>> >>> The patch implements an account "activation" and password reset webapp >>> UI to cover use cases where FreeIPA may be acting only as a backend to >>> services such as Samba or other web application that do no expose a >>> method for changing expired kerberos credentials. >>> >>> This code is based on the "migration" webapp UI. >>> >>> See ticket 1907. >>> >>> https://fedorahosted.org/freeipa/ticket/1907 >> >> Hi, thanks for the patch! I found some issues: >> >> 1. Build failed due to missing variables: >> >> install/activation/activation.py:94: [E0602, activate] Undefined >> variable 'e' >> install/activation/activation.py:97: [E0602, activate] Undefined >> variable 'e' >> >> 2. After fixing #1, I found that the activation folder is not installed >> into /usr/share/ipa. I think this needs to be fixed in >> install/Makefile.am and freeipa.spec.in. >> >> 3. After copying the folder manually I can access the activation page, >> but when test it I get this error in /var/log/httpd/errors_log: >> >> [Tue Nov 29 16:13:30 2011] [error] ERROR:root:migration context search >> failed: { >> 'info': 'TLS error -8172:Unknown code ___f 20', 'desc': "Can't contact >> LDAP serv >> er"} >> >> It probably should use ldapi instead of ldaps. See migration.py. >> >> 4. In index.html:33 I think the "Password" label should say "Current >> Password" for better clarity. Also in line 41 the "Confirm" label should >> say "Verify Password" for consistency with the rest of the UI. >> >> 5. In index.html:44 it should be
instead of
    . This is an >> existing problem in migration/index.html. >> >> 6. There are broken image links in ipa_activation.css. The images have >> been slightly renamed and moved into install/ui/images recently. >> >> 7. There's a broken link to the CSS file in invalid.html:7. >> >> 8. The intall/activation/jquery-ui.css doesn't seem to be used anywhere. >> This is also an existing problem in migration. >> >> 9. In activation.py:72 the error message still says 'migration'. >> >> 10. In activation.py:121 the return statement in the 'finally' clause is >> redundant if there's an exception because it will return another page. >> This return statement should be moved inside the 'try' clause. Also we >> might need another return statement in the 'except' clause to handle the >> case where the errno doesn't match EPERM or EIO. >> >> 11. The file actually contains 3 patches. It's better to put them in >> separate files because in case one of them needs revision you won't have >> to resend everything. Another option is to squash them into a single >> patch. >> >> 12. The second patch contains the first patch file. >> >> 13. There are some warnings about extra whitespaces when applying the >> patch. Try applying the patch with --whitespace=fix then reexporting it >> again. >> > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From mkosek at redhat.com Wed Nov 30 08:53:57 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 30 Nov 2011 09:53:57 +0100 Subject: [Freeipa-devel] make-lint failures In-Reply-To: <20111129221222.GB24514@redhat.com> References: <4ED552FE.8020000@redhat.com> <20111129221222.GB24514@redhat.com> Message-ID: <1322643237.3868.25.camel@balmora.brq.redhat.com> On Wed, 2011-11-30 at 00:12 +0200, Alexander Bokovoy wrote: > On Tue, 29 Nov 2011, Rob Crittenden wrote: > > Seeing some new make-lint failures after the reworking of ipaldap > > function signatures; > > > > $ ./make-lint > > ipaserver/install/adtrustinstance.py:101: [E1120, > > ADTRUSTInstance.__create_samba_user] No value passed for parameter > > 'modlist' in function call > > ipaserver/install/adtrustinstance.py:190: [E1120, > > ADTRUSTInstance.__create_samba_domain_object] No value passed for > > parameter 'modlist' in function call > > ipaserver/install/adtrustinstance.py:198: [E1120, > > ADTRUSTInstance.__create_samba_domain_object] No value passed for > > parameter 'modlist' in function call > > > > I wonder if the signature needs to be: > > > > def add_s(self, dn, modlist=None): > > > > For the case were dn isn't an Entry we probably need to raise an > > exception if modlist is None (or test to see what python-ldap add_s > > does). > The original LDAPObject.add_s() has modlist as non-optional argument: > def add_s(self,dn,modlist): > > I don't think it is wise to break that API assumption. > > In all the cases above it should get .add_s(entry) replaced by > .addEntry(entry). > > The reason these failures are shown is because Martin reverted the > earlier version of Sumit's patch that you mistakenly committed. Sumit > has produced new patch already but there is one minor issue in it > (another .add_s() -> .addEntry() replacement needs to be done). > I don't think this is the reason. My revert patch (Revert "Add DNS service records for Windows") just removed the lines that Rob pushed, it did not touch any add_s() call. Even the Sumit's rebased patch (freeipa-sbose-0008-5-Add-DNS-service-records-for-Windows.patch) does not solve these pylint issues. Martin From abokovoy at redhat.com Wed Nov 30 09:06:18 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 30 Nov 2011 11:06:18 +0200 Subject: [Freeipa-devel] make-lint failures In-Reply-To: <1322643237.3868.25.camel@balmora.brq.redhat.com> References: <4ED552FE.8020000@redhat.com> <20111129221222.GB24514@redhat.com> <1322643237.3868.25.camel@balmora.brq.redhat.com> Message-ID: <20111130090618.GA304@redhat.com> On Wed, 30 Nov 2011, Martin Kosek wrote: > > The original LDAPObject.add_s() has modlist as non-optional argument: > > def add_s(self,dn,modlist): > > > > I don't think it is wise to break that API assumption. > > > > In all the cases above it should get .add_s(entry) replaced by > > .addEntry(entry). > > > > The reason these failures are shown is because Martin reverted the > > earlier version of Sumit's patch that you mistakenly committed. Sumit > > has produced new patch already but there is one minor issue in it > > (another .add_s() -> .addEntry() replacement needs to be done). > > > > I don't think this is the reason. My revert patch (Revert "Add DNS > service records for Windows") just removed the lines that Rob pushed, it > did not touch any add_s() call. John's patch removed wrappers around original SimpleLDAPObject's methods that allowed to pass anything you want there without arguments checking. These wrappers are not visible to pylint on F17/Rawhide and essentially we are getting there same warnings with John's patch on F16 and earlier because with the patch pylint now sees proper signatures. We need to review all .add_s() calls to ensure their correctness. .modify_s() are simpler because they always were used with two arguments (dn and list of modifications) but in case of .add_s() we were abusing it with Entry as a single argument and were relying on the magic of wrapper methods to do transformations. John's patch gives you this: def add_s(self, dn, modlist): if isinstance(dn, Entry): return IPASimpleLDAPObject.add_s(self, dn.dn, dn.toTupleList()) else: return IPASimpleLDAPObject.add_s(self, dn, modlist) but I think it is semantically better to be explicit and use .addEntry() here, especially if you have already went and created Entry few lines before that. I would see these .add_s() methods as a convenience tool for API compliance if you need quickly convert some other LDAP-based Python code being merged to the framework. > Even the Sumit's rebased patch > (freeipa-sbose-0008-5-Add-DNS-service-records-for-Windows.patch) does > not solve these pylint issues. Because it didn't cover all the .add_s() calls. I asked Sumit to consider those cases as well. -- / Alexander Bokovoy From sbose at redhat.com Wed Nov 30 09:11:35 2011 From: sbose at redhat.com (Sumit Bose) Date: Wed, 30 Nov 2011 10:11:35 +0100 Subject: [Freeipa-devel] [PATCH] Add ipasam samba passdb backend In-Reply-To: <20111129212541.GA24514@redhat.com> References: <20111123154802.GI2205@localhost.localdomain> <20111128172227.GE2246@localhost.localdomain> <20111129210926.GK2246@localhost.localdomain> <20111129212541.GA24514@redhat.com> Message-ID: <20111130091134.GA2348@localhost.localdomain> On Tue, Nov 29, 2011 at 11:25:41PM +0200, Alexander Bokovoy wrote: > On Tue, 29 Nov 2011, Sumit Bose wrote: > > @@ -199,10 +216,11 @@ class ADTRUSTInstance(service.Service): > > self.admin_conn.addEntry(entry) > > > > entry = ipaldap.Entry(self.smb_dom_dn) > > - entry.setValues("objectclass", ["sambaDomain", "nsContainer"]) > > + entry.setValues("objectclass", [self.OBJC_DOMAIN, "nsContainer"]) > > entry.setValues("cn", self.domain_name) > > - entry.setValues("sambaDomainName", self.netbios_name) > > - entry.setValues("sambaSID", self.__gen_sid_string()) > > + entry.setValues(self.ATTR_FLAT_NAME, self.netbios_name) > > + entry.setValues(self.ATTR_SID, self.__gen_sid_string()) > > + entry.setValues(self.ATTR_GUID, str(uuid.uuid4())) > > #TODO: which MAY attributes do we want to set ? > > self.admin_conn.add_s(entry) > Could you please also convert this one to .addEntry(entry)? > I think it is the last one left... This is fixed in freeipa-sbose-0012-3-Fix-some-pylint-warnings.patch. Martin, shall I extract the add_s->addEntry changes into a separate patch so the they can reviewed indepently of the 6 ipasam patches? bye, Sumit > > -- > / Alexander Bokovoy From mkosek at redhat.com Wed Nov 30 10:22:13 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 30 Nov 2011 11:22:13 +0100 Subject: [Freeipa-devel] [PATCH] Add ipasam samba passdb backend In-Reply-To: <20111130091134.GA2348@localhost.localdomain> References: <20111123154802.GI2205@localhost.localdomain> <20111128172227.GE2246@localhost.localdomain> <20111129210926.GK2246@localhost.localdomain> <20111129212541.GA24514@redhat.com> <20111130091134.GA2348@localhost.localdomain> Message-ID: <1322648533.2924.1.camel@balmora.brq.redhat.com> On Wed, 2011-11-30 at 10:11 +0100, Sumit Bose wrote: > On Tue, Nov 29, 2011 at 11:25:41PM +0200, Alexander Bokovoy wrote: > > On Tue, 29 Nov 2011, Sumit Bose wrote: > > > @@ -199,10 +216,11 @@ class ADTRUSTInstance(service.Service): > > > self.admin_conn.addEntry(entry) > > > > > > entry = ipaldap.Entry(self.smb_dom_dn) > > > - entry.setValues("objectclass", ["sambaDomain", "nsContainer"]) > > > + entry.setValues("objectclass", [self.OBJC_DOMAIN, "nsContainer"]) > > > entry.setValues("cn", self.domain_name) > > > - entry.setValues("sambaDomainName", self.netbios_name) > > > - entry.setValues("sambaSID", self.__gen_sid_string()) > > > + entry.setValues(self.ATTR_FLAT_NAME, self.netbios_name) > > > + entry.setValues(self.ATTR_SID, self.__gen_sid_string()) > > > + entry.setValues(self.ATTR_GUID, str(uuid.uuid4())) > > > #TODO: which MAY attributes do we want to set ? > > > self.admin_conn.add_s(entry) > > Could you please also convert this one to .addEntry(entry)? > > I think it is the last one left... > > This is fixed in freeipa-sbose-0012-3-Fix-some-pylint-warnings.patch. > > Martin, shall I extract the add_s->addEntry changes into a separate > patch so the they can reviewed indepently of the 6 ipasam patches? > > bye, > Sumit > I think it is OK to review (and push) them in a scope of your patch 0012-3, we don't have to divide them. Martin From jdennis at redhat.com Wed Nov 30 13:22:02 2011 From: jdennis at redhat.com (John Dennis) Date: Wed, 30 Nov 2011 08:22:02 -0500 Subject: [Freeipa-devel] make-lint failures In-Reply-To: <20111130090618.GA304@redhat.com> References: <4ED552FE.8020000@redhat.com> <20111129221222.GB24514@redhat.com> <1322643237.3868.25.camel@balmora.brq.redhat.com> <20111130090618.GA304@redhat.com> Message-ID: <4ED62DFA.8040608@redhat.com> On 11/30/2011 04:06 AM, Alexander Bokovoy wrote: > We need to review all .add_s() calls to ensure their correctness. > .modify_s() are simpler because they always were used with two > arguments (dn and list of modifications) but in case of .add_s() we > were abusing it with Entry as a single argument and were relying on > the magic of wrapper methods to do transformations. > > John's patch gives you this: > def add_s(self, dn, modlist): > if isinstance(dn, Entry): > return IPASimpleLDAPObject.add_s(self, dn.dn, dn.toTupleList()) > else: > return IPASimpleLDAPObject.add_s(self, dn, modlist) > > but I think it is semantically better to be explicit and use > .addEntry() here, especially if you have already went and created > Entry few lines before that. +1 I agree it's better to use addEntry(). There is a fair amount of abuse of the LDAP API in the installer. What I did was a interim step to help us pay attention to what we're passing. If I recall correctly in the DN refactoring work which has been sitting in a branch of mine I replaced all calls passing an Entry to add_s() with addEntry(). Cleaning up our use of the LDAP API's is high on my list because we've abused in so many places which in turn has made doing the DN conversion work unnecessarily difficult. What I think we really need to do is use only one single IPA specific API for talking to LDAP, currently that's ldap2, but I guess that's a task for another day. I would see these .add_s() methods as a > convenience tool for API compliance if you need quickly convert some > other LDAP-based Python code being merged to the framework. > >> Even the Sumit's rebased patch >> (freeipa-sbose-0008-5-Add-DNS-service-records-for-Windows.patch) does >> not solve these pylint issues. > Because it didn't cover all the .add_s() calls. I asked Sumit to > consider those cases as well. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From sbose at redhat.com Wed Nov 30 13:40:29 2011 From: sbose at redhat.com (Sumit Bose) Date: Wed, 30 Nov 2011 14:40:29 +0100 Subject: [Freeipa-devel] Samba package name change samba-4.0 -> samba4 Message-ID: <20111130134029.GF2348@localhost.localdomain> Hi, we recently changed the name of the samba packages in the ipa-devel respository. The packages are now called samba4-* and libsmbclient4-* instead of samba-4.0-* and libsmbclient-4.0-* . The name was changed because the samba packages will updated the samba4 packages which are currently available for Fedora and RHEL6 in some time. There is no other way as to manually remove the old packages and install the new ones. Sorry for any inconveniences. bye, Sumit P.S. I wonder if someone can commit the following patch under the one-line rule? diff --git a/freeipa.spec.in b/freeipa.spec.in index 282f189..c0fbc38 100644 --- a/freeipa.spec.in +++ b/freeipa.spec.in @@ -31,7 +31,7 @@ BuildRequires: policycoreutils >= %{POLICYCOREUTILSVER} %if 0%{?fedora} >= 16 BuildRequires: systemd-units %endif -BuildRequires: samba-4.0-devel +BuildRequires: samba4-devel %endif BuildRequires: nspr-devel BuildRequires: nss-devel From sgallagh at redhat.com Wed Nov 30 13:46:04 2011 From: sgallagh at redhat.com (Stephen Gallagher) Date: Wed, 30 Nov 2011 08:46:04 -0500 Subject: [Freeipa-devel] Samba package name change samba-4.0 -> samba4 In-Reply-To: <20111130134029.GF2348@localhost.localdomain> References: <20111130134029.GF2348@localhost.localdomain> Message-ID: <1322660764.4990.17.camel@sgallagh520.sgallagh.bos.redhat.com> On Wed, 2011-11-30 at 14:40 +0100, Sumit Bose wrote: > Hi, > > we recently changed the name of the samba packages in the ipa-devel > respository. The packages are now called samba4-* and libsmbclient4-* > instead of samba-4.0-* and libsmbclient-4.0-* . > > The name was changed because the samba packages will updated the samba4 > packages which are currently available for Fedora and RHEL6 in some time. > > There is no other way as to manually remove the old packages and install the > new ones. Yes there is. Mark the new packages as "Obsoletes: samba-4.0" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From jcholast at redhat.com Wed Nov 30 14:19:10 2011 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 30 Nov 2011 15:19:10 +0100 Subject: [Freeipa-devel] [PATCH] 55 Parse comma-separated lists of values in all parameter types In-Reply-To: <1322063183.27504.26.camel@balmora.brq.redhat.com> References: <4EA00403.9060703@redhat.com> <4EA583AB.6030906@redhat.com> <4EA5877B.2020606@redhat.com> <4EA59620.30506@redhat.com> <4EB45837.2020606@redhat.com> <4EB7B023.1030004@redhat.com> <4EBD7279.8030106@redhat.com> <4ECA7807.4080606@redhat.com> <1322038359.27504.4.camel@balmora.brq.redhat.com> <4ECCC769.7070303@redhat.com> <1322063183.27504.26.camel@balmora.brq.redhat.com> Message-ID: <4ED63B5E.7060805@redhat.com> Dne 23.11.2011 16:46, Martin Kosek napsal(a): > On Wed, 2011-11-23 at 11:14 +0100, Jan Cholasta wrote: >> Dne 23.11.2011 09:52, Martin Kosek napsal(a): > ... >> >> Well, the crash is obviously not related to the patch. Fedora 16 uses a >> newer version of pylint (0.24), which causes the crash (it works fine on >> Fedora 15 with pylint 0.22). >> >> I have opened a ticket to resolve that: >> https://fedorahosted.org/freeipa/ticket/2136 > > The crash _is_ related to the patch since the combination of pylint and > the changed source code causes it :-) But of course, it does not mean > that the patch is wrong. > > If possible, I would like to push both your acked patch 55 and a patch > fixing 2136 at one time. I don't want to confuse F-16 users with this > crash when they try to compile our sources. Therefore, I moved 2136 to > November milestone. > > Martin > Fixed the crash (patch 58, see attachment) and rebased the CSV patch to current master. Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-55.3-comma-separated-list.patch Type: text/x-patch Size: 131989 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-58-make-lint-crash.patch Type: text/x-patch Size: 773 bytes Desc: not available URL: From sbose at redhat.com Wed Nov 30 14:21:24 2011 From: sbose at redhat.com (Sumit Bose) Date: Wed, 30 Nov 2011 15:21:24 +0100 Subject: [Freeipa-devel] Samba package name change samba-4.0 -> samba4 In-Reply-To: <1322660764.4990.17.camel@sgallagh520.sgallagh.bos.redhat.com> References: <20111130134029.GF2348@localhost.localdomain> <1322660764.4990.17.camel@sgallagh520.sgallagh.bos.redhat.com> Message-ID: <20111130142123.GG2348@localhost.localdomain> On Wed, Nov 30, 2011 at 08:46:04AM -0500, Stephen Gallagher wrote: > On Wed, 2011-11-30 at 14:40 +0100, Sumit Bose wrote: > > Hi, > > > > we recently changed the name of the samba packages in the ipa-devel > > respository. The packages are now called samba4-* and libsmbclient4-* > > instead of samba-4.0-* and libsmbclient-4.0-* . > > > > The name was changed because the samba packages will updated the samba4 > > packages which are currently available for Fedora and RHEL6 in some time. > > > > There is no other way as to manually remove the old packages and install the > > new ones. > > Yes there is. Mark the new packages as "Obsoletes: samba-4.0" ok, I meant there is no way, because this Obsoletes wasn't added so far. Andreas, can you add this Obsoletes with a comment that it can be removed for offical Fedora and RHEL packages? bye, Sumit > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From mkosek at redhat.com Wed Nov 30 16:10:52 2011 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 30 Nov 2011 17:10:52 +0100 Subject: [Freeipa-devel] [PATCH] 55 Parse comma-separated lists of values in all parameter types In-Reply-To: <4ED63B5E.7060805@redhat.com> References: <4EA00403.9060703@redhat.com> <4EA583AB.6030906@redhat.com> <4EA5877B.2020606@redhat.com> <4EA59620.30506@redhat.com> <4EB45837.2020606@redhat.com> <4EB7B023.1030004@redhat.com> <4EBD7279.8030106@redhat.com> <4ECA7807.4080606@redhat.com> <1322038359.27504.4.camel@balmora.brq.redhat.com> <4ECCC769.7070303@redhat.com> <1322063183.27504.26.camel@balmora.brq.redhat.com> <4ED63B5E.7060805@redhat.com> Message-ID: <1322669452.2924.25.camel@balmora.brq.redhat.com> On Wed, 2011-11-30 at 15:19 +0100, Jan Cholasta wrote: > Dne 23.11.2011 16:46, Martin Kosek napsal(a): > > On Wed, 2011-11-23 at 11:14 +0100, Jan Cholasta wrote: > >> Dne 23.11.2011 09:52, Martin Kosek napsal(a): > > ... > >> > >> Well, the crash is obviously not related to the patch. Fedora 16 uses a > >> newer version of pylint (0.24), which causes the crash (it works fine on > >> Fedora 15 with pylint 0.22). > >> > >> I have opened a ticket to resolve that: > >> https://fedorahosted.org/freeipa/ticket/2136 > > > > The crash _is_ related to the patch since the combination of pylint and > > the changed source code causes it :-) But of course, it does not mean > > that the patch is wrong. > > > > If possible, I would like to push both your acked patch 55 and a patch > > fixing 2136 at one time. I don't want to confuse F-16 users with this > > crash when they try to compile our sources. Therefore, I moved 2136 to > > November milestone. > > > > Martin > > > > Fixed the crash (patch 58, see attachment) and rebased the CSV patch to > current master. > > Honza > ACK. Both pushed to master. Martin From abokovoy at redhat.com Wed Nov 30 16:14:27 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 30 Nov 2011 18:14:27 +0200 Subject: [Freeipa-devel] [Package: freeipa-2.1.3-7.fc16 Tag: f16-updates-candidate Status: complete Built by: abbra] Message-ID: <20111130161418.GA20134@redhat.com> Hi, and update for migrating from F15 to F16 (SysV to systemd) is ready for testing. I'll push it through bodhi soon. This package upon package update should notice inactive ipa.service and trigger upgrade procedure. If there is no FreeIPA install found, the upgrade will do nothing. If there is existing installation, it will attempt to create proper systemd services' configuration like we do during ipa-server-install and also recover Dogtag instance symlinks. ----- Forwarded message from Fedora Koji Build System ----- > Date: Wed, 30 Nov 2011 16:04:21 +0000 (UTC) > From: Fedora Koji Build System > Subject: Package: freeipa-2.1.3-7.fc16 Tag: f16-updates-candidate Status: complete Built by: abbra > > Package: freeipa-2.1.3-7.fc16 > Tag: f16-updates-candidate > Status: complete > Built by: abbra > ID: 276475 > Started: Wed, 30 Nov 2011 15:59:04 UTC > Finished: Wed, 30 Nov 2011 16:03:51 UTC > Changelog: > * Wed Nov 30 2011 Alexander Bokovoy - 2.1.3-7 > - Fix typo in install of freeipa-systemd-upgrade script > > * Wed Nov 30 2011 Alexander Bokovoy - 2.1.3-6 > - Introduce upgrade script to recover existing configuration after systemd migration > as user has no means to recover FreeIPA from systemd migration > - Upgrade script: > - recovers symlinks in Dogtag instance install > - recovers systemd configuration for FreeIPA's directory server instances > - recovers freeipa.service > - migrates directory server and KDC configs to use proper keytabs for systemd services > > * Wed Oct 26 2011 Fedora Release Engineering - 2.1.3-5 > - Rebuilt for glibc bug#747377 > > > > SRPMS: > freeipa-2.1.3-7.fc16.src.rpm > > Closed tasks: > ------------- > > Task 3553281 on x86-05.phx2.fedoraproject.org > Task Type: buildSRPMFromSCM (/freeipa:ddcc80fce7f6553f71e53687528a6e6eac0bc4aa) > logs: > http://koji.fedoraproject.org/koji/getfile?taskID=3553281&name=build.log > http://koji.fedoraproject.org/koji/getfile?taskID=3553281&name=checkout.log > http://koji.fedoraproject.org/koji/getfile?taskID=3553281&name=mock_output.log > http://koji.fedoraproject.org/koji/getfile?taskID=3553281&name=root.log > http://koji.fedoraproject.org/koji/getfile?taskID=3553281&name=state.log > > Task 3553301 on x86-06.phx2.fedoraproject.org > Task Type: tagBuild (noarch) > > Task 3553286 on x86-14.phx2.fedoraproject.org > Task Type: buildArch (freeipa-2.1.3-7.fc16.src.rpm, x86_64) > logs: > http://koji.fedoraproject.org/koji/getfile?taskID=3553286&name=build.log > http://koji.fedoraproject.org/koji/getfile?taskID=3553286&name=mock_output.log > http://koji.fedoraproject.org/koji/getfile?taskID=3553286&name=root.log > http://koji.fedoraproject.org/koji/getfile?taskID=3553286&name=state.log > rpms: > http://kojipkgs.fedoraproject.org/packages/freeipa/2.1.3/7.fc16/x86_64/freeipa-admintools-2.1.3-7.fc16.x86_64.rpm > http://kojipkgs.fedoraproject.org/packages/freeipa/2.1.3/7.fc16/x86_64/freeipa-client-2.1.3-7.fc16.x86_64.rpm > http://kojipkgs.fedoraproject.org/packages/freeipa/2.1.3/7.fc16/x86_64/freeipa-debuginfo-2.1.3-7.fc16.x86_64.rpm > http://kojipkgs.fedoraproject.org/packages/freeipa/2.1.3/7.fc16/x86_64/freeipa-python-2.1.3-7.fc16.x86_64.rpm > http://kojipkgs.fedoraproject.org/packages/freeipa/2.1.3/7.fc16/x86_64/freeipa-server-2.1.3-7.fc16.x86_64.rpm > http://kojipkgs.fedoraproject.org/packages/freeipa/2.1.3/7.fc16/x86_64/freeipa-server-selinux-2.1.3-7.fc16.x86_64.rpm > > Task 3553287 on x86-03.phx2.fedoraproject.org > Task Type: buildArch (freeipa-2.1.3-7.fc16.src.rpm, i686) > logs: > http://koji.fedoraproject.org/koji/getfile?taskID=3553287&name=build.log > http://koji.fedoraproject.org/koji/getfile?taskID=3553287&name=mock_output.log > http://koji.fedoraproject.org/koji/getfile?taskID=3553287&name=root.log > http://koji.fedoraproject.org/koji/getfile?taskID=3553287&name=state.log > rpms: > http://kojipkgs.fedoraproject.org/packages/freeipa/2.1.3/7.fc16/i686/freeipa-admintools-2.1.3-7.fc16.i686.rpm > http://kojipkgs.fedoraproject.org/packages/freeipa/2.1.3/7.fc16/i686/freeipa-client-2.1.3-7.fc16.i686.rpm > http://kojipkgs.fedoraproject.org/packages/freeipa/2.1.3/7.fc16/i686/freeipa-debuginfo-2.1.3-7.fc16.i686.rpm > http://kojipkgs.fedoraproject.org/packages/freeipa/2.1.3/7.fc16/i686/freeipa-python-2.1.3-7.fc16.i686.rpm > http://kojipkgs.fedoraproject.org/packages/freeipa/2.1.3/7.fc16/i686/freeipa-server-2.1.3-7.fc16.i686.rpm > http://kojipkgs.fedoraproject.org/packages/freeipa/2.1.3/7.fc16/i686/freeipa-server-selinux-2.1.3-7.fc16.i686.rpm > > > > Task Info: http://koji.fedoraproject.org/koji/taskinfo?taskID=3553280 > Build Info: http://koji.fedoraproject.org/koji/buildinfo?buildID=276475 ----- End forwarded message ----- -- / Alexander Bokovoy From abokovoy at redhat.com Wed Nov 30 16:19:17 2011 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 30 Nov 2011 18:19:17 +0200 Subject: [Freeipa-devel] [Package: freeipa-2.1.3-7.fc16 Tag: f16-updates-candidate Status: complete Built by: abbra] In-Reply-To: <20111130161418.GA20134@redhat.com> References: <20111130161418.GA20134@redhat.com> Message-ID: <20111130161916.GA20286@redhat.com> On Wed, 30 Nov 2011, Alexander Bokovoy wrote: > > Hi, > > and update for migrating from F15 to F16 (SysV to systemd) is ready > for testing. I'll push it through bodhi soon. > > This package upon package update should notice inactive ipa.service > and trigger upgrade procedure. If there is no FreeIPA install found, > the upgrade will do nothing. If there is existing installation, it > will attempt to create proper systemd services' configuration like we > do during ipa-server-install and also recover Dogtag instance > symlinks. https://admin.fedoraproject.org/updates/freeipa-2.1.3-7.fc16 -- / Alexander Bokovoy From jdennis at redhat.com Wed Nov 30 16:31:59 2011 From: jdennis at redhat.com (John Dennis) Date: Wed, 30 Nov 2011 11:31:59 -0500 Subject: [Freeipa-devel] [PATCH 55/55] Restore default log level in server to INFO Message-ID: <201111301631.pAUGVx1H025114@int-mx01.intmail.prod.int.phx2.redhat.com> The default log level for server messages captured by httpd's error_log historically was INFO. The log_manager patch had it set to ERROR, this patch resets it back to INFO. Although it would have been trival to set the default_level to INFO in IPALogManager.configure_from_env() that is not logically the correct place. It would be much better if the default_level can be reset by simply assigning it to the log_mgr. To accomplish that LogManager.default_level was converted to a property with a getter and setter. The setter runs LogManager.apply_configuratin() after the default_level is modified. LogManager.set_default_level() was also added to allow simultaneously updating the configure_state. While testing some minor problems were observed and also fixed: * Removed some print statement which had been left in by mistake * Removed the ability to set the handler level in the config file because of chicken-and-egg issues of when handlers get created. The Env config file format is too inflexible to support detailed logging configuration. If the Env config format is ever made more flexible we can come back and add this back in. The handler config setting in Env had never been used and never worked so there is no issue in removing it. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: 0055-Restore-default-log-level-in-server-to-INFO.patch Type: text/x-patch Size: 7912 bytes Desc: not available URL: From pvoborni at redhat.com Wed Nov 30 16:37:54 2011 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 30 Nov 2011 17:37:54 +0100 Subject: [Freeipa-devel] [PATCH] 32-47 #2040, #1515 Refactor UI widgets In-Reply-To: <4ED5C96E.4000707@redhat.com> References: <4ED4E2B7.2020403@redhat.com> <4ED5C96E.4000707@redhat.com> Message-ID: <4ED65BE2.8050307@redhat.com> Comments in text. Sending also to devel list, attaching patches. On 11/30/2011 07:13 AM, Endi Sukma Dewata wrote: > On 11/29/2011 7:48 AM, Petr Vobornik wrote: >> Hi Endi, >> >> I have finally completed the #2040 and #1515 to the state that the UI is >> functional again. >> >> Unit tests are still broken, they don't reflect the changes yet. >> >> Unfortunately the nature of the effort is that the first patch breaks >> the UI and all following are fixing it, so they should be pushed/ACKed >> all or none until they are ready. Probably patch #37 and #42 maybe even #32 could be push separately, but I don't know if the rebasing is worth the time at this moment. >> >> The patches reflect the by path as I was making the UI usable again one >> entity after another - with share preparation (first six patches). > > I've not examined the code in great detail yet, I'm trying to test the > functionality first. So far major features are working, but I found some > issues below. If you think they can be fixed relatively quickly let's > focus on getting them fixed first. We can worry about cleaning up the > code later. > > >> Should I fix the units tests or rather review first yours HBAC testing >> patches? > > Let's do unit tests first because it should be part of these changes. > It's probably easier to rebase HBAC testing later. > >> I'm pretty sure, that something is still broken - its a lot of stuff to >> test. > > Here are the issues that I found: > > 1. The undo/undo all buttons for multivalued text don't work correctly. > Try editing a user, add an email, save it. Then delete the email, click > undo all (or reset), the undo button is still there. If you click it the > email will disappear before saving it. Fixed. Also fixed save method - it was creating array of array instead of only an array of values (Can be influenced by extract_child_value method). [split 2 patch #32] > > 2. Required field is not validated during update. Try editing a user, > remove the first name then save. It should have popped up an error > dialog and showed the validation error message underneath the field. Fixed - [removing section patch #36] > > 3. In the service adder dialog there's a field for the principal. The > field should not be visible. Fixed by small hack. Didn't want to rewrite whole dialog (or field). [services patch #41] > > 4. Open an automount location, click Add. It will go back to location > list. When you click the location again it will show a broken dialog. Fixed [automount patch #47] (new) > > 5. Clicking collapse all doesn't work, it will open a broken link. Fixed - [removing section patch #36] > > 6. Open a self service permission, click 'select all' checkbox. The > update should have become enabled. Fixed [aci patch #43] > > 7. Setting sudo rule category to 'all' (e.g. Anyone, Any Host) doesn't > work, it always reverts back to 'specified'. Fixed [sudo, hbac patch #45] > > 8. Go to Configuration, enable migration mode, save it. Then disable it, > it cannot be saved. This is actually an existing issue. This is a server side problem, right? I didn't manage to disable it even from CLI. > > 9. The policy classes should inherit from a common base class (e.g. > IPA.facet_policy) which defines empty base methods (e.g. post_create() > and post_load()). This way you don't need to check if a method exists > before calling it. Fixed [policy patch #37 + host #39, aci #43, dns patch #40] > > 10. In sudo.js you use apply() to call push() like this: > > spec.fields.push.apply(spec.fields, [...]); > > Why not call spec.fields.push(...) directly? > Fixed. My mindset was: i want to push an array. It would be probably OK if the array was already defined somewhere else. [sudo, hbac patch #45] -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0032-Splitting-widget-into-widget-and-field.patch Type: text/x-patch Size: 23487 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0033-Splitting-basic-widgets-into-visual-widgets-and-fiel.patch Type: text/x-patch Size: 44920 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0034-Improved-fields-dirty-status-detection-logic.patch Type: text/x-patch Size: 2417 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0035-Builders-and-collections-for-fields-and-widgets.patch Type: text/x-patch Size: 11459 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0036-Removing-sections-as-special-type-of-object.patch Type: text/x-patch Size: 51117 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0037-Added-possibility-to-define-facet-dialog-specific-po.patch Type: text/x-patch Size: 4934 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0038-Modifying-users-to-work-with-new-concept.patch Type: text/x-patch Size: 5381 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0039-Modifying-hosts-to-work-with-new-concept.patch Type: text/x-patch Size: 16326 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0040-Modifying-dns-to-work-with-new-concept.patch Type: text/x-patch Size: 10067 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0041-Modifying-services-to-work-with-new-concept.patch Type: text/x-patch Size: 9803 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0042-Separation-of-writable-update-from-field-load-method.patch Type: text/x-patch Size: 1315 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0043-Modifying-ACI-to-work-with-new-concept.patch Type: text/x-patch Size: 19177 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0044-Modifying-groups-to-work-with-new-concept.patch Type: text/x-patch Size: 1177 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0045-Code-cleanup-of-HBAC-Sudo-rules.patch Type: text/x-patch Size: 89319 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0046-Changing-definition-of-basic-fields-in-section-from-.patch Type: text/x-patch Size: 19400 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0047-Modifying-automount-to-work-with-new-concept.patch Type: text/x-patch Size: 3275 bytes Desc: not available URL: From jdennis at redhat.com Wed Nov 30 17:17:30 2011 From: jdennis at redhat.com (John Dennis) Date: Wed, 30 Nov 2011 12:17:30 -0500 Subject: [Freeipa-devel] [PATCH 55/55] Restore default log level in server to INFO In-Reply-To: <201111301631.pAUGVx1H025114@int-mx01.intmail.prod.int.phx2.redhat.com> References: <201111301631.pAUGVx1H025114@int-mx01.intmail.prod.int.phx2.redhat.com> Message-ID: <4ED6652A.5060109@redhat.com> Found & fixed minor issue with doc on default_level property, please use this version of the patch instead. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: 0055-Restore-default-log-level-in-server-to-INFO.patch Type: text/x-patch Size: 7919 bytes Desc: not available URL: From jdennis at redhat.com Wed Nov 30 22:33:22 2011 From: jdennis at redhat.com (John Dennis) Date: Wed, 30 Nov 2011 17:33:22 -0500 Subject: [Freeipa-devel] Session design document Message-ID: <4ED6AF32.6000607@redhat.com> -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ipa_session.txt URL: