From william.e.brown at adelaide.edu.au Wed May 2 02:07:43 2012 From: william.e.brown at adelaide.edu.au (William Brown) Date: Wed, 2 May 2012 11:37:43 +0930 Subject: [Freeipa-devel] DHCP integration into FreeIPA Message-ID: <3D8A99BD-30B0-4E2A-977C-D4125F73CA58@adelaide.edu.au> Hi, I believe the topic of DHCP integration has come up before. I think there have been other requests for this, but I think I would like to elaborate on some of mine (and others) thoughts on why this would be excellent in FreeIPA. When I refer to DHCP I speak of the ISC-DHCP3/4 servers. DHCP at the current point of time is difficult to manage in a larger and smaller business or network setup. In the smaller setup, there may not be enough expertise to go around which presents a key person risk, and for a large business, with hundreds to thousands of workstations, managing the dhcp configuration by hand becomes quite hard. As a result, some people have created tools that generate the configuration file and copy it out to servers, but this is quite a kludgy solution. Alternately, you can store the DHCP configuration is LDAP. Again, a tool must be written to manage this LDAP branch, as having people edit it by hand is inadvisable. However, as a result, these tools aren't released into the open source world, so no one can benefit from their presence. FreeIPA already has the majority of components in place to fill this gap (Namely, 389DS, DNS and access to hosts) - with a goal of managing Users and Hosts effectively, in my view, DHCP is one last pieces of the host management puzzle. DHCP would be similar to DNS in FreeIPA, in that it would be an optional component. During the install, just because you have opted for having DHCP support, should not make your FreeIPA server a DHCP server. The DHCP server "role" could be allocated to other hosts via the freeIPA admin tools. That way you don't need to install a FreeIPA domain controller at every location that needs DHCP. You also avoid the chicken and egg problem of "How does my FreeIPA server get an IP if the DHCP server is on another host that relies upon FreeIPA being available". This could also potentially take advantage of the concept of "locations" also. Having DHCP support would allow users to quickly and reliably setup network infrastructure, namely, DNS and DHCP on their systems. Additionally, having FreeIPA DHCP aware, would mean that for subnets you control, you can automatically generate the reverse hosts zone into DNS. You would gain an avenue of updating DNS names for hosts, without necessarily having the FreeIPA client tools installed. You could supplement this to show which hosts on a network are and are not part of the FreeIPA domain to allow easier auditing of systems. Users gain easy access to redundancy in DHCP server configuration, that is more difficult to achieve than with the traditional configuration files. Permissions over the control of DHCP (And potentially even subnets within) can be delegated to users and roles. The FreeIPA join tool can automatically create static host entries, and transmit the DHCP DUID (Both for IPv4 and IPv6) to the FreeIPA servers. Even if you don't "assign" an IPA to this static entry, this simplifies administration of hosts on a network. (Have you ever sat down and entered in 100 machines mac addresses manually into a web UI? It's not fun). In the future, this kind of integration would mean that an administrator could easily add PXE boot arguments to the DHCP server for tools like satellite kickstarting. (Which may even be exposed over an API and satellite can just hook into that .... the potential is great.) FreeIPA join can automatically enable DHCP6 on clients, allowing more network flexibility than standard router advertisement. You avoid people needing to write their own DHCP management solution that may have bugs or other latent issues, in favour of a high quality tool provided by FreeIPA. This becomes a very attractive feature to help with FreeIPA adoption. Thoughts, questions, comments? Sincerely, William Brown Research & Teaching, Technology Services The University of Adelaide, AUSTRALIA 5005 CRICOS Provider Number 00123M ----------------------------------------------------------------------------- IMPORTANT: This message may contain confidential or legally privileged information. If you think it was sent to you by mistake, please delete all copies and advise the sender. For the purposes of the SPAM Act 2003, this email is authorised by The University of Adelaide. pgp.mit.edu http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail URL: From pvoborni at redhat.com Wed May 2 13:33:56 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 02 May 2012 15:33:56 +0200 Subject: [Freeipa-devel] [PATCHES] 124-132 Inconsistent ways to show/change entry status Message-ID: <4FA137C4.7000101@redhat.com> This bunch of patches are implementing ticket #2247. They introduce some new logic and types of internal objects. There might be design issues (mainly in state evaluation). I would appreciate some opinions on what might be improved. See patch comments for more details. What I think might be the main concerns: Action list definition: Now action lists are defined on facet level and facet header takes them from facet. Would in be better to define action list - the widget and actions separately. The widget could be defined in header and actions on facet level. State evaluation: The patches are adding support for some kind of state evaluation. The state is represented by array of string. I'm thinking that the design might not be robust. Is a string good enough? We might have a problem with conditions like to "have this particular access right' (#2318). There are state_evaluators and state_listeners. They are practically the same but the execution point and parameters are different. Should they be somehow joined? Code placement: There's a lot of new objects and for some of them it is not clear to what code file they should be placed. FYI: In close future I would like to address some problems in UI architecture. I'm in a middle of designing phase, so there is nothing to present at the moment. The main topics are: * reduce the need of overriding methods when a new widgets or capabilities are added * make it more declarative to enhance extendibility It may be done by: * better inheriting model to support events * build phases (preinit, init, postinit, create, load) to improve spec object creating and initialization of created object. * path representation of an event/attribute/model property to support bindings to various events/attrs from anywhere * introduce model and model bindings, converters between command output/model/human readable representation -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0124-General-builder-support.patch Type: text/x-patch Size: 2605 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0125-Action-lists.patch Type: text/x-patch Size: 22063 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0126-Control-buttons.patch Type: text/x-patch Size: 9506 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0127-Redefined-details-control-buttons.patch Type: text/x-patch Size: 6697 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0128-Redefined-search-control-buttons.patch Type: text/x-patch Size: 8198 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0129-Hide-search-facet-add-delete-buttons-in-self-service.patch Type: text/x-patch Size: 8018 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0130-Batch-action-for-search-page-control-buttons.patch Type: text/x-patch Size: 1792 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0131-General-details-facet-actions.patch Type: text/x-patch Size: 7346 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0132-Consistent-change-of-entry-status.patch Type: text/x-patch Size: 21838 bytes Desc: not available URL: From mkosek at redhat.com Wed May 2 13:47:59 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 02 May 2012 15:47:59 +0200 Subject: [Freeipa-devel] [PATCH] 256 Make ipa 2.2 client capable of joining an older server Message-ID: <1335966479.7781.11.camel@balmora.brq.redhat.com> Testing instructions included in the ticket. --- IPA server of version 2.2 and higher supports Kerberos S4U2Proxy delegation, i.e. ipa command no longer forwards Kerberos TGT to the server during authentication. However, when IPA client of version 2.2 and higher tries to join an older IPA server, the installer crashes because the pre-2.2 server expects the TGT to be forwarded. This patch adds a fallback to ipa-client-install which would detect this situation and tries connecting with TGT forwarding enabled again. https://fedorahosted.org/freeipa/ticket/2697 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-256-ipa-client-install-delegate.patch Type: text/x-patch Size: 2970 bytes Desc: not available URL: From rcritten at redhat.com Wed May 2 14:32:54 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 02 May 2012 10:32:54 -0400 Subject: [Freeipa-devel] [PATCH] 256 Make ipa 2.2 client capable of joining an older server In-Reply-To: <1335966479.7781.11.camel@balmora.brq.redhat.com> References: <1335966479.7781.11.camel@balmora.brq.redhat.com> Message-ID: <4FA14596.5010506@redhat.com> Martin Kosek wrote: > Testing instructions included in the ticket. > --- > IPA server of version 2.2 and higher supports Kerberos S4U2Proxy > delegation, i.e. ipa command no longer forwards Kerberos TGT to the > server during authentication. However, when IPA client of version > 2.2 and higher tries to join an older IPA server, the installer > crashes because the pre-2.2 server expects the TGT to be forwarded. > > This patch adds a fallback to ipa-client-install which would detect > this situation and tries connecting with TGT forwarding enabled > again. > > https://fedorahosted.org/freeipa/ticket/2697 Still working on testing this, just a couple of initial comments. I'd like to see the second and 3rd exceptions be logged as well as printed to stderr (this is a common problem in ipa-client-install, we don't log as much as we should). Will it be confusing to print the bit about S4U2Proxy? I think simplyfing as "you are running a new client than the IPA server so some capabilities may not be available" or something like that. rob From ohamada at redhat.com Wed May 2 15:49:40 2012 From: ohamada at redhat.com (Ondrej Hamada) Date: Wed, 02 May 2012 17:49:40 +0200 Subject: [Freeipa-devel] [PATCH] 23 Allow one letter net/hostgroups names Message-ID: <4FA15794.9020501@redhat.com> https://fedorahosted.org/freeipa/ticket/2671 Changed regex validating net/hostgroup names to allow single letter names. Unit-tests added. But the current validation allows weird (host|net)group names like: ".", ".-", "..". I'm just not sure, do we really want to allow stuff like this? Patch also fixes one of netgroup and host unit-tests. The error message in hostname validation function has changed (in ticket #1966). -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ohamada-23-Allow-one-letter-net-hostgroups-names.patch Type: text/x-patch Size: 18923 bytes Desc: not available URL: From mkosek at redhat.com Wed May 2 16:14:14 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 02 May 2012 18:14:14 +0200 Subject: [Freeipa-devel] [PATCH] 256 Make ipa 2.2 client capable of joining an older server In-Reply-To: <4FA14596.5010506@redhat.com> References: <1335966479.7781.11.camel@balmora.brq.redhat.com> <4FA14596.5010506@redhat.com> Message-ID: <1335975254.7781.17.camel@balmora.brq.redhat.com> On Wed, 2012-05-02 at 10:32 -0400, Rob Crittenden wrote: > Martin Kosek wrote: > > Testing instructions included in the ticket. > > --- > > IPA server of version 2.2 and higher supports Kerberos S4U2Proxy > > delegation, i.e. ipa command no longer forwards Kerberos TGT to the > > server during authentication. However, when IPA client of version > > 2.2 and higher tries to join an older IPA server, the installer > > crashes because the pre-2.2 server expects the TGT to be forwarded. > > > > This patch adds a fallback to ipa-client-install which would detect > > this situation and tries connecting with TGT forwarding enabled > > again. > > > > https://fedorahosted.org/freeipa/ticket/2697 > > Still working on testing this, just a couple of initial comments. > > I'd like to see the second and 3rd exceptions be logged as well as > printed to stderr (this is a common problem in ipa-client-install, we > don't log as much as we should). > > Will it be confusing to print the bit about S4U2Proxy? I think > simplyfing as "you are running a new client than the IPA server so some > capabilities may not be available" or something like that. > > rob The attached patch has a better error reporting and logging. I also added user realm to keytab kinit as you suggested on IRC, it should make the kinit more bullet-proof. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-256-2-ipa-client-install-delegate.patch Type: text/x-patch Size: 3518 bytes Desc: not available URL: From dpal at redhat.com Wed May 2 16:26:48 2012 From: dpal at redhat.com (Dmitri Pal) Date: Wed, 02 May 2012 12:26:48 -0400 Subject: [Freeipa-devel] DHCP integration into FreeIPA In-Reply-To: <3D8A99BD-30B0-4E2A-977C-D4125F73CA58@adelaide.edu.au> References: <3D8A99BD-30B0-4E2A-977C-D4125F73CA58@adelaide.edu.au> Message-ID: <4FA16048.6010304@redhat.com> On 05/01/2012 10:07 PM, William Brown wrote: > Hi, > > I believe the topic of DHCP integration has come up before. I think > there have been other requests for this, but I think I would like to > elaborate on some of mine (and others) thoughts on why this would be > excellent in FreeIPA. When I refer to DHCP I speak of the ISC-DHCP3/4 > servers. > > DHCP at the current point of time is difficult to manage in a larger > and smaller business or network setup. In the smaller setup, there may > not be enough expertise to go around which presents a key person risk, > and for a large business, with hundreds to thousands of workstations, > managing the dhcp configuration by hand becomes quite hard. As a > result, some people have created tools that generate the configuration > file and copy it out to servers, but this is quite a kludgy solution. > Alternately, you can store the DHCP configuration is LDAP. Again, a > tool must be written to manage this LDAP branch, as having people edit > it by hand is inadvisable. However, as a result, these tools aren't > released into the open source world, so no one can benefit from their > presence. > > FreeIPA already has the majority of components in place to fill this > gap (Namely, 389DS, DNS and access to hosts) - with a goal of managing > Users and Hosts effectively, in my view, DHCP is one last pieces of > the host management puzzle. > > DHCP would be similar to DNS in FreeIPA, in that it would be an > optional component. > > During the install, just because you have opted for having DHCP > support, should not make your FreeIPA server a DHCP server. The DHCP > server "role" could be allocated to other hosts via the freeIPA admin > tools. That way you don't need to install a FreeIPA domain controller > at every location that needs DHCP. You also avoid the chicken and egg > problem of "How does my FreeIPA server get an IP if the DHCP server is > on another host that relies upon FreeIPA being available". This could > also potentially take advantage of the concept of "locations" also. > > Having DHCP support would allow users to quickly and reliably setup > network infrastructure, namely, DNS and DHCP on their systems. > Additionally, having FreeIPA DHCP aware, would mean that for subnets > you control, you can automatically generate the reverse hosts zone > into DNS. > > You would gain an avenue of updating DNS names for hosts, without > necessarily having the FreeIPA client tools installed. You could > supplement this to show which hosts on a network are and are not part > of the FreeIPA domain to allow easier auditing of systems. > > Users gain easy access to redundancy in DHCP server configuration, > that is more difficult to achieve than with the traditional > configuration files. > > Permissions over the control of DHCP (And potentially even subnets > within) can be delegated to users and roles. > > The FreeIPA join tool can automatically create static host entries, > and transmit the DHCP DUID (Both for IPv4 and IPv6) to the FreeIPA > servers. Even if you don't "assign" an IPA to this static entry, this > simplifies administration of hosts on a network. (Have you ever sat > down and entered in 100 machines mac addresses manually into a web UI? > It's not fun). In the future, this kind of integration would mean that > an administrator could easily add PXE boot arguments to the DHCP > server for tools like satellite kickstarting. (Which may even be > exposed over an API and satellite can just hook into that .... the > potential is great.) > > FreeIPA join can automatically enable DHCP6 on clients, allowing more > network flexibility than standard router advertisement. > > You avoid people needing to write their own DHCP management solution > that may have bugs or other latent issues, in favour of a high quality > tool provided by FreeIPA. This becomes a very attractive feature to > help with FreeIPA adoption. > > > Thoughts, questions, comments? > It makes sense as we start to understand more and more requirements, thank you. However we are currently swamped with other features and bugs. You can look into trac to see how many things are there on our plate. Would you be able to help and contribute to this effort? Thanks Dmitri > Sincerely, > > William Brown > > Research & Teaching, Technology Services > The University of Adelaide, AUSTRALIA 5005 > > CRICOS Provider Number 00123M > ----------------------------------------------------------------------------- > IMPORTANT: This message may contain confidential or legally privileged > information. If you think it was sent to you by mistake, please delete all > copies and advise the sender. For the purposes of the SPAM Act 2003, this > email is authorised by The University of Adelaide. > > pgp.mit.edu > http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 > > > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcritten at redhat.com Wed May 2 17:47:19 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 02 May 2012 13:47:19 -0400 Subject: [Freeipa-devel] [PATCH] 256 Make ipa 2.2 client capable of joining an older server In-Reply-To: <1335975254.7781.17.camel@balmora.brq.redhat.com> References: <1335966479.7781.11.camel@balmora.brq.redhat.com> <4FA14596.5010506@redhat.com> <1335975254.7781.17.camel@balmora.brq.redhat.com> Message-ID: <4FA17327.6030907@redhat.com> Martin Kosek wrote: > On Wed, 2012-05-02 at 10:32 -0400, Rob Crittenden wrote: >> Martin Kosek wrote: >>> Testing instructions included in the ticket. >>> --- >>> IPA server of version 2.2 and higher supports Kerberos S4U2Proxy >>> delegation, i.e. ipa command no longer forwards Kerberos TGT to the >>> server during authentication. However, when IPA client of version >>> 2.2 and higher tries to join an older IPA server, the installer >>> crashes because the pre-2.2 server expects the TGT to be forwarded. >>> >>> This patch adds a fallback to ipa-client-install which would detect >>> this situation and tries connecting with TGT forwarding enabled >>> again. >>> >>> https://fedorahosted.org/freeipa/ticket/2697 >> >> Still working on testing this, just a couple of initial comments. >> >> I'd like to see the second and 3rd exceptions be logged as well as >> printed to stderr (this is a common problem in ipa-client-install, we >> don't log as much as we should). >> >> Will it be confusing to print the bit about S4U2Proxy? I think >> simplyfing as "you are running a new client than the IPA server so some >> capabilities may not be available" or something like that. >> >> rob > > The attached patch has a better error reporting and logging. I also > added user realm to keytab kinit as you suggested on IRC, it should make > the kinit more bullet-proof. > > Martin ACK, pushed to master and ipa-2-2 rob From william at firstyear.id.au Wed May 2 22:01:51 2012 From: william at firstyear.id.au (William Brown) Date: Thu, 3 May 2012 07:31:51 +0930 Subject: [Freeipa-devel] DHCP integration into FreeIPA In-Reply-To: <4FA16048.6010304@redhat.com> References: <3D8A99BD-30B0-4E2A-977C-D4125F73CA58@adelaide.edu.au> <4FA16048.6010304@redhat.com> Message-ID: <5237D5A0-522E-46F6-9A1A-92C13ED31922@firstyear.id.au> Hi, > > It makes sense as we start to understand more and more requirements, > thank you. > However we are currently swamped with other features and bugs. You can > look into trac to see how many things are there on our plate. > > Would you be able to help and contribute to this effort? I can probably make a start soon - I'm personally swamped with work at the moment also. Some advice on "how" to start would be good also in terms of what kind of documentation you would like submitted, and the best way to start coding this would also be appreciated. Sincerely, William Brown pgp.mit.edu http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail URL: From pvoborni at redhat.com Thu May 3 08:59:05 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 03 May 2012 10:59:05 +0200 Subject: [Freeipa-devel] [PATCH] 133 Action list for user password Message-ID: <4FA248D9.50406@redhat.com> Currently the user password is shown as follows in the details page: Password: Reset Password This is inconsistent with the rest of the page because the 'Reset Password' is an action, not the value of the password. Now password is shown as follows: Password: ******* (if set) Password: (if not set) Reset password action was moved to Action List. https://fedorahosted.org/freeipa/ticket/2248 Note: I will address enabling/disabling 'reset password' option in action list in ticket #2318. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0133-Action-list-for-user-password.patch Type: text/x-patch Size: 10882 bytes Desc: not available URL: From pspacek at redhat.com Thu May 3 09:25:43 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 03 May 2012 11:25:43 +0200 Subject: [Freeipa-devel] [PATCH 0019] Add proper DN escaping before LDAP library calls Message-ID: <4FA24F17.2000106@redhat.com> Hello, this patch adds missing DNS->LDAP escaping conversion. It's necessary to prevent (potential) LDAP injection attacks in future. Code isn't very nice, because DNS users decimal escaping \123, LDAP uses hexadecimal escaping \ab and set of escaped characters is smaller in DNS than in LDAP. Any improvements are welcome. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0019-Add-proper-DN-escaping-before-LDAP-library-calls.patch Type: text/x-patch Size: 6387 bytes Desc: not available URL: From atkac at redhat.com Thu May 3 12:18:43 2012 From: atkac at redhat.com (Adam Tkac) Date: Thu, 3 May 2012 14:18:43 +0200 Subject: [Freeipa-devel] [PATCH 0018] Deadlock detection logic In-Reply-To: <4F96B000.204@redhat.com> References: <4F96A8E4.7010002@redhat.com> <4F96B000.204@redhat.com> Message-ID: <20120503121842.GA14952@redhat.com> On Tue, Apr 24, 2012 at 03:52:00PM +0200, Petr Spacek wrote: > On 04/24/2012 03:21 PM, Petr Spacek wrote: > >Hello, > > > >this patch adds deadlock detection (based on simple timeout) to current code. > >If (probable) deadlock is detected, current action is stopped with proper error. > > > >It properly detects "Simo's deadlock" with 'connections' parameter == 1. > >(Described in https://fedorahosted.org/bind-dyndb-ldap/ticket/66) > > > >Deadlock itself will be fixed by separate patch. > > > >Petr^2 Spacek > > Self-NACK :-) > > Second version of the patch is attached. ldap_pool_getconnection() > and ldap_pool_putconnection() now has same interface and more > consistent behaviour. > > Overall functionality is same. Hi, although I'm not fully happy with current design of the detection logic, we can include it before we create something better, for example based on thread IDs (one thread can acquire semaphore only one time). Please check my comments inside the patch. Regards, Adam > From ed7596a9410269c0c29418247971f262b7fa77f3 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Tue, 24 Apr 2012 15:09:32 +0200 > Subject: [PATCH] Add simple semaphore deadlock detection logic. > Signed-off-by: Petr Spacek > > --- > src/ldap_helper.c | 78 ++++++++++++++++++++++++++++++++--------------------- > src/semaphore.c | 26 +++++++++++++++--- > src/semaphore.h | 6 +++- > 3 files changed, 74 insertions(+), 36 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index 47c0559..f4c4d02 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -296,9 +296,10 @@ ldap_delete_zone2(ldap_instance_t *inst, dns_name_t *name, isc_boolean_t lock); > static isc_result_t ldap_pool_create(isc_mem_t *mctx, unsigned int connections, > ldap_pool_t **poolp); > static void ldap_pool_destroy(ldap_pool_t **poolp); > -static ldap_connection_t * ldap_pool_getconnection(ldap_pool_t *pool); > +static isc_result_t ldap_pool_getconnection(ldap_pool_t *pool, > + ldap_connection_t ** conn); > static void ldap_pool_putconnection(ldap_pool_t *pool, > - ldap_connection_t *ldap_conn); > + ldap_connection_t ** conn); > static isc_result_t ldap_pool_connect(ldap_pool_t *pool, > ldap_instance_t *ldap_inst); > > @@ -402,6 +403,10 @@ new_ldap_instance(isc_mem_t *mctx, const char *db_name, > ldap_settings[i++].target = &ldap_inst->dyn_update; > CHECK(set_settings(ldap_settings, argv)); > > + /* Set timer for deadlock detection inside semaphore_wait_timed . */ > + if (semaphore_wait_timeout.seconds < ldap_inst->timeout+SEM_WAIT_TIMEOUT_ADD) > + semaphore_wait_timeout.seconds = ldap_inst->timeout+SEM_WAIT_TIMEOUT_ADD; I'm affraid such low timeout (12 sec by default) will cause too many false positives. I recommend to set semaphore timeout to at least 60 seconds. > /* Validate and check settings. */ > str_toupper(ldap_inst->sasl_mech); > if (ldap_inst->connections < 1) { > @@ -1089,7 +1094,7 @@ isc_result_t > refresh_zones_from_ldap(ldap_instance_t *ldap_inst) > { > isc_result_t result = ISC_R_SUCCESS; > - ldap_connection_t *ldap_conn; > + ldap_connection_t *ldap_conn = NULL; > int zone_count = 0; > ldap_entry_t *entry; > dns_rbt_t *rbt = NULL; > @@ -1114,7 +1119,7 @@ refresh_zones_from_ldap(ldap_instance_t *ldap_inst) > > log_debug(2, "refreshing list of zones for %s", ldap_inst->db_name); > > - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); > + CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); > > CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(ldap_inst->base), > LDAP_SCOPE_SUBTREE, config_attrs, 0, > @@ -1227,7 +1232,7 @@ cleanup: > if (invalidate_nodechain) > dns_rbtnodechain_invalidate(&chain); > > - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); > + ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > > log_debug(2, "finished refreshing list of zones"); > > @@ -1381,7 +1386,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > dns_name_t *origin, ldapdb_nodelist_t *nodelist) > { > isc_result_t result; > - ldap_connection_t *ldap_conn; > + ldap_connection_t *ldap_conn = NULL; > ldap_entry_t *entry; > ld_string_t *string = NULL; > ldapdb_node_t *node; > @@ -1392,7 +1397,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > REQUIRE(nodelist != NULL); > > /* RRs aren't in the cache, perform ordinary LDAP query */ > - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); > + CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); > > INIT_LIST(*nodelist); > CHECK(str_new(mctx, &string)); > @@ -1439,7 +1444,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > result = ISC_R_SUCCESS; > > cleanup: > - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); > + ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > str_destroy(&string); > > return result; > @@ -1450,7 +1455,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > dns_name_t *origin, ldapdb_rdatalist_t *rdatalist) > { > isc_result_t result; > - ldap_connection_t *ldap_conn; > + ldap_connection_t *ldap_conn = NULL; > ldap_entry_t *entry; > ld_string_t *string = NULL; > ldap_cache_t *cache; > @@ -1468,12 +1473,11 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > return result; > > /* RRs aren't in the cache, perform ordinary LDAP query */ > - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); > - > INIT_LIST(*rdatalist); > CHECK(str_new(mctx, &string)); > CHECK(dnsname_to_dn(ldap_inst->zone_register, name, string)); > > + CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); > CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(string), > LDAP_SCOPE_BASE, NULL, 0, "(objectClass=idnsRecord)")); > > @@ -1500,7 +1504,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > result = ISC_R_NOTFOUND; > > cleanup: > - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); > + ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > str_destroy(&string); > > if (result != ISC_R_SUCCESS) > @@ -2250,7 +2254,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > zone_dn += 1; /* skip whitespace */ > } > > - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); > + CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); > CHECK(ldap_query(ldap_inst, ldap_conn, zone_dn, > LDAP_SCOPE_BASE, attrs, 0, > "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); > @@ -2481,9 +2485,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > } > > cleanup: > - if (ldap_conn != NULL) > - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); > - > + ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > str_destroy(&owner_dn_ptr); > str_destroy(&owner_dn); > free_ldapmod(mctx, &change[0]); > @@ -2565,15 +2567,18 @@ ldap_pool_destroy(ldap_pool_t **poolp) > MEM_PUT_AND_DETACH(pool); > } > > -static ldap_connection_t * > -ldap_pool_getconnection(ldap_pool_t *pool) > +static isc_result_t > +ldap_pool_getconnection(ldap_pool_t *pool, ldap_connection_t ** conn) > { > ldap_connection_t *ldap_conn = NULL; > unsigned int i; > + isc_result_t result; > > REQUIRE(pool != NULL); > + REQUIRE(conn != NULL && *conn == NULL); > + ldap_conn = *conn; > > - semaphore_wait(&pool->conn_semaphore); > + CHECK(semaphore_wait_timed(&pool->conn_semaphore)); > for (i = 0; i < pool->connections; i++) { > ldap_conn = pool->conns[i]; > if (isc_mutex_trylock(&ldap_conn->lock) == ISC_R_SUCCESS) > @@ -2583,16 +2588,30 @@ ldap_pool_getconnection(ldap_pool_t *pool) > RUNTIME_CHECK(ldap_conn != NULL); > > INIT_LIST(ldap_conn->ldap_entries); > + *conn = ldap_conn; > > - return ldap_conn; > +cleanup: > + if (result != ISC_R_SUCCESS) { > + log_error("timeout in ldap_pool_getconnection(): try to raise " > + "'connections' parameter; potential deadlock?"); > + } > + return result; > } > > static void > -ldap_pool_putconnection(ldap_pool_t *pool, ldap_connection_t *ldap_conn) > +ldap_pool_putconnection(ldap_pool_t *pool, ldap_connection_t **conn) > { > + REQUIRE(conn); Although this is correct, please use REQUIRE(conn != NULL); > + ldap_connection_t *ldap_conn = *conn; > + > + if (ldap_conn == NULL) > + return; > + > ldap_query_free(ldap_conn); > UNLOCK(&ldap_conn->lock); > semaphore_signal(&pool->conn_semaphore); > + > + *conn = NULL; > } > > static isc_result_t > @@ -2719,7 +2738,7 @@ update_action(isc_task_t *task, isc_event_t *event) > ldap_psearchevent_t *pevent = (ldap_psearchevent_t *)event; > isc_result_t result ; > ldap_instance_t *inst = NULL; > - ldap_connection_t *conn; > + ldap_connection_t *conn = NULL; > ldap_entry_t *entry; > isc_boolean_t delete = ISC_TRUE; > isc_mem_t *mctx; > @@ -2736,7 +2755,7 @@ update_action(isc_task_t *task, isc_event_t *event) > if (result != ISC_R_SUCCESS) > goto cleanup; > > - conn = ldap_pool_getconnection(inst->pool); > + CHECK(ldap_pool_getconnection(inst->pool, &conn)); > > CHECK(ldap_query(inst, conn, pevent->dn, > LDAP_SCOPE_BASE, attrs, 0, > @@ -2754,14 +2773,13 @@ update_action(isc_task_t *task, isc_event_t *event) > if (delete) > CHECK(ldap_delete_zone(inst, pevent->dn, ISC_TRUE)); > > - ldap_pool_putconnection(inst->pool, conn); > - > cleanup: > if (result != ISC_R_SUCCESS) > log_error("update_action (psearch) failed for %s. " > "Zones can be outdated, run `rndc reload`", > pevent->dn); > > + ldap_pool_putconnection(inst->pool, &conn); > isc_mem_free(mctx, pevent->dbname); > isc_mem_free(mctx, pevent->dn); > isc_mem_detach(&mctx); > @@ -2790,7 +2808,7 @@ update_config(isc_task_t *task, isc_event_t *event) > if (result != ISC_R_SUCCESS) > goto cleanup; > > - conn = ldap_pool_getconnection(inst->pool); > + CHECK(ldap_pool_getconnection(inst->pool, &conn)); > > CHECK(ldap_query(inst, conn, pevent->dn, > LDAP_SCOPE_BASE, attrs, 0, > @@ -2809,14 +2827,12 @@ update_config(isc_task_t *task, isc_event_t *event) > > > cleanup: > - if (conn != NULL) > - ldap_pool_putconnection(inst->pool, conn); > - > if (result != ISC_R_SUCCESS) > log_error("update_config (psearch) failed for %s. " > "Configuration can be outdated, run `rndc reload`", > pevent->dn); > > + ldap_pool_putconnection(inst->pool, &conn); > isc_mem_free(mctx, pevent->dbname); > isc_mem_free(mctx, pevent->dn); > isc_mem_detach(&mctx); > @@ -3079,7 +3095,7 @@ ldap_psearch_watcher(isc_threadarg_t arg) > tv.tv_usec = 0; > > /* Pick connection, one is reserved purely for this thread */ > - conn = ldap_pool_getconnection(inst->pool); > + CHECK(ldap_pool_getconnection(inst->pool, &conn)); > > /* Try to connect. */ > while (conn->handle == NULL) { > @@ -3189,7 +3205,7 @@ soft_err: > log_debug(1, "Ending ldap_psearch_watcher"); > > cleanup: > - ldap_pool_putconnection(inst->pool, conn); > + ldap_pool_putconnection(inst->pool, &conn); > > return (isc_threadresult_t)0; > } > diff --git a/src/semaphore.c b/src/semaphore.c > index 41d6a30..76e6aa2 100644 > --- a/src/semaphore.c > +++ b/src/semaphore.c > @@ -27,8 +27,19 @@ > #include > #include > #include > +#include > > #include "semaphore.h" > +#include "util.h" > + > +/* > + * Timer setting for deadlock detection. Format: seconds, nanoseconds. > + * These values will be overwriten during initialization > + * from set_settings() with max(setting+SEM_WAIT_TIMEOUT_ADD, curr_value). > + * > + * Initial value can be useful in early phases of initialization. > + */ > +isc_interval_t semaphore_wait_timeout = { 3, 0 }; > > /* > * Initialize a semaphore. > @@ -74,20 +85,27 @@ semaphore_destroy(semaphore_t *sem) > /* > * Wait on semaphore. This operation will try to acquire a lock on the > * semaphore. If the semaphore is already acquired as many times at it allows, > - * the function will block until someone releases the lock. > + * the function will block until someone releases the lock OR timeout expire. > + * > + * @return ISC_R_SUCCESS or ISC_R_TIMEDOUT or other errors from ISC libs > */ > -void > -semaphore_wait(semaphore_t *sem) > +isc_result_t > +semaphore_wait_timed(semaphore_t *sem) > { > + isc_result_t result; > + isc_time_t abs_timeout; > REQUIRE(sem != NULL); > > LOCK(&sem->mutex); > + CHECK(isc_time_nowplusinterval(&abs_timeout, &semaphore_wait_timeout)); The isc_time_nowplusinterval doesn't modify shared data, please call it before LOCK(). > while (sem->value <= 0) > - WAIT(&sem->cond, &sem->mutex); > + CHECK(WAITUNTIL(&sem->cond, &sem->mutex, &abs_timeout)); > sem->value--; > > +cleanup: > UNLOCK(&sem->mutex); > + return result; > } > > /* > diff --git a/src/semaphore.h b/src/semaphore.h > index 4ca4f65..5aef5d2 100644 > --- a/src/semaphore.h > +++ b/src/semaphore.h > @@ -24,6 +24,10 @@ > #include > #include > > +/* How many seconds will be added to user-defined connection parameter 'timeout'. */ > +#define SEM_WAIT_TIMEOUT_ADD 2 /* seconds */ > +extern isc_interval_t semaphore_wait_timeout; > + > /* > * Semaphore can be "acquired" multiple times. However, it has a maximum > * number of times someone can acquire him. If a semaphore is already acquired > @@ -40,7 +44,7 @@ typedef struct semaphore semaphore_t; > /* Public functions. */ > isc_result_t semaphore_init(semaphore_t *sem, int value); > void semaphore_destroy(semaphore_t *sem); > -void semaphore_wait(semaphore_t *sem); > +isc_result_t semaphore_wait_timed(semaphore_t *sem); > void semaphore_signal(semaphore_t *sem); > > #endif /* !_LD_SEMAPHORE_H_ */ > -- > 1.7.7.6 > -- Adam Tkac, Red Hat, Inc. From pvoborni at redhat.com Thu May 3 13:19:12 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 03 May 2012 15:19:12 +0200 Subject: [Freeipa-devel] [PATCHES] 124-132 Inconsistent ways to show/change entry status In-Reply-To: <4FA137C4.7000101@redhat.com> References: <4FA137C4.7000101@redhat.com> Message-ID: <4FA285D0.90301@redhat.com> I found that limitation of maximum pkey length in facet header is not working well. Attaching patch #134 which actually calculates it. On 05/02/2012 03:33 PM, Petr Vobornik wrote: > This bunch of patches are implementing ticket #2247. They introduce some > new logic and types of internal objects. There might be design issues > (mainly in state evaluation). I would appreciate some opinions on what > might be improved. > > See patch comments for more details. > > What I think might be the main concerns: > > Action list definition: > Now action lists are defined on facet level and facet header takes them > from facet. Would in be better to define action list - the widget and > actions separately. The widget could be defined in header and actions on > facet level. > > State evaluation: > The patches are adding support for some kind of state evaluation. The > state is represented by array of string. I'm thinking that the design > might not be robust. Is a string good enough? We might have a problem > with conditions like to "have this particular access right' (#2318). > There are state_evaluators and state_listeners. They are practically the > same but the execution point and parameters are different. Should they > be somehow joined? > > Code placement: > There's a lot of new objects and for some of them it is not clear to > what code file they should be placed. > > FYI: In close future I would like to address some problems in UI > architecture. I'm in a middle of designing phase, so there is nothing to > present at the moment. The main topics are: > * reduce the need of overriding methods when a new widgets or > capabilities are added > * make it more declarative to enhance extendibility > It may be done by: > * better inheriting model to support events > * build phases (preinit, init, postinit, create, load) to improve spec > object creating and initialization of created object. > * path representation of an event/attribute/model property to support > bindings to various events/attrs from anywhere > * introduce model and model bindings, converters between command > output/model/human readable representation > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0134-Improved-calculation-of-max-pkey-length-in-facet-hea.patch Type: text/x-patch Size: 2233 bytes Desc: not available URL: From pvoborni at redhat.com Thu May 3 13:26:56 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 03 May 2012 15:26:56 +0200 Subject: [Freeipa-devel] [PATCHES] 124-132 Inconsistent ways to show/change entry status In-Reply-To: <4FA285D0.90301@redhat.com> References: <4FA137C4.7000101@redhat.com> <4FA285D0.90301@redhat.com> Message-ID: <4FA287A0.2070107@redhat.com> On 05/03/2012 03:19 PM, Petr Vobornik wrote: > I found that limitation of maximum pkey length in facet header is not > working well. Attaching patch #134 which actually calculates it. I found useless line in the patch. Corrected version attached. > > On 05/02/2012 03:33 PM, Petr Vobornik wrote: >> This bunch of patches are implementing ticket #2247. They introduce some >> new logic and types of internal objects. There might be design issues >> (mainly in state evaluation). I would appreciate some opinions on what >> might be improved. >> >> See patch comments for more details. >> >> What I think might be the main concerns: >> >> Action list definition: >> Now action lists are defined on facet level and facet header takes them >> from facet. Would in be better to define action list - the widget and >> actions separately. The widget could be defined in header and actions on >> facet level. >> >> State evaluation: >> The patches are adding support for some kind of state evaluation. The >> state is represented by array of string. I'm thinking that the design >> might not be robust. Is a string good enough? We might have a problem >> with conditions like to "have this particular access right' (#2318). >> There are state_evaluators and state_listeners. They are practically the >> same but the execution point and parameters are different. Should they >> be somehow joined? >> >> Code placement: >> There's a lot of new objects and for some of them it is not clear to >> what code file they should be placed. >> >> FYI: In close future I would like to address some problems in UI >> architecture. I'm in a middle of designing phase, so there is nothing to >> present at the moment. The main topics are: >> * reduce the need of overriding methods when a new widgets or >> capabilities are added >> * make it more declarative to enhance extendibility >> It may be done by: >> * better inheriting model to support events >> * build phases (preinit, init, postinit, create, load) to improve spec >> object creating and initialization of created object. >> * path representation of an event/attribute/model property to support >> bindings to various events/attrs from anywhere >> * introduce model and model bindings, converters between command >> output/model/human readable representation >> >> >> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0134-1-Improved-calculation-of-max-pkey-length-in-facet-hea.patch Type: text/x-patch Size: 2145 bytes Desc: not available URL: From pspacek at redhat.com Thu May 3 13:46:36 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 03 May 2012 15:46:36 +0200 Subject: [Freeipa-devel] [PATCH 0019] Add proper DN escaping before LDAP library calls In-Reply-To: <4FA24F17.2000106@redhat.com> References: <4FA24F17.2000106@redhat.com> Message-ID: <4FA28C3C.1020309@redhat.com> On 05/03/2012 11:25 AM, Petr Spacek wrote: > Hello, > > this patch adds missing DNS->LDAP escaping conversion. It's necessary to > prevent (potential) LDAP injection attacks in future. > > Code isn't very nice, because DNS users decimal escaping \123, LDAP uses > hexadecimal escaping \ab and set of escaped characters is smaller in DNS than > in LDAP. > > Any improvements are welcome. > > Petr^2 Spacek Here is second version of the patch. Changes: - comments - order of [._-] in if() - function was renamed to dns_to_ldap_dn_escape() Escaping logic itself wasn't changed. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0019-2-Add-proper-DN-escaping-before-LDAP-library-calls.patch Type: text/x-patch Size: 6903 bytes Desc: not available URL: From pspacek at redhat.com Thu May 3 13:46:43 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 03 May 2012 15:46:43 +0200 Subject: [Freeipa-devel] [PATCH 0019] Add proper DN escaping before LDAP library calls In-Reply-To: <4FA24F17.2000106@redhat.com> References: <4FA24F17.2000106@redhat.com> Message-ID: <4FA28C43.6060408@redhat.com> On 05/03/2012 11:25 AM, Petr Spacek wrote: > Hello, > > this patch adds missing DNS->LDAP escaping conversion. It's necessary to > prevent (potential) LDAP injection attacks in future. > > Code isn't very nice, because DNS users decimal escaping \123, LDAP uses > hexadecimal escaping \ab and set of escaped characters is smaller in DNS than > in LDAP. > > Any improvements are welcome. > > Petr^2 Spacek Here is second version of the patch. Changes: - comments - order of [._-] in if() - function was renamed to dns_to_ldap_dn_escape() Escaping logic itself wasn't changed. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0019-2-Add-proper-DN-escaping-before-LDAP-library-calls.patch Type: text/x-patch Size: 6903 bytes Desc: not available URL: From pviktori at redhat.com Thu May 3 14:30:39 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 03 May 2012 16:30:39 +0200 Subject: [Freeipa-devel] [PATCH] 0045 Update hostname validator error messages in tests Message-ID: <4FA2968F.6090909@redhat.com> A recent patch changed the error message from the hostname validator. Update the tests to reflect this change. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0045-Update-hostname-validator-error-messages-in-tests.patch Type: text/x-patch Size: 4489 bytes Desc: not available URL: From mkosek at redhat.com Thu May 3 14:58:21 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 03 May 2012 16:58:21 +0200 Subject: [Freeipa-devel] [PATCH] 0045 Update hostname validator error messages in tests In-Reply-To: <4FA2968F.6090909@redhat.com> References: <4FA2968F.6090909@redhat.com> Message-ID: <1336057101.20409.4.camel@balmora.brq.redhat.com> On Thu, 2012-05-03 at 16:30 +0200, Petr Viktorin wrote: > A recent patch changed the error message from the hostname > validator. Update the tests to reflect this change. > Thanks, this fixes the test failures. All tests are clean. Pushed to master, ipa-2-2. Martin From ohamada at redhat.com Thu May 3 15:08:11 2012 From: ohamada at redhat.com (Ondrej Hamada) Date: Thu, 03 May 2012 17:08:11 +0200 Subject: [Freeipa-devel] [PATCH] 23 Allow one letter net/hostgroups names In-Reply-To: <4FA15794.9020501@redhat.com> References: <4FA15794.9020501@redhat.com> Message-ID: <4FA29F5B.5060501@redhat.com> On 05/02/2012 05:49 PM, Ondrej Hamada wrote: > https://fedorahosted.org/freeipa/ticket/2671 > > Changed regex validating net/hostgroup names to allow single letter > names. Unit-tests added. > > But the current validation allows weird (host|net)group names like: > ".", ".-", "..". > I'm just not sure, do we really want to allow stuff like this? > > Patch also fixes one of netgroup and host unit-tests. The error > message in hostname validation function has changed (in ticket #1966). > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel The unit-test for #1966 were corrected by Petr?. Rebased patch attached. -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ohamada-23-2-Allow-one-letter-net-hostgroups-names.patch Type: text/x-patch Size: 17534 bytes Desc: not available URL: From mkosek at redhat.com Thu May 3 15:18:48 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 03 May 2012 17:18:48 +0200 Subject: [Freeipa-devel] [PATCH] 0042-0048 AD trusts support (master) In-Reply-To: <1335446337.20837.20.camel@balmora.brq.redhat.com> References: <20120403104135.GD23171@redhat.com> <20120403145749.GD8996@localhost.localdomain> <1334242615.777.3.camel@balmora.brq.redhat.com> <20120412150803.GA24623@redhat.com> <1334243807.777.6.camel@balmora.brq.redhat.com> <1334903980.2985.6.camel@balmora.brq.redhat.com> <1335446337.20837.20.camel@balmora.brq.redhat.com> Message-ID: <1336058328.20409.13.camel@balmora.brq.redhat.com> On Thu, 2012-04-26 at 15:18 +0200, Martin Kosek wrote: > On Fri, 2012-04-20 at 08:39 +0200, Martin Kosek wrote: > > On Thu, 2012-04-12 at 17:16 +0200, Martin Kosek wrote: > > > On Thu, 2012-04-12 at 18:08 +0300, Alexander Bokovoy wrote: > > > > Hi Martin! > > > > > > > > On Thu, 12 Apr 2012, Martin Kosek wrote: > > > ... > > > > >3) I would not try to import ipaserver.dcerpc every time the command is > > > > >executed: > > > > >+ try: > > > > >+ import ipaserver.dcerpc > > > > >+ except Exception, e: > > > > >+ raise errors.NotFound(name=_('AD Trust setup'), > > > > >+ reason=_('Cannot perform join operation without Samba > > > > >4 python bindings installed')) > > > > > > > > > >I would rather do it once in the beginning and set a flag: > > > > > > > > > >try: > > > > > import ipaserver.dcerpc > > > > > _bindings_installed = True > > > > >except Exception: > > > > > _bindings_installed = False > > > > > > > > > >... > > > > The idea was that this code is only executed on the server. We need to > > > > differentiate between: > > > > - running on client > > > > - running on server, no samba4 python bindings > > > > - running on server with samba4 python bindings > > > > > > > > By making it executed all time you are affecting the client code as > > > > well while with current approach it only affects server side. > > > > > > Across our code base, this situation is currently solved with this > > > condition: > > > > > > if api.env.in_server and api.env.context in ['lite', 'server']: > > > # try-import block > > > > > > > > > > > > > > > >+ def execute(self, *keys, **options): > > > > >+ # Join domain using full credentials and with random trustdom > > > > >+ # secret (will be generated by the join method) > > > > >+ trustinstance = None > > > > >+ if not _bindings_installed: > > > > >+ raise errors.NotFound(name=_('AD Trust setup'), > > > > >+ reason=_('Cannot perform join operation without Samba > > > > >4 python bindings installed')) > > > > > > > > > > > > > > >4) Another import inside a function: > > > > >+ def arcfour_encrypt(key, data): > > > > >+ from Crypto.Cipher import ARC4 > > > > >+ c = ARC4.new(key) > > > > >+ return c.encrypt(data) > > > > Same here, it is only needed on server side. > > > > > > > > Let us get consensus over 3) and 4) and I'll fix patches altogether (and > > > > push). > > > > > > > > > > Yeah, I would fix in the same way as 3). > > > > > > > I am running another run of test to finish my review of your patches, > > but I stumbled in 389-ds error when I was installing IPA server from > > package built from your git tree: > > git://fedorapeople.org/home/fedora/abbra/public_git/freeipa.git > > > > # rpm -q freeipa-server 389-ds-base > > freeipa-server-2.99.0GITc30f375-0.fc17.x86_64 > > 389-ds-base-1.2.11-0.1.a1.fc17.x86_64 > > # ipa-server-install -p kokos123 -a kokos123 > > ... > > [16/18]: issuing RA agent certificate > > [17/18]: adding RA agent as a trusted user > > [18/18]: Configure HTTP to proxy connections > > done configuring pki-cad. > > Configuring directory server: Estimated time 1 minute > > [1/35]: creating directory server user > > [2/35]: creating directory server instance > > [3/35]: adding default schema > > [4/35]: enabling memberof plugin > > [5/35]: enabling referential integrity plugin > > [6/35]: enabling winsync plugin > > [7/35]: configuring replication version plugin > > [8/35]: enabling IPA enrollment plugin > > [9/35]: enabling ldapi > > [10/35]: configuring uniqueness plugin > > [11/35]: configuring uuid plugin > > [12/35]: configuring modrdn plugin > > [13/35]: enabling entryUSN plugin > > [14/35]: configuring lockout plugin > > [15/35]: creating indices > > [16/35]: configuring ssl for ds instance > > [17/35]: configuring certmap.conf > > [18/35]: configure autobind for root > > [19/35]: configure new location for managed entries > > [20/35]: restarting directory server > > [21/35]: adding default layout > > [22/35]: adding delegation layout > > ipa : CRITICAL Failed to load delegation.ldif: Command > > '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > > -f /tmp/tmpdXcWF3 -x -D cn=Directory Manager -y /tmp/tmp8qtnOS' returned > > non-zero exit status 255 > > [23/35]: adding replication acis > > ipa : CRITICAL Failed to load replica-acis.ldif: Command > > '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > > -f /tmp/tmptivfJ_ -x -D cn=Directory Manager -y /tmp/tmpr_Z1lp' returned > > non-zero exit status 255 > > [24/35]: creating container for managed entries > > ipa : CRITICAL Failed to load managed-entries.ldif: Command > > '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > > -f /tmp/tmpNkmoDk -x -D cn=Directory Manager -y /tmp/tmpXU0lbx' returned > > non-zero exit status 255 > > [25/35]: configuring user private groups > > ipa : CRITICAL Failed to load user_private_groups.ldif: Command > > '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > > -f /tmp/tmp7uDqaG -x -D cn=Directory Manager -y /tmp/tmp6E_uPl' returned > > non-zero exit status 255 > > [26/35]: configuring netgroups from hostgroups > > ipa : CRITICAL Failed to load host_nis_groups.ldif: Command > > '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > > -f /tmp/tmphxoVQf -x -D cn=Directory Manager -y /tmp/tmpsAhhwd' returned > > non-zero exit status 255 > > [27/35]: creating default Sudo bind user > > ipa : CRITICAL Failed to load sudobind.ldif: Command > > '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > > -f /tmp/tmpCVpYqT -x -D cn=Directory Manager -y /tmp/tmp97b_6d' returned > > non-zero exit status 255 > > [28/35]: creating default Auto Member layout > > ipa : CRITICAL Failed to load automember.ldif: Command > > '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > > -f /tmp/tmpvcFbwK -x -D cn=Directory Manager -y /tmp/tmpSUownE' returned > > non-zero exit status 255 > > [29/35]: creating default HBAC rule allow_all > > ipa : CRITICAL Failed to load default-hbac.ldif: Command > > '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > > -f /tmp/tmpYoYkBy -x -D cn=Directory Manager -y /tmp/tmp_9le4C' returned > > non-zero exit status 255 > > [30/35]: initializing group membership > > ipa : CRITICAL Failed to load memberof-task.ldif: Command > > '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > > -f /tmp/tmpD9mIxC -x -D cn=Directory Manager -y /tmp/tmpeTqozO' returned > > non-zero exit status 255 > > Unexpected error - see ipaserver-install.log for details: > > {'desc': "Can't contact LDAP server"} > > > > > > # tail /var/log/dirsrv/slapd-IDM-LAB-BOS-REDHAT-COM/errors > > [20/Apr/2012:02:19:16 -0400] - 389-Directory/1.2.11.a1 B2012.090.2135 > > starting up > > [20/Apr/2012:02:19:16 -0400] attrcrypt - No symmetric key found for > > cipher AES in backend userRoot, attempting to create one... > > [20/Apr/2012:02:19:16 -0400] attrcrypt - Key for cipher AES successfully > > generated and stored > > [20/Apr/2012:02:19:16 -0400] attrcrypt - No symmetric key found for > > cipher 3DES in backend userRoot, attempting to create one... > > [20/Apr/2012:02:19:16 -0400] attrcrypt - Key for cipher 3DES > > successfully generated and stored > > [20/Apr/2012:02:19:16 -0400] - slapd started. Listening on All > > Interfaces port 389 for LDAP requests > > [20/Apr/2012:02:19:16 -0400] - Listening on All Interfaces port 636 for > > LDAPS requests > > [20/Apr/2012:02:19:16 -0400] - Listening > > on /var/run/slapd-IDM-LAB-BOS-REDHAT-COM.socket for LDAPI requests > > [20/Apr/2012:02:19:17 -0400] - Skipping CoS Definition cn=Password > > Policy,cn=accounts,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com--no CoS > > Templates found, which should be added before the CoS Definition. > > [20/Apr/2012:02:19:17 -0400] entryrdn-index - _entryrdn_put_data: Adding > > the self link (62) failed: BDB0068 DB_LOCK_DEADLOCK: Locker killed to > > resolve a deadlock (-30993) > > > > Martin > > > > I reproduced this issue even on another clean VM, I filed a BZ for that: > https://bugzilla.redhat.com/show_bug.cgi?id=816590 > > Martin > With the development version of the fix for DS issue, I was able to continue with the review. I found the following issues: 1) You add cifs s4u2proxy record twice. This leads to an error message during ipa-adtrust-install: # ipa-server-install --setup-dns # ipa-adtrust-install ... [6/13]: setting password for the samba user [7/13]: adding cifs Kerberos principal ipa : CRITICAL Failed to add key for cifs/vm-109.idm.lab.bos.redhat.com at IDM.LAB.BOS.REDHAT.COM [8/13]: adding admin(group) SIDs [9/13]: activating CLDAP plugin ... 2) Typo in ipa-adtrust-install info text: Additionally you have to make sure the FreeIPA LDAP server cannot reached by any domain controller in the Active Directory domain by closing the s/reached/cannot be reached/ 3) Another s4u2proxy error in ipa-replica-install: # ipa-replica-install INFO_FILE ... [20/30]: restarting directory server [21/30]: setting up initial replication Starting replication, please wait until this has completed. Update in progress Update in progress Update in progress Update succeeded [22/30]: adding replication acis [23/30]: setting Auto Member configuration [24/30]: enabling S4U2Proxy delegation ipa : CRITICAL Failed to load replica-s4u2proxy.ldif: Command '/usr/bin/ldapmodify -h vm-098.idm.lab.bos.redhat.com -v -f /tmp/tmpGFqASL -x -D cn=Directory Manager -y /tmp/tmpBuxVf4' returned non-zero exit status 247 [25/30]: initializing group membership This is an error from ipareplica-install log: 2012-05-03T14:54:05Z DEBUG args=/usr/bin/ldapmodify -h vm-098.idm.lab.bos.redhat.com -v -f /tmp/ tmpGFqASL -x -D cn=Directory Manager -y /tmp/tmpBuxVf4 2012-05-03T14:54:05Z DEBUG stdout= 2012-05-03T14:54:05Z DEBUG stderr=ldap_initialize( ldap://vm-098.idm.lab.bos.redhat.com ) ldapmodify: wrong attributeType at line 5, entry "cn=ipa-http-delegation,cn=s4u2proxy,cn=etc,dc=idm, dc=lab,dc=bos,dc=redhat,dc=com" 4) When I run ipa-adtrust-install on the replica, I received the same error as in 1) 5) Removal of cifs S4U2Proxy records does not work because the removal code does not specify the right service name (s/ldap/cifs): dn3 = DN(u'cn=ipa-cifs-delegation-targets', api.env.container_s4u2proxy, self.suffix) member_principal3 = "ldap/%(fqdn)s@%(realm)s" % dict(fqdn=replica, realm=realm) 6) I miss some help or examples in trust help: # ipa help trust Manage trust relationship between realms Topic commands: But I suppose it can be added as an enhancement later. This is all for now, I don't have an environment to test the trusts itselves. But fixing these basic issues should be sufficient for us to be able to at least push this work to master. Martin From nkinder at redhat.com Thu May 3 15:31:17 2012 From: nkinder at redhat.com (Nathan Kinder) Date: Thu, 03 May 2012 08:31:17 -0700 Subject: [Freeipa-devel] [PATCH] 0042-0048 AD trusts support (master) In-Reply-To: <1336058328.20409.13.camel@balmora.brq.redhat.com> References: <20120403104135.GD23171@redhat.com> <20120403145749.GD8996@localhost.localdomain> <1334242615.777.3.camel@balmora.brq.redhat.com> <20120412150803.GA24623@redhat.com> <1334243807.777.6.camel@balmora.brq.redhat.com> <1334903980.2985.6.camel@balmora.brq.redhat.com> <1335446337.20837.20.camel@balmora.brq.redhat.com> <1336058328.20409.13.camel@balmora.brq.redhat.com> Message-ID: <4FA2A4C5.4030908@redhat.com> On 05/03/2012 08:18 AM, Martin Kosek wrote: > On Thu, 2012-04-26 at 15:18 +0200, Martin Kosek wrote: >> On Fri, 2012-04-20 at 08:39 +0200, Martin Kosek wrote: >>> On Thu, 2012-04-12 at 17:16 +0200, Martin Kosek wrote: >>>> On Thu, 2012-04-12 at 18:08 +0300, Alexander Bokovoy wrote: >>>>> Hi Martin! >>>>> >>>>> On Thu, 12 Apr 2012, Martin Kosek wrote: >>>> ... >>>>>> 3) I would not try to import ipaserver.dcerpc every time the command is >>>>>> executed: >>>>>> + try: >>>>>> + import ipaserver.dcerpc >>>>>> + except Exception, e: >>>>>> + raise errors.NotFound(name=_('AD Trust setup'), >>>>>> + reason=_('Cannot perform join operation without Samba >>>>>> 4 python bindings installed')) >>>>>> >>>>>> I would rather do it once in the beginning and set a flag: >>>>>> >>>>>> try: >>>>>> import ipaserver.dcerpc >>>>>> _bindings_installed = True >>>>>> except Exception: >>>>>> _bindings_installed = False >>>>>> >>>>>> ... >>>>> The idea was that this code is only executed on the server. We need to >>>>> differentiate between: >>>>> - running on client >>>>> - running on server, no samba4 python bindings >>>>> - running on server with samba4 python bindings >>>>> >>>>> By making it executed all time you are affecting the client code as >>>>> well while with current approach it only affects server side. >>>> Across our code base, this situation is currently solved with this >>>> condition: >>>> >>>> if api.env.in_server and api.env.context in ['lite', 'server']: >>>> # try-import block >>>> >>>>> >>>>>> + def execute(self, *keys, **options): >>>>>> + # Join domain using full credentials and with random trustdom >>>>>> + # secret (will be generated by the join method) >>>>>> + trustinstance = None >>>>>> + if not _bindings_installed: >>>>>> + raise errors.NotFound(name=_('AD Trust setup'), >>>>>> + reason=_('Cannot perform join operation without Samba >>>>>> 4 python bindings installed')) >>>>>> >>>>>> >>>>>> 4) Another import inside a function: >>>>>> + def arcfour_encrypt(key, data): >>>>>> + from Crypto.Cipher import ARC4 >>>>>> + c = ARC4.new(key) >>>>>> + return c.encrypt(data) >>>>> Same here, it is only needed on server side. >>>>> >>>>> Let us get consensus over 3) and 4) and I'll fix patches altogether (and >>>>> push). >>>>> >>>> Yeah, I would fix in the same way as 3). >>>> >>> I am running another run of test to finish my review of your patches, >>> but I stumbled in 389-ds error when I was installing IPA server from >>> package built from your git tree: >>> git://fedorapeople.org/home/fedora/abbra/public_git/freeipa.git >>> >>> # rpm -q freeipa-server 389-ds-base >>> freeipa-server-2.99.0GITc30f375-0.fc17.x86_64 >>> 389-ds-base-1.2.11-0.1.a1.fc17.x86_64 >>> # ipa-server-install -p kokos123 -a kokos123 >>> ... >>> [16/18]: issuing RA agent certificate >>> [17/18]: adding RA agent as a trusted user >>> [18/18]: Configure HTTP to proxy connections >>> done configuring pki-cad. >>> Configuring directory server: Estimated time 1 minute >>> [1/35]: creating directory server user >>> [2/35]: creating directory server instance >>> [3/35]: adding default schema >>> [4/35]: enabling memberof plugin >>> [5/35]: enabling referential integrity plugin >>> [6/35]: enabling winsync plugin >>> [7/35]: configuring replication version plugin >>> [8/35]: enabling IPA enrollment plugin >>> [9/35]: enabling ldapi >>> [10/35]: configuring uniqueness plugin >>> [11/35]: configuring uuid plugin >>> [12/35]: configuring modrdn plugin >>> [13/35]: enabling entryUSN plugin >>> [14/35]: configuring lockout plugin >>> [15/35]: creating indices >>> [16/35]: configuring ssl for ds instance >>> [17/35]: configuring certmap.conf >>> [18/35]: configure autobind for root >>> [19/35]: configure new location for managed entries >>> [20/35]: restarting directory server >>> [21/35]: adding default layout >>> [22/35]: adding delegation layout >>> ipa : CRITICAL Failed to load delegation.ldif: Command >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v >>> -f /tmp/tmpdXcWF3 -x -D cn=Directory Manager -y /tmp/tmp8qtnOS' returned >>> non-zero exit status 255 >>> [23/35]: adding replication acis >>> ipa : CRITICAL Failed to load replica-acis.ldif: Command >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v >>> -f /tmp/tmptivfJ_ -x -D cn=Directory Manager -y /tmp/tmpr_Z1lp' returned >>> non-zero exit status 255 >>> [24/35]: creating container for managed entries >>> ipa : CRITICAL Failed to load managed-entries.ldif: Command >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v >>> -f /tmp/tmpNkmoDk -x -D cn=Directory Manager -y /tmp/tmpXU0lbx' returned >>> non-zero exit status 255 >>> [25/35]: configuring user private groups >>> ipa : CRITICAL Failed to load user_private_groups.ldif: Command >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v >>> -f /tmp/tmp7uDqaG -x -D cn=Directory Manager -y /tmp/tmp6E_uPl' returned >>> non-zero exit status 255 >>> [26/35]: configuring netgroups from hostgroups >>> ipa : CRITICAL Failed to load host_nis_groups.ldif: Command >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v >>> -f /tmp/tmphxoVQf -x -D cn=Directory Manager -y /tmp/tmpsAhhwd' returned >>> non-zero exit status 255 >>> [27/35]: creating default Sudo bind user >>> ipa : CRITICAL Failed to load sudobind.ldif: Command >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v >>> -f /tmp/tmpCVpYqT -x -D cn=Directory Manager -y /tmp/tmp97b_6d' returned >>> non-zero exit status 255 >>> [28/35]: creating default Auto Member layout >>> ipa : CRITICAL Failed to load automember.ldif: Command >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v >>> -f /tmp/tmpvcFbwK -x -D cn=Directory Manager -y /tmp/tmpSUownE' returned >>> non-zero exit status 255 >>> [29/35]: creating default HBAC rule allow_all >>> ipa : CRITICAL Failed to load default-hbac.ldif: Command >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v >>> -f /tmp/tmpYoYkBy -x -D cn=Directory Manager -y /tmp/tmp_9le4C' returned >>> non-zero exit status 255 >>> [30/35]: initializing group membership >>> ipa : CRITICAL Failed to load memberof-task.ldif: Command >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v >>> -f /tmp/tmpD9mIxC -x -D cn=Directory Manager -y /tmp/tmpeTqozO' returned >>> non-zero exit status 255 >>> Unexpected error - see ipaserver-install.log for details: >>> {'desc': "Can't contact LDAP server"} >>> >>> >>> # tail /var/log/dirsrv/slapd-IDM-LAB-BOS-REDHAT-COM/errors >>> [20/Apr/2012:02:19:16 -0400] - 389-Directory/1.2.11.a1 B2012.090.2135 >>> starting up >>> [20/Apr/2012:02:19:16 -0400] attrcrypt - No symmetric key found for >>> cipher AES in backend userRoot, attempting to create one... >>> [20/Apr/2012:02:19:16 -0400] attrcrypt - Key for cipher AES successfully >>> generated and stored >>> [20/Apr/2012:02:19:16 -0400] attrcrypt - No symmetric key found for >>> cipher 3DES in backend userRoot, attempting to create one... >>> [20/Apr/2012:02:19:16 -0400] attrcrypt - Key for cipher 3DES >>> successfully generated and stored >>> [20/Apr/2012:02:19:16 -0400] - slapd started. Listening on All >>> Interfaces port 389 for LDAP requests >>> [20/Apr/2012:02:19:16 -0400] - Listening on All Interfaces port 636 for >>> LDAPS requests >>> [20/Apr/2012:02:19:16 -0400] - Listening >>> on /var/run/slapd-IDM-LAB-BOS-REDHAT-COM.socket for LDAPI requests >>> [20/Apr/2012:02:19:17 -0400] - Skipping CoS Definition cn=Password >>> Policy,cn=accounts,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com--no CoS >>> Templates found, which should be added before the CoS Definition. >>> [20/Apr/2012:02:19:17 -0400] entryrdn-index - _entryrdn_put_data: Adding >>> the self link (62) failed: BDB0068 DB_LOCK_DEADLOCK: Locker killed to >>> resolve a deadlock (-30993) >>> >>> Martin >>> >> I reproduced this issue even on another clean VM, I filed a BZ for that: >> https://bugzilla.redhat.com/show_bug.cgi?id=816590 >> >> Martin >> > With the development version of the fix for DS issue, I was able to > continue with the review. I found the following issues: Please start using 389-ds-base-1.2.11.1-1.fc17, which is in testing now. Karma would be much appreciated. > > 1) You add cifs s4u2proxy record twice. This leads to an error message > during ipa-adtrust-install: > > # ipa-server-install --setup-dns > # ipa-adtrust-install > ... > [6/13]: setting password for the samba user > [7/13]: adding cifs Kerberos principal > ipa : CRITICAL Failed to add key for > cifs/vm-109.idm.lab.bos.redhat.com at IDM.LAB.BOS.REDHAT.COM > [8/13]: adding admin(group) SIDs > [9/13]: activating CLDAP plugin > ... > > > 2) Typo in ipa-adtrust-install info text: > > Additionally you have to make sure the FreeIPA LDAP server cannot reached > by any domain controller in the Active Directory domain by closing the > > s/reached/cannot be reached/ > > 3) Another s4u2proxy error in ipa-replica-install: > > # ipa-replica-install INFO_FILE > ... > [20/30]: restarting directory server > [21/30]: setting up initial replication > Starting replication, please wait until this has completed. > Update in progress > Update in progress > Update in progress > Update succeeded > [22/30]: adding replication acis > [23/30]: setting Auto Member configuration > [24/30]: enabling S4U2Proxy delegation > ipa : CRITICAL Failed to load replica-s4u2proxy.ldif: Command > '/usr/bin/ldapmodify -h vm-098.idm.lab.bos.redhat.com -v > -f /tmp/tmpGFqASL -x -D cn=Directory Manager -y /tmp/tmpBuxVf4' returned > non-zero exit status 247 > [25/30]: initializing group membership > > This is an error from ipareplica-install log: > > 2012-05-03T14:54:05Z DEBUG args=/usr/bin/ldapmodify -h > vm-098.idm.lab.bos.redhat.com -v -f /tmp/ tmpGFqASL -x -D > cn=Directory Manager -y /tmp/tmpBuxVf4 > 2012-05-03T14:54:05Z DEBUG stdout= > 2012-05-03T14:54:05Z DEBUG > stderr=ldap_initialize( ldap://vm-098.idm.lab.bos.redhat.com ) > ldapmodify: wrong attributeType at line 5, entry > "cn=ipa-http-delegation,cn=s4u2proxy,cn=etc,dc=idm, > dc=lab,dc=bos,dc=redhat,dc=com" > > 4) When I run ipa-adtrust-install on the replica, I received the same > error as in 1) > > 5) Removal of cifs S4U2Proxy records does not work because the removal > code does not specify the right service name (s/ldap/cifs): > > dn3 = DN(u'cn=ipa-cifs-delegation-targets', api.env.container_s4u2proxy, self.suffix) > member_principal3 = "ldap/%(fqdn)s@%(realm)s" % dict(fqdn=replica, realm=realm) > > 6) I miss some help or examples in trust help: > > # ipa help trust > Manage trust relationship between realms > > Topic commands: > > But I suppose it can be added as an enhancement later. > > This is all for now, I don't have an environment to test the trusts > itselves. But fixing these basic issues should be sufficient for us to > be able to at least push this work to master. > > Martin > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From mkosek at redhat.com Thu May 3 15:44:38 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 03 May 2012 17:44:38 +0200 Subject: [Freeipa-devel] [PATCH] 23 Allow one letter net/hostgroups names In-Reply-To: <4FA29F5B.5060501@redhat.com> References: <4FA15794.9020501@redhat.com> <4FA29F5B.5060501@redhat.com> Message-ID: <1336059878.20409.19.camel@balmora.brq.redhat.com> On Thu, 2012-05-03 at 17:08 +0200, Ondrej Hamada wrote: > On 05/02/2012 05:49 PM, Ondrej Hamada wrote: > > https://fedorahosted.org/freeipa/ticket/2671 > > > > Changed regex validating net/hostgroup names to allow single letter > > names. Unit-tests added. > > > > But the current validation allows weird (host|net)group names like: > > ".", ".-", "..". > > I'm just not sure, do we really want to allow stuff like this? > > > > Patch also fixes one of netgroup and host unit-tests. The error > > message in hostname validation function has changed (in ticket > > #1966). > > > > NACK. 1) This breaks the hostgroup tests as you overwrite dn1 variable: +hostgroup_single = u'a' +dn1 = DN(('cn',hostgroup_single),('cn','hostgroups'),('cn','accounts'), + api.env.basedn) + 2) The extra comment in netgroup tests is redundant: + result=dict( +# dn=u'ipauniqueid=%s,cn=ng,cn=alt,%s' % (fuzzy_uuid, api.env.basedn), + dn=fuzzy_netgroupdn, 3) I don't think that we need to bump IPA_API_VERSION_MINOR since we just changed the validating pattern and thus this really does not change the API itself. Martin From pviktori at redhat.com Thu May 3 16:25:29 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 03 May 2012 18:25:29 +0200 Subject: [Freeipa-devel] [PATCH] 0046 Don't fail when adding default objectclasses using config-mod Message-ID: <4FA2B179.4050108@redhat.com> Fix another setattr internal error that QA found. https://fedorahosted.org/freeipa/ticket/2706 -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0046-Don-t-fail-when-adding-default-objectclasses-using-c.patch Type: text/x-patch Size: 3784 bytes Desc: not available URL: From ohamada at redhat.com Thu May 3 17:37:37 2012 From: ohamada at redhat.com (Ondrej Hamada) Date: Thu, 03 May 2012 19:37:37 +0200 Subject: [Freeipa-devel] More types of replica in FreeIPA In-Reply-To: <4F966899.9040901@redhat.com> References: <4F4E41FC.6020606@redhat.com> <4F7B2937.4060300@redhat.com> <1333544545.22628.287.camel@willson.li.ssimo.org> <4F8CA7B0.2080000@redhat.com> <1334666521.16658.171.camel@willson.li.ssimo.org> <4F8F084C.5090106@redhat.com> <4F900287.8080606@redhat.com> <1334840631.16658.325.camel@willson.li.ssimo.org> <4F901CD1.5050703@redhat.com> <1334853233.16658.345.camel@willson.li.ssimo.org> <4F9060D0.3010204@redhat.com> <1334864682.16658.358.camel@willson.li.ssimo.org> <4F907599.6060609@redhat.com> <1334870909.16658.437.camel@willson.li.ssimo.org> <4F9090F1.1000005@redhat.com> <1334879014.16658.499.camel@willson.li.ssimo.org> <4F91C27D.6070507@redhat.com> <1334953792.16658.526.camel@willson.li.ssimo.org> <4F955F11.1000003@redhat.com> <1335189758.16658.611.camel@willson.li.ssimo.org> <4F958B98.2020702@redhat.com> <1335202525.16658.640.camel@willson.li.ssimo.org> <4F959681.7090501@redhat.com> <1335203925.16658.647.camel@willson.li.ssimo.org> <4F966899.9040901@redhat.com> Message-ID: <4FA2C261.9040005@redhat.com> On 04/24/2012 10:47 AM, Ondrej Hamada wrote: > On 04/23/2012 07:58 PM, Simo Sorce wrote: >> On Mon, 2012-04-23 at 13:50 -0400, Dmitri Pal wrote: >>> Ah OK. Another semantic difference. Doing it in phases is one thing and >>> delivering is another. Let us say we identified 10 things that needs to >>> be implemented. The problem is so huge that Ondrej would likely be able >>> to tackle only couple items from the list. So what should be do with >>> the >>> rest if it is not possible to deliver until all 10 items are completed? >> Ok, so most of the work here is in the KDC, so I think we should first >> go to MIT, present the problem and see what htey think about the >> solution we have in mind. I will try to have a preliminary discussion >> With Tom and Greg about the general idea this week to see what they >> think. >> >> Once that is done we can slice the implementation how we want in a >> private branch until it is fully backed. MIT wouldn't, rightly so, >> accept a half backed solution I would guess, but we also do not need to >> try to rush patches in. Once cleanup work in the KDC has been done as >> part of the 1.11 work I think these interfaces will change little so >> there shouldn't be a risk of wasting too much time to follow upstream >> while we work on one of these problems at a time. >> >>> IMO the work can be started and deferred till someone else can come >>> back >>> and continue what Ondrej have started and bring it to the shape when we >>> are comfortable releasing it. >> Absolutely, esp if we can start after he changes MIT plans to make in >> 1.11 or at least if we plan together so we know which internal >> interfaces are going to be destabilized so we can plan ahead. >> >>> Ondra it time for you to sit down, read this thread thoroughly and >>> craft >>> a design out of it. Then you would be able to focus on a reasonable >>> subset of what is possible to complete in the remaining time frame. > Ok, will do. I would like to start with the login server scenario. It > will be possible to use it later as a 'training field' for the > fractional replication and help deciding what entries should and > shouldn't be replicated. >> Ok. >> Simo. >> > > As I said before, I'm going to start with "authentication only" server. That will be the first iteration. (I also want to present it in my thesis as the implementation part) Both the Hub and Consumer will be read only. In case of Hub the machine should contain only directory server that will be configured to behave as a hub. Consumers should behave same way as Dmitri described few posts above - means they will use ldap with pam-proxy to sssd. The sssd will be authenticating the user against master server. It might use caching to enable some user to authenticate when the master is unreachable. The consumer should be using chaining and trying to contact the master directly. Replicas will replicate all data, just the confidential attributes such as passwords will be excluded from replication. Main enhancements will be made in ipa-tools, mainly the ipa-replica-install and ipa-replica-manage. Also the ipa-client-install will be updated as the client in such environment won't use Kerberos. I think that at this stage those changes should be stored separately - I mean not pushing them into upstream. Can you agree on that? The second iteration should be focusing on development of plugins for handling the account locking situation and similiar situations that need to write some data to the replica. It might also focus on fractional replication if it will be available in directory server. I suppose that there won't be any more iterations necessary for the authentication server. Besides working on the second iteration we can also start with the eSSO part. I assume that the account locks and fractional replication will definitely have something in common. -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada From simo at redhat.com Thu May 3 18:00:36 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 03 May 2012 14:00:36 -0400 Subject: [Freeipa-devel] More types of replica in FreeIPA In-Reply-To: <4FA2C261.9040005@redhat.com> References: <4F4E41FC.6020606@redhat.com> <4F7B2937.4060300@redhat.com> <1333544545.22628.287.camel@willson.li.ssimo.org> <4F8CA7B0.2080000@redhat.com> <1334666521.16658.171.camel@willson.li.ssimo.org> <4F8F084C.5090106@redhat.com> <4F900287.8080606@redhat.com> <1334840631.16658.325.camel@willson.li.ssimo.org> <4F901CD1.5050703@redhat.com> <1334853233.16658.345.camel@willson.li.ssimo.org> <4F9060D0.3010204@redhat.com> <1334864682.16658.358.camel@willson.li.ssimo.org> <4F907599.6060609@redhat.com> <1334870909.16658.437.camel@willson.li.ssimo.org> <4F9090F1.1000005@redhat.com> <1334879014.16658.499.camel@willson.li.ssimo.org> <4F91C27D.6070507@redhat.com> <1334953792.16658.526.camel@willson.li.ssimo.org> <4F955F11.1000003@redhat.com> <1335189758.16658.611.camel@willson.li.ssimo.org> <4F958B98.2020702@redhat.com> <1335202525.16658.640.camel@willson.li.ssimo.org> <4F959681.7090501@redhat.com> <1335203925.16658.647.camel@willson.li.ssimo.org> <4F966899.9040901@redhat.com> <4FA2C261.9040005@redhat.com> Message-ID: <1336068036.5722.132.camel@willson.li.ssimo.org> On Thu, 2012-05-03 at 19:37 +0200, Ondrej Hamada wrote: > On 04/24/2012 10:47 AM, Ondrej Hamada wrote: > > On 04/23/2012 07:58 PM, Simo Sorce wrote: > >> On Mon, 2012-04-23 at 13:50 -0400, Dmitri Pal wrote: > >>> Ah OK. Another semantic difference. Doing it in phases is one thing and > >>> delivering is another. Let us say we identified 10 things that needs to > >>> be implemented. The problem is so huge that Ondrej would likely be able > >>> to tackle only couple items from the list. So what should be do with > >>> the > >>> rest if it is not possible to deliver until all 10 items are completed? > >> Ok, so most of the work here is in the KDC, so I think we should first > >> go to MIT, present the problem and see what htey think about the > >> solution we have in mind. I will try to have a preliminary discussion > >> With Tom and Greg about the general idea this week to see what they > >> think. > >> > >> Once that is done we can slice the implementation how we want in a > >> private branch until it is fully backed. MIT wouldn't, rightly so, > >> accept a half backed solution I would guess, but we also do not need to > >> try to rush patches in. Once cleanup work in the KDC has been done as > >> part of the 1.11 work I think these interfaces will change little so > >> there shouldn't be a risk of wasting too much time to follow upstream > >> while we work on one of these problems at a time. > >> > >>> IMO the work can be started and deferred till someone else can come > >>> back > >>> and continue what Ondrej have started and bring it to the shape when we > >>> are comfortable releasing it. > >> Absolutely, esp if we can start after he changes MIT plans to make in > >> 1.11 or at least if we plan together so we know which internal > >> interfaces are going to be destabilized so we can plan ahead. > >> > >>> Ondra it time for you to sit down, read this thread thoroughly and > >>> craft > >>> a design out of it. Then you would be able to focus on a reasonable > >>> subset of what is possible to complete in the remaining time frame. > > Ok, will do. I would like to start with the login server scenario. It > > will be possible to use it later as a 'training field' for the > > fractional replication and help deciding what entries should and > > shouldn't be replicated. > >> Ok. > >> Simo. > >> > > > > > As I said before, I'm going to start with "authentication only" server. > That will be the first iteration. (I also want to present it in my > thesis as the implementation part) > > Both the Hub and Consumer will be read only. In case of Hub the machine > should contain only directory server that will be configured to behave > as a hub. Consumers should behave same way as Dmitri described few posts > above - means they will use ldap with pam-proxy to sssd. The sssd will > be authenticating the user against master server. It might use caching > to enable some user to authenticate when the master is unreachable. The > consumer should be using chaining and trying to contact the master > directly. No this is useless, just let it replicate userPassword, so that you do not need to set up this complicated pam-proxy+sssd. It really is not worth it. > Replicas will replicate all data, just the confidential attributes such > as passwords will be excluded from replication. exclude only kerberos keys, let userPassword replicate for now, later we weill find how to replicate it only for a subset of users. > Main enhancements will be made in ipa-tools, mainly the > ipa-replica-install and ipa-replica-manage. Also the ipa-client-install > will be updated as the client in such environment won't use Kerberos. I > think that at this stage those changes should be stored separately - I > mean not pushing them into upstream. > > Can you agree on that? Agree on not pushing some of it. Cleanups are always welcome though. > The second iteration should be focusing on development of plugins for > handling the account locking situation and similiar situations that need > to write some data to the replica. It might also focus on fractional > replication if it will be available in directory server. I suppose that > there won't be any more iterations necessary for the authentication server. Hopefully not. > Besides working on the second iteration we can also start with the eSSO > part. I assume that the account locks and fractional replication will > definitely have something in common. Probably not, but the main part with SSO is managing the krb changes. Simo. -- Simo Sorce * Red Hat, Inc * New York From michael.tiernan at gmail.com Thu May 3 18:16:00 2012 From: michael.tiernan at gmail.com (Michael C Tiernan) Date: Thu, 3 May 2012 14:16:00 -0400 (EDT) Subject: [Freeipa-devel] Minor error in announcement. In-Reply-To: <23470452.1118.1336068814950.JavaMail.mtiernan@MCTMBP.local> Message-ID: <2690901.1124.1336068957388.JavaMail.mtiernan@MCTMBP.local> Yes I'm new. I thought I'd point out, the document: [Freeipa-interest] Announcing FreeIPA v2.1.90 beta 1 Release Date: Mon, 05 Mar 2012 17:28:12 -0500 The FreeIPA team is proud to announce version 2.1.90 beta 1. This will eventually become FreeIPA v2.2.0. Has in it the link: It can be downloaded from http://www.freeipa.org/Downloads Which is incorrect and produces an error. (I believe you want "downloads") -- << MCT >> Michael C Tiernan. Is God a performance artist? http://www.linkedin.com/in/mtiernan From rcritten at redhat.com Thu May 3 19:49:16 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 03 May 2012 15:49:16 -0400 Subject: [Freeipa-devel] Announcing FreeIPA v2.2.0 Release Message-ID: <4FA2E13C.9040006@redhat.com> The FreeIPA team is proud to announce version FreeIPA v2.2.0. It can be downloaded from http://www.freeipa.org/Downloads. A build is on the way to updates-testing for Fedora 17. Fedora 15 and 16 are not supported by FreeIPA 2.2.0 due to missing dependencies. == Highlights in 2.2.0 == * Forms-based login. If Kerberos Single-Sign-On authentication fails, you now have the option to authenticate through a form-base login page using your domain username and password. You an also go directly to the page named /ipa/ui/login.html to do form-based authentication without attempting a Kerberos login at all * Logout from the UI * Support for SSH known-hosts with sssd 1.8.0. This will create a known-hosts file dynamically based on information stored in IPA. * SELinux user maps to control a user's SELinux context depending on what host they log into (requires sssd 1.8.0+). * Support for global configuration of the name server stored in LDAP, including a list of global forwarders, forward policy, DNS zone refresh poll timeout. * Enhanced per-zone configuration, including query and transfer policy, and conditional forwarding. * DNS record CLI and Web UI is vastly improved, including an improved validation of supported DNS record types, an ability to create compound DNS records (like LOC or SRV) by its parts. * Migration improvements including being able to specify the basedn, translation of stored DN values. User-Private groups are no longer being created for migrated users. * We recommend that the compat plugin be disabled during migration to avoid unnecessary overhead. * On new installations the default users group, ipausers, is now non-POSIX to speed up user enumeration in SSSD. To make ipausers a POSIX group run ipa group-mod --posix ipausers. * The WebUI now has support for HBAC testing and Automember mananagement. == Upgrading == An IPA server can be upgraded simply by installing updated rpms. The server does not need to be shut down in advance. If you have multiple servers you may upgrade them one at a time. It is expected that all servers will be upgraded in a relatively short period (days or weeks not months). They should be able to co-exist peacefully but new features will not be available on old servers and enrolling a new client against an old server will result in the SSH keys not being uploaded. Downgrading a server once upgraded is not supported. Upgrading from 2.1.90 rc1 has not been tested. An enrolled client does not need the new packages installed unless you want to re-enroll it. SSH keys for already installed clients are not uploaded, you will have to re-enroll the client or manually upload the keys. == Feedback == Please provide comments, bugs and other feedback via the freeipa-devel mailing list: http://www.redhat.com/mailman/listinfo/freeipa-devel == Detailed Changelog since 2.1.90 rc 1 == Alexander Bokovoy (1): * When changing multiple booleans with setsebool, pass each of them separately. Jan Cholasta (9): * Wait for child process to terminate after receiving SIGINT in ipautil.run. * Parse zone indices in IPv6 addresses in CheckedIPAddress. * Fix uses of O=REALM instead of the configured certificate subject base. * Fix the procedure for getting default values of command parameters. * Change parameters to use only default_from for dynamic default values. * Check whether the default user group is POSIX when adding new user with --noprivate. * Check configured maximum user login length on user rename. * Fix internal error when renaming user with an empty string. * Set the "KerberosAuthentication" option in sshd_config to "no" instead of "yes". John Dennis (7): * Replace broken i18n shell test with Python test * improve handling of ds instances during uninstall * Use indexed format specifiers in i18n strings * text unit test should validate using installed mo file * Validate DN & RDN parameters for migrate command * don't append basedn to container if it is included * Fix name error in hbactest Lars Sjostrom (1): * Add disovery domain if client domain is different from server domain Martin Kosek (29): * Ignore case in yes/no prompts * Refresh resolvers after DNS install * Fix migration plugin compat check * Fix ipa-replica-manage TLS connection error * Treat UPGs correctly in winsync replication * Allow port numbers for idnsForwarders * Add missing global options in dnsconfig * Fix precallback validators in DNS plugin * Harden raw record processing in DNS plugin * Fix LDAP effective rights control with python-ldap 2.4.x * Avoid deleting DNS zone when a context is reused * Fix default SOA serial format * Amend permissions for new DNS attributes * Improve user awareness about dnsconfig * Fix dnsrecord-del interactive mode * Tolerate UDP port failures in conncheck * Improve automount indirect map error message * Forbid public access to DNS tree * Configure SELinux for httpd during upgrades * Fix installation when server hostname is not in a default domain * Return correct record name in DNS plugin * Fix dnsrecord_add interactive mode * Fix DNS and permissions unit tests * Raise proper exception when LDAP limits are exceeded * Do not fail migration because of duplicate groups * Fix help of --hostname option in ipa-client-install * Sort password policies properly with --pkey-only * Improve error message in zonemgr validator * Make ipa 2.2 client capable of joining an older server Ondrej Hamada (7): * More exception handlers in ipa-client-install * Search allowed attributes in superior objectclasses * Typos in FreeIPA messages * Netgroup nisdomain and hosts validation * Confusing default user groups * Unable to rename permission object * Fix empty external member processing Petr Viktorin (22): * Allow removing sudo commands with special characters from command groups * Enforce that required attributes can't be set to None in CRUD Update * Mark most config options as required * Don't crash when searching with empty relationship options * Remove ipausers' gidnumber from tests * Use nose tools to check for exceptions * Only split CSV in the client, quote instead of escaping * Add missing BuildRequires * Use valid argument names in tests * Add CLI parsing tests * Allow multi-line CSV parameters * Move test skipping to class setup * Fix little test errors * Test the batch plugin * Defer conversion and validation until after --{add,del,set}attr are handled * Limit permission and selfservice names to alphanumerics, -, _, space * Convert --setattr values for attributes marked no_update * Fix expected error messages in tests * Remove pattern_errmsg from API.txt * Pass make-test arguments through to Nose * Document the 'nonempty' flag * Additional tests for pwpolicy Petr Vobornik (22): * Fixed mask validation in network_validator * Fixed checkbox value in table without pkey * Certificate serial number in hex format - ui testing data * Fixed evaluating checkbox dirty status * Better hbactest validation message * Content is no more overwritten by error message * Show_content on refresh success * Fixed rpm build warning - extension.js listed twice * Add support of new options in dnsconfig * DNS forwarder validator * Added mac address to host page * Facet expiration flag * Inter-facet expiration * Reworked netgroup Web UI to allow setting user/host category * Fixed: permission attrs table didn't update its available options on load * Added attrs field to permission for target=subtree * DNS forward policy: checkboxes changed to radio buttons * Removed mutex option from checkboxes * Removal of memberofindirect_permissons from privileges * User is notified that password needs to be reset in forms-based login * Added permission field to delegation * Paging disable for password policies Rob Crittenden (34): * Fix NSS no_init in the NSSHTTPS class * Set minimum version of selinux-policy to pick up memcached fix * Fix nsslapd-anonlimitsdn dn in cn=config * Set SELinux boolean httpd_manage_ipa so ipa_memcached will work. * Don't set dbdir in the connection until after the connection is created. * Display serial number as HEX (DECIMAL) when showing certificates. * Add subject key identifier to the dogtag server cert profile. * Configure a basic ldap.conf for OpenLDAP in /etc/openldap/ldap.conf * Import the ipaserver plugins based on context, not env.in_server. * Don't allow hosts and services of IPA masters to be disabled. * Use a consistent parameter name in errors, defaulting to cli_name. * No longer shell escape the DM password when calling pkisilent. * Fix test failure testing rename with an invalid hostname. * Fix attributes that contain DNs when migrating. * Normalize the primary key value to lowercase during migration. * Fix unit tests to work with new comma-support, validation requirements * Set minimum version of 389-ds-base to 1.2.10.4-2 to fix upgrade issue * Set nsslapd-minssf-exclude-rootdse to on so the DSE is always available. * Add requires on python-krbV to client subpackage * Fix failure count interval attribute name in query for password policy. * Handle updating replication agreements that lack nsDS5ReplicatedAttributeList * Don't create private groups for migrated users, check for valid gidnumber * Add updated Output format for batch to API.txt * Make revocation_reason required when revoking a certificate. * Add missing comma to list of services that cannot be disabled. * Return consistent value when hostcat and usercat is all. * Dereference pointer when comparing password history in qsort compare. * Configure certmonger to execute restart scripts on renewal. * Remove the running state when uninstalling DS instances. * Return consistent expiration message for forms-based login * Use mixed-case for Read DNS Entries permission * Update docs for user-status, always show disabled, time for each server. Simo Sorce (1): * Fix memleak and silence Coverity defects From dpal at redhat.com Thu May 3 19:57:06 2012 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 03 May 2012 15:57:06 -0400 Subject: [Freeipa-devel] Announcing FreeIPA v2.2.0 Release In-Reply-To: <4FA2E13C.9040006@redhat.com> References: <4FA2E13C.9040006@redhat.com> Message-ID: <4FA2E312.7040201@redhat.com> On 05/03/2012 03:49 PM, Rob Crittenden wrote: > The FreeIPA team is proud to announce version FreeIPA v2.2.0. > > It can be downloaded from http://www.freeipa.org/Downloads. > It can be downloaded from http://www.freeipa.org/downloads. > A build is on the way to updates-testing for Fedora 17. Fedora 15 and > 16 are not supported by FreeIPA 2.2.0 due to missing dependencies. > > == Highlights in 2.2.0 == > > * Forms-based login. If Kerberos Single-Sign-On authentication fails, > you now have the option to authenticate through a form-base login page > using your domain username and password. You an also go directly to > the page named /ipa/ui/login.html to do form-based authentication > without attempting a Kerberos login at all > * Logout from the UI > * Support for SSH known-hosts with sssd 1.8.0. This will create a > known-hosts file dynamically based on information stored in IPA. > * SELinux user maps to control a user's SELinux context depending on > what host they log into (requires sssd 1.8.0+). > * Support for global configuration of the name server stored in LDAP, > including a list of global forwarders, forward policy, DNS zone > refresh poll timeout. > * Enhanced per-zone configuration, including query and transfer > policy, and conditional forwarding. > * DNS record CLI and Web UI is vastly improved, including an improved > validation of supported DNS record types, an ability to create > compound DNS records (like LOC or SRV) by its parts. > * Migration improvements including being able to specify the basedn, > translation of stored DN values. User-Private groups are no longer > being created for migrated users. > * We recommend that the compat plugin be disabled during migration to > avoid unnecessary overhead. > * On new installations the default users group, ipausers, is now > non-POSIX to speed up user enumeration in SSSD. To make ipausers a > POSIX group run ipa group-mod --posix ipausers. > * The WebUI now has support for HBAC testing and Automember > mananagement. > > == Upgrading == > > An IPA server can be upgraded simply by installing updated rpms. The > server does not need to be shut down in advance. > > If you have multiple servers you may upgrade them one at a time. It is > expected that all servers will be upgraded in a relatively short > period (days or weeks not months). They should be able to co-exist > peacefully but new features will not be available on old servers and > enrolling a new client against an old server will result in the SSH > keys not being uploaded. > > Downgrading a server once upgraded is not supported. > > Upgrading from 2.1.90 rc1 has not been tested. > > An enrolled client does not need the new packages installed unless you > want to re-enroll it. SSH keys for already installed clients are not > uploaded, you will have to re-enroll the client or manually upload the > keys. > > == Feedback == > > Please provide comments, bugs and other feedback via the freeipa-devel > mailing list: http://www.redhat.com/mailman/listinfo/freeipa-devel > > == Detailed Changelog since 2.1.90 rc 1 == > > Alexander Bokovoy (1): > * When changing multiple booleans with setsebool, pass each of them > separately. > > Jan Cholasta (9): > * Wait for child process to terminate after receiving SIGINT in > ipautil.run. > * Parse zone indices in IPv6 addresses in CheckedIPAddress. > * Fix uses of O=REALM instead of the configured certificate subject > base. > * Fix the procedure for getting default values of command parameters. > * Change parameters to use only default_from for dynamic default values. > * Check whether the default user group is POSIX when adding new user > with --noprivate. > * Check configured maximum user login length on user rename. > * Fix internal error when renaming user with an empty string. > * Set the "KerberosAuthentication" option in sshd_config to "no" > instead of "yes". > > John Dennis (7): > * Replace broken i18n shell test with Python test > * improve handling of ds instances during uninstall > * Use indexed format specifiers in i18n strings > * text unit test should validate using installed mo file > * Validate DN & RDN parameters for migrate command > * don't append basedn to container if it is included > * Fix name error in hbactest > > Lars Sjostrom (1): > * Add disovery domain if client domain is different from server domain > > Martin Kosek (29): > * Ignore case in yes/no prompts > * Refresh resolvers after DNS install > * Fix migration plugin compat check > * Fix ipa-replica-manage TLS connection error > * Treat UPGs correctly in winsync replication > * Allow port numbers for idnsForwarders > * Add missing global options in dnsconfig > * Fix precallback validators in DNS plugin > * Harden raw record processing in DNS plugin > * Fix LDAP effective rights control with python-ldap 2.4.x > * Avoid deleting DNS zone when a context is reused > * Fix default SOA serial format > * Amend permissions for new DNS attributes > * Improve user awareness about dnsconfig > * Fix dnsrecord-del interactive mode > * Tolerate UDP port failures in conncheck > * Improve automount indirect map error message > * Forbid public access to DNS tree > * Configure SELinux for httpd during upgrades > * Fix installation when server hostname is not in a default domain > * Return correct record name in DNS plugin > * Fix dnsrecord_add interactive mode > * Fix DNS and permissions unit tests > * Raise proper exception when LDAP limits are exceeded > * Do not fail migration because of duplicate groups > * Fix help of --hostname option in ipa-client-install > * Sort password policies properly with --pkey-only > * Improve error message in zonemgr validator > * Make ipa 2.2 client capable of joining an older server > > Ondrej Hamada (7): > * More exception handlers in ipa-client-install > * Search allowed attributes in superior objectclasses > * Typos in FreeIPA messages > * Netgroup nisdomain and hosts validation > * Confusing default user groups > * Unable to rename permission object > * Fix empty external member processing > > Petr Viktorin (22): > * Allow removing sudo commands with special characters from command > groups > * Enforce that required attributes can't be set to None in CRUD Update > * Mark most config options as required > * Don't crash when searching with empty relationship options > * Remove ipausers' gidnumber from tests > * Use nose tools to check for exceptions > * Only split CSV in the client, quote instead of escaping > * Add missing BuildRequires > * Use valid argument names in tests > * Add CLI parsing tests > * Allow multi-line CSV parameters > * Move test skipping to class setup > * Fix little test errors > * Test the batch plugin > * Defer conversion and validation until after --{add,del,set}attr are > handled > * Limit permission and selfservice names to alphanumerics, -, _, space > * Convert --setattr values for attributes marked no_update > * Fix expected error messages in tests > * Remove pattern_errmsg from API.txt > * Pass make-test arguments through to Nose > * Document the 'nonempty' flag > * Additional tests for pwpolicy > > Petr Vobornik (22): > * Fixed mask validation in network_validator > * Fixed checkbox value in table without pkey > * Certificate serial number in hex format - ui testing data > * Fixed evaluating checkbox dirty status > * Better hbactest validation message > * Content is no more overwritten by error message > * Show_content on refresh success > * Fixed rpm build warning - extension.js listed twice > * Add support of new options in dnsconfig > * DNS forwarder validator > * Added mac address to host page > * Facet expiration flag > * Inter-facet expiration > * Reworked netgroup Web UI to allow setting user/host category > * Fixed: permission attrs table didn't update its available options > on load > * Added attrs field to permission for target=subtree > * DNS forward policy: checkboxes changed to radio buttons > * Removed mutex option from checkboxes > * Removal of memberofindirect_permissons from privileges > * User is notified that password needs to be reset in forms-based login > * Added permission field to delegation > * Paging disable for password policies > > Rob Crittenden (34): > * Fix NSS no_init in the NSSHTTPS class > * Set minimum version of selinux-policy to pick up memcached fix > * Fix nsslapd-anonlimitsdn dn in cn=config > * Set SELinux boolean httpd_manage_ipa so ipa_memcached will work. > * Don't set dbdir in the connection until after the connection is > created. > * Display serial number as HEX (DECIMAL) when showing certificates. > * Add subject key identifier to the dogtag server cert profile. > * Configure a basic ldap.conf for OpenLDAP in /etc/openldap/ldap.conf > * Import the ipaserver plugins based on context, not env.in_server. > * Don't allow hosts and services of IPA masters to be disabled. > * Use a consistent parameter name in errors, defaulting to cli_name. > * No longer shell escape the DM password when calling pkisilent. > * Fix test failure testing rename with an invalid hostname. > * Fix attributes that contain DNs when migrating. > * Normalize the primary key value to lowercase during migration. > * Fix unit tests to work with new comma-support, validation requirements > * Set minimum version of 389-ds-base to 1.2.10.4-2 to fix upgrade issue > * Set nsslapd-minssf-exclude-rootdse to on so the DSE is always > available. > * Add requires on python-krbV to client subpackage > * Fix failure count interval attribute name in query for password > policy. > * Handle updating replication agreements that lack > nsDS5ReplicatedAttributeList > * Don't create private groups for migrated users, check for valid > gidnumber > * Add updated Output format for batch to API.txt > * Make revocation_reason required when revoking a certificate. > * Add missing comma to list of services that cannot be disabled. > * Return consistent value when hostcat and usercat is all. > * Dereference pointer when comparing password history in qsort compare. > * Configure certmonger to execute restart scripts on renewal. > * Remove the running state when uninstalling DS instances. > * Return consistent expiration message for forms-based login > * Use mixed-case for Read DNS Entries permission > * Update docs for user-status, always show disabled, time for each > server. > > Simo Sorce (1): > * Fix memleak and silence Coverity defects > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Thu May 3 20:02:08 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 03 May 2012 16:02:08 -0400 Subject: [Freeipa-devel] Minor error in announcement. In-Reply-To: <2690901.1124.1336068957388.JavaMail.mtiernan@MCTMBP.local> References: <2690901.1124.1336068957388.JavaMail.mtiernan@MCTMBP.local> Message-ID: <4FA2E440.1000702@redhat.com> Michael C Tiernan wrote: > Yes I'm new. > I thought I'd point out, the document: > [Freeipa-interest] Announcing FreeIPA v2.1.90 beta 1 Release > Date: Mon, 05 Mar 2012 17:28:12 -0500 > > The FreeIPA team is proud to announce version 2.1.90 beta 1. This will eventually become FreeIPA v2.2.0. > > Has in it the link: > It can be downloaded from http://www.freeipa.org/Downloads > > Which is incorrect and produces an error. (I believe you want "downloads") It really should be http://freeipa.org/page/Downloads I just goofed on the last announcement too. +1 for consistency anyway. rob From mkosek at redhat.com Fri May 4 06:41:58 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 04 May 2012 08:41:58 +0200 Subject: [Freeipa-devel] [PATCH] 0042-0048 AD trusts support (master) In-Reply-To: <4FA2A4C5.4030908@redhat.com> References: <20120403104135.GD23171@redhat.com> <20120403145749.GD8996@localhost.localdomain> <1334242615.777.3.camel@balmora.brq.redhat.com> <20120412150803.GA24623@redhat.com> <1334243807.777.6.camel@balmora.brq.redhat.com> <1334903980.2985.6.camel@balmora.brq.redhat.com> <1335446337.20837.20.camel@balmora.brq.redhat.com> <1336058328.20409.13.camel@balmora.brq.redhat.com> <4FA2A4C5.4030908@redhat.com> Message-ID: <1336113718.28933.1.camel@balmora.brq.redhat.com> On Thu, 2012-05-03 at 08:31 -0700, Nathan Kinder wrote: > On 05/03/2012 08:18 AM, Martin Kosek wrote: > > On Thu, 2012-04-26 at 15:18 +0200, Martin Kosek wrote: > >> On Fri, 2012-04-20 at 08:39 +0200, Martin Kosek wrote: > >>> On Thu, 2012-04-12 at 17:16 +0200, Martin Kosek wrote: > >>>> On Thu, 2012-04-12 at 18:08 +0300, Alexander Bokovoy wrote: > >>>>> Hi Martin! > >>>>> > >>>>> On Thu, 12 Apr 2012, Martin Kosek wrote: > >>>> ... > >>>>>> 3) I would not try to import ipaserver.dcerpc every time the command is > >>>>>> executed: > >>>>>> + try: > >>>>>> + import ipaserver.dcerpc > >>>>>> + except Exception, e: > >>>>>> + raise errors.NotFound(name=_('AD Trust setup'), > >>>>>> + reason=_('Cannot perform join operation without Samba > >>>>>> 4 python bindings installed')) > >>>>>> > >>>>>> I would rather do it once in the beginning and set a flag: > >>>>>> > >>>>>> try: > >>>>>> import ipaserver.dcerpc > >>>>>> _bindings_installed = True > >>>>>> except Exception: > >>>>>> _bindings_installed = False > >>>>>> > >>>>>> ... > >>>>> The idea was that this code is only executed on the server. We need to > >>>>> differentiate between: > >>>>> - running on client > >>>>> - running on server, no samba4 python bindings > >>>>> - running on server with samba4 python bindings > >>>>> > >>>>> By making it executed all time you are affecting the client code as > >>>>> well while with current approach it only affects server side. > >>>> Across our code base, this situation is currently solved with this > >>>> condition: > >>>> > >>>> if api.env.in_server and api.env.context in ['lite', 'server']: > >>>> # try-import block > >>>> > >>>>> > >>>>>> + def execute(self, *keys, **options): > >>>>>> + # Join domain using full credentials and with random trustdom > >>>>>> + # secret (will be generated by the join method) > >>>>>> + trustinstance = None > >>>>>> + if not _bindings_installed: > >>>>>> + raise errors.NotFound(name=_('AD Trust setup'), > >>>>>> + reason=_('Cannot perform join operation without Samba > >>>>>> 4 python bindings installed')) > >>>>>> > >>>>>> > >>>>>> 4) Another import inside a function: > >>>>>> + def arcfour_encrypt(key, data): > >>>>>> + from Crypto.Cipher import ARC4 > >>>>>> + c = ARC4.new(key) > >>>>>> + return c.encrypt(data) > >>>>> Same here, it is only needed on server side. > >>>>> > >>>>> Let us get consensus over 3) and 4) and I'll fix patches altogether (and > >>>>> push). > >>>>> > >>>> Yeah, I would fix in the same way as 3). > >>>> > >>> I am running another run of test to finish my review of your patches, > >>> but I stumbled in 389-ds error when I was installing IPA server from > >>> package built from your git tree: > >>> git://fedorapeople.org/home/fedora/abbra/public_git/freeipa.git > >>> > >>> # rpm -q freeipa-server 389-ds-base > >>> freeipa-server-2.99.0GITc30f375-0.fc17.x86_64 > >>> 389-ds-base-1.2.11-0.1.a1.fc17.x86_64 > >>> # ipa-server-install -p kokos123 -a kokos123 > >>> ... > >>> [16/18]: issuing RA agent certificate > >>> [17/18]: adding RA agent as a trusted user > >>> [18/18]: Configure HTTP to proxy connections > >>> done configuring pki-cad. > >>> Configuring directory server: Estimated time 1 minute > >>> [1/35]: creating directory server user > >>> [2/35]: creating directory server instance > >>> [3/35]: adding default schema > >>> [4/35]: enabling memberof plugin > >>> [5/35]: enabling referential integrity plugin > >>> [6/35]: enabling winsync plugin > >>> [7/35]: configuring replication version plugin > >>> [8/35]: enabling IPA enrollment plugin > >>> [9/35]: enabling ldapi > >>> [10/35]: configuring uniqueness plugin > >>> [11/35]: configuring uuid plugin > >>> [12/35]: configuring modrdn plugin > >>> [13/35]: enabling entryUSN plugin > >>> [14/35]: configuring lockout plugin > >>> [15/35]: creating indices > >>> [16/35]: configuring ssl for ds instance > >>> [17/35]: configuring certmap.conf > >>> [18/35]: configure autobind for root > >>> [19/35]: configure new location for managed entries > >>> [20/35]: restarting directory server > >>> [21/35]: adding default layout > >>> [22/35]: adding delegation layout > >>> ipa : CRITICAL Failed to load delegation.ldif: Command > >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > >>> -f /tmp/tmpdXcWF3 -x -D cn=Directory Manager -y /tmp/tmp8qtnOS' returned > >>> non-zero exit status 255 > >>> [23/35]: adding replication acis > >>> ipa : CRITICAL Failed to load replica-acis.ldif: Command > >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > >>> -f /tmp/tmptivfJ_ -x -D cn=Directory Manager -y /tmp/tmpr_Z1lp' returned > >>> non-zero exit status 255 > >>> [24/35]: creating container for managed entries > >>> ipa : CRITICAL Failed to load managed-entries.ldif: Command > >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > >>> -f /tmp/tmpNkmoDk -x -D cn=Directory Manager -y /tmp/tmpXU0lbx' returned > >>> non-zero exit status 255 > >>> [25/35]: configuring user private groups > >>> ipa : CRITICAL Failed to load user_private_groups.ldif: Command > >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > >>> -f /tmp/tmp7uDqaG -x -D cn=Directory Manager -y /tmp/tmp6E_uPl' returned > >>> non-zero exit status 255 > >>> [26/35]: configuring netgroups from hostgroups > >>> ipa : CRITICAL Failed to load host_nis_groups.ldif: Command > >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > >>> -f /tmp/tmphxoVQf -x -D cn=Directory Manager -y /tmp/tmpsAhhwd' returned > >>> non-zero exit status 255 > >>> [27/35]: creating default Sudo bind user > >>> ipa : CRITICAL Failed to load sudobind.ldif: Command > >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > >>> -f /tmp/tmpCVpYqT -x -D cn=Directory Manager -y /tmp/tmp97b_6d' returned > >>> non-zero exit status 255 > >>> [28/35]: creating default Auto Member layout > >>> ipa : CRITICAL Failed to load automember.ldif: Command > >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > >>> -f /tmp/tmpvcFbwK -x -D cn=Directory Manager -y /tmp/tmpSUownE' returned > >>> non-zero exit status 255 > >>> [29/35]: creating default HBAC rule allow_all > >>> ipa : CRITICAL Failed to load default-hbac.ldif: Command > >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > >>> -f /tmp/tmpYoYkBy -x -D cn=Directory Manager -y /tmp/tmp_9le4C' returned > >>> non-zero exit status 255 > >>> [30/35]: initializing group membership > >>> ipa : CRITICAL Failed to load memberof-task.ldif: Command > >>> '/usr/bin/ldapmodify -h vm-079.idm.lab.bos.redhat.com -v > >>> -f /tmp/tmpD9mIxC -x -D cn=Directory Manager -y /tmp/tmpeTqozO' returned > >>> non-zero exit status 255 > >>> Unexpected error - see ipaserver-install.log for details: > >>> {'desc': "Can't contact LDAP server"} > >>> > >>> > >>> # tail /var/log/dirsrv/slapd-IDM-LAB-BOS-REDHAT-COM/errors > >>> [20/Apr/2012:02:19:16 -0400] - 389-Directory/1.2.11.a1 B2012.090.2135 > >>> starting up > >>> [20/Apr/2012:02:19:16 -0400] attrcrypt - No symmetric key found for > >>> cipher AES in backend userRoot, attempting to create one... > >>> [20/Apr/2012:02:19:16 -0400] attrcrypt - Key for cipher AES successfully > >>> generated and stored > >>> [20/Apr/2012:02:19:16 -0400] attrcrypt - No symmetric key found for > >>> cipher 3DES in backend userRoot, attempting to create one... > >>> [20/Apr/2012:02:19:16 -0400] attrcrypt - Key for cipher 3DES > >>> successfully generated and stored > >>> [20/Apr/2012:02:19:16 -0400] - slapd started. Listening on All > >>> Interfaces port 389 for LDAP requests > >>> [20/Apr/2012:02:19:16 -0400] - Listening on All Interfaces port 636 for > >>> LDAPS requests > >>> [20/Apr/2012:02:19:16 -0400] - Listening > >>> on /var/run/slapd-IDM-LAB-BOS-REDHAT-COM.socket for LDAPI requests > >>> [20/Apr/2012:02:19:17 -0400] - Skipping CoS Definition cn=Password > >>> Policy,cn=accounts,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com--no CoS > >>> Templates found, which should be added before the CoS Definition. > >>> [20/Apr/2012:02:19:17 -0400] entryrdn-index - _entryrdn_put_data: Adding > >>> the self link (62) failed: BDB0068 DB_LOCK_DEADLOCK: Locker killed to > >>> resolve a deadlock (-30993) > >>> > >>> Martin > >>> > >> I reproduced this issue even on another clean VM, I filed a BZ for that: > >> https://bugzilla.redhat.com/show_bug.cgi?id=816590 > >> > >> Martin > >> > > With the development version of the fix for DS issue, I was able to > > continue with the review. I found the following issues: > Please start using 389-ds-base-1.2.11.1-1.fc17, which is in testing > now. Karma would be much appreciated. Will do! I just tested it and it works so far - karma+1 from me. Martin From ohamada at redhat.com Fri May 4 08:37:10 2012 From: ohamada at redhat.com (Ondrej Hamada) Date: Fri, 04 May 2012 10:37:10 +0200 Subject: [Freeipa-devel] [PATCH] 23 Allow one letter net/hostgroups names In-Reply-To: <1336059878.20409.19.camel@balmora.brq.redhat.com> References: <4FA15794.9020501@redhat.com> <4FA29F5B.5060501@redhat.com> <1336059878.20409.19.camel@balmora.brq.redhat.com> Message-ID: <4FA39536.8040207@redhat.com> On 05/03/2012 05:44 PM, Martin Kosek wrote: > On Thu, 2012-05-03 at 17:08 +0200, Ondrej Hamada wrote: >> On 05/02/2012 05:49 PM, Ondrej Hamada wrote: >>> https://fedorahosted.org/freeipa/ticket/2671 >>> >>> Changed regex validating net/hostgroup names to allow single letter >>> names. Unit-tests added. >>> >>> But the current validation allows weird (host|net)group names like: >>> ".", ".-", "..". >>> I'm just not sure, do we really want to allow stuff like this? >>> >>> Patch also fixes one of netgroup and host unit-tests. The error >>> message in hostname validation function has changed (in ticket >>> #1966). >>> >>> > NACK. > > 1) This breaks the hostgroup tests as you overwrite dn1 variable: > > +hostgroup_single = u'a' > +dn1 = DN(('cn',hostgroup_single),('cn','hostgroups'),('cn','accounts'), > + api.env.basedn) > + > > > 2) The extra comment in netgroup tests is redundant: > > + result=dict( > +# dn=u'ipauniqueid=%s,cn=ng,cn=alt,%s' % > (fuzzy_uuid, api.env.basedn), > + dn=fuzzy_netgroupdn, > > 3) I don't think that we need to bump IPA_API_VERSION_MINOR since we > just changed the validating pattern and thus this really does not change > the API itself. > > Martin > corrected patch attached -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ohamada-23-3-Allow-one-letter-net-hostgroups-names.patch Type: text/x-patch Size: 17002 bytes Desc: not available URL: From ohamada at redhat.com Fri May 4 11:25:16 2012 From: ohamada at redhat.com (Ondrej Hamada) Date: Fri, 04 May 2012 13:25:16 +0200 Subject: [Freeipa-devel] [PATCH] 0044 Validate externalhost (when added by --addattr/--setattr) In-Reply-To: <4F9E81E7.3030005@redhat.com> References: <4F9E81E7.3030005@redhat.com> Message-ID: <4FA3BC9C.2010309@redhat.com> On 04/30/2012 02:13 PM, Petr Viktorin wrote: > > Change the externalhost attribute of hbacrule, netgroup > and sudorule into a full-fledged Parameter, and attach > a validator to it. > > RFC 1123 specifies that only [-a-z0-9] are allowed, but apparently > Windows and some phones also use underscores in hostnames. > So the new validator allows the underscore. > > > https://fedorahosted.org/freeipa/ticket/2649 > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel 1) Current validation of external hostnames does not require them to be fully qualified, but you do. It's inconsistent. 2) one test case failed: FAIL: Test adding an invalid external host to Sudo rule using ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ohamada/2649/tests/test_xmlrpc/test_sudorule_plugin.py", line 500, in test_a_sudorule_mod_externalhost_invalid_addattr "character") AssertionError -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- An HTML attachment was scrubbed... URL: From pvoborni at redhat.com Fri May 4 12:18:55 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 04 May 2012 14:18:55 +0200 Subject: [Freeipa-devel] [PATCH] 135 Host page fixed to work with disabled DNS support Message-ID: <4FA3C92F.7070206@redhat.com> When DNS support was disabled there were following errors in Web UI: 1) Host details page was not filled with data 2) Host adder dialog was broken -> unusable 3) DNS tab was displayed in navigation The bugs were fixed by: 1) Was caused by entity_link_widget. The widget was modified to do not show link if other_entity (in this case dnsrecord) is not present. 2) Was caused by host_fqdn_widget. The widget is unusable because without DNS support it doesn't have access to DNS zone entity. The section with this widget was removed. Also IP address field was removed because it shouln't be used without DNS support. New 'fqdn' text box was added for specifying hostname. 3) New DNS config entity was initialized but it wasn't shown because it caused some JavaScript error. The dnsconfig's init method was modified to throw expected exception. Now no dns entity is initialized and therefore DNS tab in navigation is not displayed. https://fedorahosted.org/freeipa/ticket/2728 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0135-Host-page-fixed-to-work-with-disabled-DNS-support.patch Type: text/x-patch Size: 3470 bytes Desc: not available URL: From mkosek at redhat.com Fri May 4 15:45:44 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 04 May 2012 17:45:44 +0200 Subject: [Freeipa-devel] [PATCH] 257 Fix python Requires in Fedora 17 build Message-ID: <1336146344.2581.18.camel@balmora.brq.redhat.com> This one actually took me some time to track it down (details are in a patch description). To check the result, simply build freeipa on Fedora 17 with "make rpms", install rpms on the machine and check Requires of freeipa-admintools package: $ rpm -qR freeipa-admintools Before the patch, there was a requirement for "/bin/python" which effectively blocked an update of python package until freeipa packages were removed. With this patch, there should be a correct requirement for "/usr/bin/python" and python updates will work again - yay. Our newest freeipa package on F-17 (2.2.0-1) is not affected, koji F-17 build root may have a different $PATH which translates "python" to "/usr/bin/python" and not "/bin/python". Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-257-fix-python-requires-in-fedora-17-build.patch Type: text/x-patch Size: 2103 bytes Desc: not available URL: From rmeggins at redhat.com Sat May 5 14:46:56 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Sat, 05 May 2012 08:46:56 -0600 Subject: [Freeipa-devel] 389-ds-base-1.2.11.3 has been pushed to testing Message-ID: <4FA53D60.3070505@redhat.com> This is the F-17 candidate. It fixes the deadlock and the managed entry deletion issues found by Rob. Please give it some karma before the F-17 deadline on Monday. Thanks! From mkosek at redhat.com Mon May 7 06:40:55 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 07 May 2012 08:40:55 +0200 Subject: [Freeipa-devel] [PATCH] 23 Allow one letter net/hostgroups names In-Reply-To: <4FA39536.8040207@redhat.com> References: <4FA15794.9020501@redhat.com> <4FA29F5B.5060501@redhat.com> <1336059878.20409.19.camel@balmora.brq.redhat.com> <4FA39536.8040207@redhat.com> Message-ID: <1336372855.29911.1.camel@balmora.brq.redhat.com> On Fri, 2012-05-04 at 10:37 +0200, Ondrej Hamada wrote: > On 05/03/2012 05:44 PM, Martin Kosek wrote: > > On Thu, 2012-05-03 at 17:08 +0200, Ondrej Hamada wrote: > >> On 05/02/2012 05:49 PM, Ondrej Hamada wrote: > >>> https://fedorahosted.org/freeipa/ticket/2671 > >>> > >>> Changed regex validating net/hostgroup names to allow single letter > >>> names. Unit-tests added. > >>> > >>> But the current validation allows weird (host|net)group names like: > >>> ".", ".-", "..". > >>> I'm just not sure, do we really want to allow stuff like this? > >>> > >>> Patch also fixes one of netgroup and host unit-tests. The error > >>> message in hostname validation function has changed (in ticket > >>> #1966). > >>> > >>> > > NACK. > > > > 1) This breaks the hostgroup tests as you overwrite dn1 variable: > > > > +hostgroup_single = u'a' > > +dn1 = DN(('cn',hostgroup_single),('cn','hostgroups'),('cn','accounts'), > > + api.env.basedn) > > + > > > > > > 2) The extra comment in netgroup tests is redundant: > > > > + result=dict( > > +# dn=u'ipauniqueid=%s,cn=ng,cn=alt,%s' % > > (fuzzy_uuid, api.env.basedn), > > + dn=fuzzy_netgroupdn, > > > > 3) I don't think that we need to bump IPA_API_VERSION_MINOR since we > > just changed the validating pattern and thus this really does not change > > the API itself. > > > > Martin > > > corrected patch attached > ACK. Pushed to master. Martin From mkosek at redhat.com Mon May 7 07:12:18 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 07 May 2012 09:12:18 +0200 Subject: [Freeipa-devel] 389-ds-base-1.2.11.3 has been pushed to testing In-Reply-To: <4FA53D60.3070505@redhat.com> References: <4FA53D60.3070505@redhat.com> Message-ID: <1336374738.29911.4.camel@balmora.brq.redhat.com> On Sat, 2012-05-05 at 08:46 -0600, Rich Megginson wrote: > This is the F-17 candidate. It fixes the deadlock and the managed entry > deletion issues found by Rob. Please give it some karma before the F-17 > deadline on Monday. Thanks! > In my VM, this version causes FreeIPA installation error. I filed a BZ with all logs attached: https://bugzilla.redhat.com/show_bug.cgi?id=819409 Martin From pspacek at redhat.com Mon May 7 10:35:56 2012 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 07 May 2012 12:35:56 +0200 Subject: [Freeipa-devel] [PATCH 0018] Deadlock detection logic In-Reply-To: <20120503121842.GA14952@redhat.com> References: <4F96A8E4.7010002@redhat.com> <4F96B000.204@redhat.com> <20120503121842.GA14952@redhat.com> Message-ID: <4FA7A58C.80403@redhat.com> On 05/03/2012 02:18 PM, Adam Tkac wrote: > On Tue, Apr 24, 2012 at 03:52:00PM +0200, Petr Spacek wrote: >> On 04/24/2012 03:21 PM, Petr Spacek wrote: >>> Hello, >>> >>> this patch adds deadlock detection (based on simple timeout) to current code. >>> If (probable) deadlock is detected, current action is stopped with proper error. >>> >>> It properly detects "Simo's deadlock" with 'connections' parameter == 1. >>> (Described in https://fedorahosted.org/bind-dyndb-ldap/ticket/66) >>> >>> Deadlock itself will be fixed by separate patch. >>> >>> Petr^2 Spacek >> >> Self-NACK :-) >> >> Second version of the patch is attached. ldap_pool_getconnection() >> and ldap_pool_putconnection() now has same interface and more >> consistent behaviour. >> >> Overall functionality is same. > > Hi, > > although I'm not fully happy with current design of the detection logic, we can > include it before we create something better, for example based on thread IDs > (one thread can acquire semaphore only one time). I agree, it's far from ideal. Connection and result handling needs redesign at first. After that I can modify detection logic to be more accurate. > > Please check my comments inside the patch. All done. Petr^2 Spacek > > Regards, Adam > >> From ed7596a9410269c0c29418247971f262b7fa77f3 Mon Sep 17 00:00:00 2001 >> From: Petr Spacek >> Date: Tue, 24 Apr 2012 15:09:32 +0200 >> Subject: [PATCH] Add simple semaphore deadlock detection logic. >> Signed-off-by: Petr Spacek >> >> --- >> src/ldap_helper.c | 78 ++++++++++++++++++++++++++++++++--------------------- >> src/semaphore.c | 26 +++++++++++++++--- >> src/semaphore.h | 6 +++- >> 3 files changed, 74 insertions(+), 36 deletions(-) >> >> diff --git a/src/ldap_helper.c b/src/ldap_helper.c >> index 47c0559..f4c4d02 100644 >> --- a/src/ldap_helper.c >> +++ b/src/ldap_helper.c >> @@ -296,9 +296,10 @@ ldap_delete_zone2(ldap_instance_t *inst, dns_name_t *name, isc_boolean_t lock); >> static isc_result_t ldap_pool_create(isc_mem_t *mctx, unsigned int connections, >> ldap_pool_t **poolp); >> static void ldap_pool_destroy(ldap_pool_t **poolp); >> -static ldap_connection_t * ldap_pool_getconnection(ldap_pool_t *pool); >> +static isc_result_t ldap_pool_getconnection(ldap_pool_t *pool, >> + ldap_connection_t ** conn); >> static void ldap_pool_putconnection(ldap_pool_t *pool, >> - ldap_connection_t *ldap_conn); >> + ldap_connection_t ** conn); >> static isc_result_t ldap_pool_connect(ldap_pool_t *pool, >> ldap_instance_t *ldap_inst); >> >> @@ -402,6 +403,10 @@ new_ldap_instance(isc_mem_t *mctx, const char *db_name, >> ldap_settings[i++].target =&ldap_inst->dyn_update; >> CHECK(set_settings(ldap_settings, argv)); >> >> + /* Set timer for deadlock detection inside semaphore_wait_timed . */ >> + if (semaphore_wait_timeout.seconds< ldap_inst->timeout+SEM_WAIT_TIMEOUT_ADD) >> + semaphore_wait_timeout.seconds = ldap_inst->timeout+SEM_WAIT_TIMEOUT_ADD; > > I'm affraid such low timeout (12 sec by default) will cause too many false > positives. I recommend to set semaphore timeout to at least 60 seconds. > >> /* Validate and check settings. */ >> str_toupper(ldap_inst->sasl_mech); >> if (ldap_inst->connections< 1) { >> @@ -1089,7 +1094,7 @@ isc_result_t >> refresh_zones_from_ldap(ldap_instance_t *ldap_inst) >> { >> isc_result_t result = ISC_R_SUCCESS; >> - ldap_connection_t *ldap_conn; >> + ldap_connection_t *ldap_conn = NULL; >> int zone_count = 0; >> ldap_entry_t *entry; >> dns_rbt_t *rbt = NULL; >> @@ -1114,7 +1119,7 @@ refresh_zones_from_ldap(ldap_instance_t *ldap_inst) >> >> log_debug(2, "refreshing list of zones for %s", ldap_inst->db_name); >> >> - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); >> + CHECK(ldap_pool_getconnection(ldap_inst->pool,&ldap_conn)); >> >> CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(ldap_inst->base), >> LDAP_SCOPE_SUBTREE, config_attrs, 0, >> @@ -1227,7 +1232,7 @@ cleanup: >> if (invalidate_nodechain) >> dns_rbtnodechain_invalidate(&chain); >> >> - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); >> + ldap_pool_putconnection(ldap_inst->pool,&ldap_conn); >> >> log_debug(2, "finished refreshing list of zones"); >> >> @@ -1381,7 +1386,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam >> dns_name_t *origin, ldapdb_nodelist_t *nodelist) >> { >> isc_result_t result; >> - ldap_connection_t *ldap_conn; >> + ldap_connection_t *ldap_conn = NULL; >> ldap_entry_t *entry; >> ld_string_t *string = NULL; >> ldapdb_node_t *node; >> @@ -1392,7 +1397,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam >> REQUIRE(nodelist != NULL); >> >> /* RRs aren't in the cache, perform ordinary LDAP query */ >> - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); >> + CHECK(ldap_pool_getconnection(ldap_inst->pool,&ldap_conn)); >> >> INIT_LIST(*nodelist); >> CHECK(str_new(mctx,&string)); >> @@ -1439,7 +1444,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam >> result = ISC_R_SUCCESS; >> >> cleanup: >> - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); >> + ldap_pool_putconnection(ldap_inst->pool,&ldap_conn); >> str_destroy(&string); >> >> return result; >> @@ -1450,7 +1455,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na >> dns_name_t *origin, ldapdb_rdatalist_t *rdatalist) >> { >> isc_result_t result; >> - ldap_connection_t *ldap_conn; >> + ldap_connection_t *ldap_conn = NULL; >> ldap_entry_t *entry; >> ld_string_t *string = NULL; >> ldap_cache_t *cache; >> @@ -1468,12 +1473,11 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na >> return result; >> >> /* RRs aren't in the cache, perform ordinary LDAP query */ >> - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); >> - >> INIT_LIST(*rdatalist); >> CHECK(str_new(mctx,&string)); >> CHECK(dnsname_to_dn(ldap_inst->zone_register, name, string)); >> >> + CHECK(ldap_pool_getconnection(ldap_inst->pool,&ldap_conn)); >> CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(string), >> LDAP_SCOPE_BASE, NULL, 0, "(objectClass=idnsRecord)")); >> >> @@ -1500,7 +1504,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na >> result = ISC_R_NOTFOUND; >> >> cleanup: >> - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); >> + ldap_pool_putconnection(ldap_inst->pool,&ldap_conn); >> str_destroy(&string); >> >> if (result != ISC_R_SUCCESS) >> @@ -2250,7 +2254,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, >> zone_dn += 1; /* skip whitespace */ >> } >> >> - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); >> + CHECK(ldap_pool_getconnection(ldap_inst->pool,&ldap_conn)); >> CHECK(ldap_query(ldap_inst, ldap_conn, zone_dn, >> LDAP_SCOPE_BASE, attrs, 0, >> "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); >> @@ -2481,9 +2485,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, >> } >> >> cleanup: >> - if (ldap_conn != NULL) >> - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); >> - >> + ldap_pool_putconnection(ldap_inst->pool,&ldap_conn); >> str_destroy(&owner_dn_ptr); >> str_destroy(&owner_dn); >> free_ldapmod(mctx,&change[0]); >> @@ -2565,15 +2567,18 @@ ldap_pool_destroy(ldap_pool_t **poolp) >> MEM_PUT_AND_DETACH(pool); >> } >> >> -static ldap_connection_t * >> -ldap_pool_getconnection(ldap_pool_t *pool) >> +static isc_result_t >> +ldap_pool_getconnection(ldap_pool_t *pool, ldap_connection_t ** conn) >> { >> ldap_connection_t *ldap_conn = NULL; >> unsigned int i; >> + isc_result_t result; >> >> REQUIRE(pool != NULL); >> + REQUIRE(conn != NULL&& *conn == NULL); >> + ldap_conn = *conn; >> >> - semaphore_wait(&pool->conn_semaphore); >> + CHECK(semaphore_wait_timed(&pool->conn_semaphore)); >> for (i = 0; i< pool->connections; i++) { >> ldap_conn = pool->conns[i]; >> if (isc_mutex_trylock(&ldap_conn->lock) == ISC_R_SUCCESS) >> @@ -2583,16 +2588,30 @@ ldap_pool_getconnection(ldap_pool_t *pool) >> RUNTIME_CHECK(ldap_conn != NULL); >> >> INIT_LIST(ldap_conn->ldap_entries); >> + *conn = ldap_conn; >> >> - return ldap_conn; >> +cleanup: >> + if (result != ISC_R_SUCCESS) { >> + log_error("timeout in ldap_pool_getconnection(): try to raise " >> + "'connections' parameter; potential deadlock?"); >> + } >> + return result; >> } >> >> static void >> -ldap_pool_putconnection(ldap_pool_t *pool, ldap_connection_t *ldap_conn) >> +ldap_pool_putconnection(ldap_pool_t *pool, ldap_connection_t **conn) >> { >> + REQUIRE(conn); > > Although this is correct, please use REQUIRE(conn != NULL); > >> + ldap_connection_t *ldap_conn = *conn; >> + >> + if (ldap_conn == NULL) >> + return; >> + >> ldap_query_free(ldap_conn); >> UNLOCK(&ldap_conn->lock); >> semaphore_signal(&pool->conn_semaphore); >> + >> + *conn = NULL; >> } >> >> static isc_result_t >> @@ -2719,7 +2738,7 @@ update_action(isc_task_t *task, isc_event_t *event) >> ldap_psearchevent_t *pevent = (ldap_psearchevent_t *)event; >> isc_result_t result ; >> ldap_instance_t *inst = NULL; >> - ldap_connection_t *conn; >> + ldap_connection_t *conn = NULL; >> ldap_entry_t *entry; >> isc_boolean_t delete = ISC_TRUE; >> isc_mem_t *mctx; >> @@ -2736,7 +2755,7 @@ update_action(isc_task_t *task, isc_event_t *event) >> if (result != ISC_R_SUCCESS) >> goto cleanup; >> >> - conn = ldap_pool_getconnection(inst->pool); >> + CHECK(ldap_pool_getconnection(inst->pool,&conn)); >> >> CHECK(ldap_query(inst, conn, pevent->dn, >> LDAP_SCOPE_BASE, attrs, 0, >> @@ -2754,14 +2773,13 @@ update_action(isc_task_t *task, isc_event_t *event) >> if (delete) >> CHECK(ldap_delete_zone(inst, pevent->dn, ISC_TRUE)); >> >> - ldap_pool_putconnection(inst->pool, conn); >> - >> cleanup: >> if (result != ISC_R_SUCCESS) >> log_error("update_action (psearch) failed for %s. " >> "Zones can be outdated, run `rndc reload`", >> pevent->dn); >> >> + ldap_pool_putconnection(inst->pool,&conn); >> isc_mem_free(mctx, pevent->dbname); >> isc_mem_free(mctx, pevent->dn); >> isc_mem_detach(&mctx); >> @@ -2790,7 +2808,7 @@ update_config(isc_task_t *task, isc_event_t *event) >> if (result != ISC_R_SUCCESS) >> goto cleanup; >> >> - conn = ldap_pool_getconnection(inst->pool); >> + CHECK(ldap_pool_getconnection(inst->pool,&conn)); >> >> CHECK(ldap_query(inst, conn, pevent->dn, >> LDAP_SCOPE_BASE, attrs, 0, >> @@ -2809,14 +2827,12 @@ update_config(isc_task_t *task, isc_event_t *event) >> >> >> cleanup: >> - if (conn != NULL) >> - ldap_pool_putconnection(inst->pool, conn); >> - >> if (result != ISC_R_SUCCESS) >> log_error("update_config (psearch) failed for %s. " >> "Configuration can be outdated, run `rndc reload`", >> pevent->dn); >> >> + ldap_pool_putconnection(inst->pool,&conn); >> isc_mem_free(mctx, pevent->dbname); >> isc_mem_free(mctx, pevent->dn); >> isc_mem_detach(&mctx); >> @@ -3079,7 +3095,7 @@ ldap_psearch_watcher(isc_threadarg_t arg) >> tv.tv_usec = 0; >> >> /* Pick connection, one is reserved purely for this thread */ >> - conn = ldap_pool_getconnection(inst->pool); >> + CHECK(ldap_pool_getconnection(inst->pool,&conn)); >> >> /* Try to connect. */ >> while (conn->handle == NULL) { >> @@ -3189,7 +3205,7 @@ soft_err: >> log_debug(1, "Ending ldap_psearch_watcher"); >> >> cleanup: >> - ldap_pool_putconnection(inst->pool, conn); >> + ldap_pool_putconnection(inst->pool,&conn); >> >> return (isc_threadresult_t)0; >> } >> diff --git a/src/semaphore.c b/src/semaphore.c >> index 41d6a30..76e6aa2 100644 >> --- a/src/semaphore.c >> +++ b/src/semaphore.c >> @@ -27,8 +27,19 @@ >> #include >> #include >> #include >> +#include >> >> #include "semaphore.h" >> +#include "util.h" >> + >> +/* >> + * Timer setting for deadlock detection. Format: seconds, nanoseconds. >> + * These values will be overwriten during initialization >> + * from set_settings() with max(setting+SEM_WAIT_TIMEOUT_ADD, curr_value). >> + * >> + * Initial value can be useful in early phases of initialization. >> + */ >> +isc_interval_t semaphore_wait_timeout = { 3, 0 }; >> >> /* >> * Initialize a semaphore. >> @@ -74,20 +85,27 @@ semaphore_destroy(semaphore_t *sem) >> /* >> * Wait on semaphore. This operation will try to acquire a lock on the >> * semaphore. If the semaphore is already acquired as many times at it allows, >> - * the function will block until someone releases the lock. >> + * the function will block until someone releases the lock OR timeout expire. >> + * >> + * @return ISC_R_SUCCESS or ISC_R_TIMEDOUT or other errors from ISC libs >> */ >> -void >> -semaphore_wait(semaphore_t *sem) >> +isc_result_t >> +semaphore_wait_timed(semaphore_t *sem) >> { >> + isc_result_t result; >> + isc_time_t abs_timeout; >> REQUIRE(sem != NULL); >> >> LOCK(&sem->mutex); >> + CHECK(isc_time_nowplusinterval(&abs_timeout,&semaphore_wait_timeout)); > > The isc_time_nowplusinterval doesn't modify shared data, please call it before > LOCK(). > >> while (sem->value<= 0) >> - WAIT(&sem->cond,&sem->mutex); >> + CHECK(WAITUNTIL(&sem->cond,&sem->mutex,&abs_timeout)); >> sem->value--; >> >> +cleanup: >> UNLOCK(&sem->mutex); >> + return result; >> } >> >> /* >> diff --git a/src/semaphore.h b/src/semaphore.h >> index 4ca4f65..5aef5d2 100644 >> --- a/src/semaphore.h >> +++ b/src/semaphore.h >> @@ -24,6 +24,10 @@ >> #include >> #include >> >> +/* How many seconds will be added to user-defined connection parameter 'timeout'. */ >> +#define SEM_WAIT_TIMEOUT_ADD 2 /* seconds */ >> +extern isc_interval_t semaphore_wait_timeout; >> + >> /* >> * Semaphore can be "acquired" multiple times. However, it has a maximum >> * number of times someone can acquire him. If a semaphore is already acquired >> @@ -40,7 +44,7 @@ typedef struct semaphore semaphore_t; >> /* Public functions. */ >> isc_result_t semaphore_init(semaphore_t *sem, int value); >> void semaphore_destroy(semaphore_t *sem); >> -void semaphore_wait(semaphore_t *sem); >> +isc_result_t semaphore_wait_timed(semaphore_t *sem); >> void semaphore_signal(semaphore_t *sem); >> >> #endif /* !_LD_SEMAPHORE_H_ */ >> -- >> 1.7.7.6 >> > > -------------- next part -------------- From ff06086e55d64cd6893a6d064d8e101f09a71956 Mon Sep 17 00:00:00 2001 From: Petr Spacek Date: Tue, 24 Apr 2012 15:09:32 +0200 Subject: [PATCH] Add simple semaphore deadlock detection logic. Signed-off-by: Petr Spacek --- src/ldap_helper.c | 78 ++++++++++++++++++++++++++++++++--------------------- src/semaphore.c | 26 +++++++++++++++--- src/semaphore.h | 6 +++- 3 files changed, 74 insertions(+), 36 deletions(-) diff --git a/src/ldap_helper.c b/src/ldap_helper.c index 47c05599cb48d22e172244b2c3229b9320f37d59..304c37296f8f3a428c4c72b45fe4154b2c9e954c 100644 --- a/src/ldap_helper.c +++ b/src/ldap_helper.c @@ -296,9 +296,10 @@ ldap_delete_zone2(ldap_instance_t *inst, dns_name_t *name, isc_boolean_t lock); static isc_result_t ldap_pool_create(isc_mem_t *mctx, unsigned int connections, ldap_pool_t **poolp); static void ldap_pool_destroy(ldap_pool_t **poolp); -static ldap_connection_t * ldap_pool_getconnection(ldap_pool_t *pool); +static isc_result_t ldap_pool_getconnection(ldap_pool_t *pool, + ldap_connection_t ** conn); static void ldap_pool_putconnection(ldap_pool_t *pool, - ldap_connection_t *ldap_conn); + ldap_connection_t ** conn); static isc_result_t ldap_pool_connect(ldap_pool_t *pool, ldap_instance_t *ldap_inst); @@ -402,6 +403,10 @@ new_ldap_instance(isc_mem_t *mctx, const char *db_name, ldap_settings[i++].target = &ldap_inst->dyn_update; CHECK(set_settings(ldap_settings, argv)); + /* Set timer for deadlock detection inside semaphore_wait_timed . */ + if (semaphore_wait_timeout.seconds < ldap_inst->timeout*SEM_WAIT_TIMEOUT_MUL) + semaphore_wait_timeout.seconds = ldap_inst->timeout*SEM_WAIT_TIMEOUT_MUL; + /* Validate and check settings. */ str_toupper(ldap_inst->sasl_mech); if (ldap_inst->connections < 1) { @@ -1089,7 +1094,7 @@ isc_result_t refresh_zones_from_ldap(ldap_instance_t *ldap_inst) { isc_result_t result = ISC_R_SUCCESS; - ldap_connection_t *ldap_conn; + ldap_connection_t *ldap_conn = NULL; int zone_count = 0; ldap_entry_t *entry; dns_rbt_t *rbt = NULL; @@ -1114,7 +1119,7 @@ refresh_zones_from_ldap(ldap_instance_t *ldap_inst) log_debug(2, "refreshing list of zones for %s", ldap_inst->db_name); - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); + CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(ldap_inst->base), LDAP_SCOPE_SUBTREE, config_attrs, 0, @@ -1227,7 +1232,7 @@ cleanup: if (invalidate_nodechain) dns_rbtnodechain_invalidate(&chain); - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); + ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); log_debug(2, "finished refreshing list of zones"); @@ -1381,7 +1386,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam dns_name_t *origin, ldapdb_nodelist_t *nodelist) { isc_result_t result; - ldap_connection_t *ldap_conn; + ldap_connection_t *ldap_conn = NULL; ldap_entry_t *entry; ld_string_t *string = NULL; ldapdb_node_t *node; @@ -1392,7 +1397,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam REQUIRE(nodelist != NULL); /* RRs aren't in the cache, perform ordinary LDAP query */ - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); + CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); INIT_LIST(*nodelist); CHECK(str_new(mctx, &string)); @@ -1439,7 +1444,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam result = ISC_R_SUCCESS; cleanup: - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); + ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); str_destroy(&string); return result; @@ -1450,7 +1455,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na dns_name_t *origin, ldapdb_rdatalist_t *rdatalist) { isc_result_t result; - ldap_connection_t *ldap_conn; + ldap_connection_t *ldap_conn = NULL; ldap_entry_t *entry; ld_string_t *string = NULL; ldap_cache_t *cache; @@ -1468,12 +1473,11 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na return result; /* RRs aren't in the cache, perform ordinary LDAP query */ - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); - INIT_LIST(*rdatalist); CHECK(str_new(mctx, &string)); CHECK(dnsname_to_dn(ldap_inst->zone_register, name, string)); + CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(string), LDAP_SCOPE_BASE, NULL, 0, "(objectClass=idnsRecord)")); @@ -1500,7 +1504,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na result = ISC_R_NOTFOUND; cleanup: - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); + ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); str_destroy(&string); if (result != ISC_R_SUCCESS) @@ -2250,7 +2254,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, zone_dn += 1; /* skip whitespace */ } - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); + CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); CHECK(ldap_query(ldap_inst, ldap_conn, zone_dn, LDAP_SCOPE_BASE, attrs, 0, "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); @@ -2481,9 +2485,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, } cleanup: - if (ldap_conn != NULL) - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); - + ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); str_destroy(&owner_dn_ptr); str_destroy(&owner_dn); free_ldapmod(mctx, &change[0]); @@ -2565,34 +2567,51 @@ ldap_pool_destroy(ldap_pool_t **poolp) MEM_PUT_AND_DETACH(pool); } -static ldap_connection_t * -ldap_pool_getconnection(ldap_pool_t *pool) +static isc_result_t +ldap_pool_getconnection(ldap_pool_t *pool, ldap_connection_t ** conn) { ldap_connection_t *ldap_conn = NULL; unsigned int i; + isc_result_t result; REQUIRE(pool != NULL); + REQUIRE(conn != NULL && *conn == NULL); + ldap_conn = *conn; - semaphore_wait(&pool->conn_semaphore); + CHECK(semaphore_wait_timed(&pool->conn_semaphore)); for (i = 0; i < pool->connections; i++) { ldap_conn = pool->conns[i]; if (isc_mutex_trylock(&ldap_conn->lock) == ISC_R_SUCCESS) break; } RUNTIME_CHECK(ldap_conn != NULL); INIT_LIST(ldap_conn->ldap_entries); + *conn = ldap_conn; - return ldap_conn; +cleanup: + if (result != ISC_R_SUCCESS) { + log_error("timeout in ldap_pool_getconnection(): try to raise " + "'connections' parameter; potential deadlock?"); + } + return result; } static void -ldap_pool_putconnection(ldap_pool_t *pool, ldap_connection_t *ldap_conn) +ldap_pool_putconnection(ldap_pool_t *pool, ldap_connection_t **conn) { + REQUIRE(conn != NULL); + ldap_connection_t *ldap_conn = *conn; + + if (ldap_conn == NULL) + return; + ldap_query_free(ldap_conn); UNLOCK(&ldap_conn->lock); semaphore_signal(&pool->conn_semaphore); + + *conn = NULL; } static isc_result_t @@ -2719,7 +2738,7 @@ update_action(isc_task_t *task, isc_event_t *event) ldap_psearchevent_t *pevent = (ldap_psearchevent_t *)event; isc_result_t result ; ldap_instance_t *inst = NULL; - ldap_connection_t *conn; + ldap_connection_t *conn = NULL; ldap_entry_t *entry; isc_boolean_t delete = ISC_TRUE; isc_mem_t *mctx; @@ -2736,7 +2755,7 @@ update_action(isc_task_t *task, isc_event_t *event) if (result != ISC_R_SUCCESS) goto cleanup; - conn = ldap_pool_getconnection(inst->pool); + CHECK(ldap_pool_getconnection(inst->pool, &conn)); CHECK(ldap_query(inst, conn, pevent->dn, LDAP_SCOPE_BASE, attrs, 0, @@ -2754,14 +2773,13 @@ update_action(isc_task_t *task, isc_event_t *event) if (delete) CHECK(ldap_delete_zone(inst, pevent->dn, ISC_TRUE)); - ldap_pool_putconnection(inst->pool, conn); - cleanup: if (result != ISC_R_SUCCESS) log_error("update_action (psearch) failed for %s. " "Zones can be outdated, run `rndc reload`", pevent->dn); + ldap_pool_putconnection(inst->pool, &conn); isc_mem_free(mctx, pevent->dbname); isc_mem_free(mctx, pevent->dn); isc_mem_detach(&mctx); @@ -2790,7 +2808,7 @@ update_config(isc_task_t *task, isc_event_t *event) if (result != ISC_R_SUCCESS) goto cleanup; - conn = ldap_pool_getconnection(inst->pool); + CHECK(ldap_pool_getconnection(inst->pool, &conn)); CHECK(ldap_query(inst, conn, pevent->dn, LDAP_SCOPE_BASE, attrs, 0, @@ -2809,14 +2827,12 @@ update_config(isc_task_t *task, isc_event_t *event) cleanup: - if (conn != NULL) - ldap_pool_putconnection(inst->pool, conn); - if (result != ISC_R_SUCCESS) log_error("update_config (psearch) failed for %s. " "Configuration can be outdated, run `rndc reload`", pevent->dn); + ldap_pool_putconnection(inst->pool, &conn); isc_mem_free(mctx, pevent->dbname); isc_mem_free(mctx, pevent->dn); isc_mem_detach(&mctx); @@ -3079,7 +3095,7 @@ ldap_psearch_watcher(isc_threadarg_t arg) tv.tv_usec = 0; /* Pick connection, one is reserved purely for this thread */ - conn = ldap_pool_getconnection(inst->pool); + CHECK(ldap_pool_getconnection(inst->pool, &conn)); /* Try to connect. */ while (conn->handle == NULL) { @@ -3189,7 +3205,7 @@ soft_err: log_debug(1, "Ending ldap_psearch_watcher"); cleanup: - ldap_pool_putconnection(inst->pool, conn); + ldap_pool_putconnection(inst->pool, &conn); return (isc_threadresult_t)0; } diff --git a/src/semaphore.c b/src/semaphore.c index 41d6a306b76022a35967cd157ff767cdf2f2588d..352219f113a233218b5522beea5520dddbd748e6 100644 --- a/src/semaphore.c +++ b/src/semaphore.c @@ -27,8 +27,19 @@ #include #include #include +#include #include "semaphore.h" +#include "util.h" + +/* + * Timer setting for deadlock detection. Format: seconds, nanoseconds. + * These values will be overwriten during initialization + * from set_settings() with max(setting+SEM_WAIT_TIMEOUT_ADD, curr_value). + * + * Initial value can be useful in early phases of initialization. + */ +isc_interval_t semaphore_wait_timeout = { 3, 0 }; /* * Initialize a semaphore. @@ -74,20 +85,27 @@ semaphore_destroy(semaphore_t *sem) /* * Wait on semaphore. This operation will try to acquire a lock on the * semaphore. If the semaphore is already acquired as many times at it allows, - * the function will block until someone releases the lock. + * the function will block until someone releases the lock OR timeout expire. + * + * @return ISC_R_SUCCESS or ISC_R_TIMEDOUT or other errors from ISC libs */ -void -semaphore_wait(semaphore_t *sem) +isc_result_t +semaphore_wait_timed(semaphore_t *sem) { + isc_result_t result; + isc_time_t abs_timeout; REQUIRE(sem != NULL); + CHECK(isc_time_nowplusinterval(&abs_timeout, &semaphore_wait_timeout)); LOCK(&sem->mutex); while (sem->value <= 0) - WAIT(&sem->cond, &sem->mutex); + CHECK(WAITUNTIL(&sem->cond, &sem->mutex, &abs_timeout)); sem->value--; +cleanup: UNLOCK(&sem->mutex); + return result; } /* diff --git a/src/semaphore.h b/src/semaphore.h index 4ca4f652f599b520d759f656f42aa782c4ba9b38..1367747c03b9c022dce82b1b0191a496c5d359af 100644 --- a/src/semaphore.h +++ b/src/semaphore.h @@ -24,6 +24,10 @@ #include #include +/* Multiplier for to user-defined connection parameter 'timeout'. */ +#define SEM_WAIT_TIMEOUT_MUL 6 /* times */ +extern isc_interval_t semaphore_wait_timeout; + /* * Semaphore can be "acquired" multiple times. However, it has a maximum * number of times someone can acquire him. If a semaphore is already acquired @@ -40,7 +44,7 @@ typedef struct semaphore semaphore_t; /* Public functions. */ isc_result_t semaphore_init(semaphore_t *sem, int value); void semaphore_destroy(semaphore_t *sem); -void semaphore_wait(semaphore_t *sem); +isc_result_t semaphore_wait_timed(semaphore_t *sem); void semaphore_signal(semaphore_t *sem); #endif /* !_LD_SEMAPHORE_H_ */ -- 1.7.7.6 From atkac at redhat.com Mon May 7 10:50:12 2012 From: atkac at redhat.com (Adam Tkac) Date: Mon, 07 May 2012 12:50:12 +0200 Subject: [Freeipa-devel] [PATCH 0018] Deadlock detection logic In-Reply-To: <4FA7A58C.80403@redhat.com> References: <4F96A8E4.7010002@redhat.com> <4F96B000.204@redhat.com> <20120503121842.GA14952@redhat.com> <4FA7A58C.80403@redhat.com> Message-ID: <4FA7A8E4.4010405@redhat.com> On 05/07/2012 12:35 PM, Petr Spacek wrote: > On 05/03/2012 02:18 PM, Adam Tkac wrote: >> On Tue, Apr 24, 2012 at 03:52:00PM +0200, Petr Spacek wrote: >>> On 04/24/2012 03:21 PM, Petr Spacek wrote: >>>> Hello, >>>> >>>> this patch adds deadlock detection (based on simple timeout) to >>>> current code. >>>> If (probable) deadlock is detected, current action is stopped with >>>> proper error. >>>> >>>> It properly detects "Simo's deadlock" with 'connections' parameter >>>> == 1. >>>> (Described in https://fedorahosted.org/bind-dyndb-ldap/ticket/66) >>>> >>>> Deadlock itself will be fixed by separate patch. >>>> >>>> Petr^2 Spacek >>> >>> Self-NACK :-) >>> >>> Second version of the patch is attached. ldap_pool_getconnection() >>> and ldap_pool_putconnection() now has same interface and more >>> consistent behaviour. >>> >>> Overall functionality is same. >> >> Hi, >> >> although I'm not fully happy with current design of the detection >> logic, we can >> include it before we create something better, for example based on >> thread IDs >> (one thread can acquire semaphore only one time). > I agree, it's far from ideal. Connection and result handling needs > redesign at first. After that I can modify detection logic to be more > accurate. > >> >> Please check my comments inside the patch. > All done. Ack. Please push it to master. Regards, Adam > >> >> Regards, Adam >> >>> From ed7596a9410269c0c29418247971f262b7fa77f3 Mon Sep 17 00:00:00 2001 >>> From: Petr Spacek >>> Date: Tue, 24 Apr 2012 15:09:32 +0200 >>> Subject: [PATCH] Add simple semaphore deadlock detection logic. >>> Signed-off-by: Petr Spacek >>> >>> --- >>> src/ldap_helper.c | 78 >>> ++++++++++++++++++++++++++++++++--------------------- >>> src/semaphore.c | 26 +++++++++++++++--- >>> src/semaphore.h | 6 +++- >>> 3 files changed, 74 insertions(+), 36 deletions(-) >>> >>> diff --git a/src/ldap_helper.c b/src/ldap_helper.c >>> index 47c0559..f4c4d02 100644 >>> --- a/src/ldap_helper.c >>> +++ b/src/ldap_helper.c >>> @@ -296,9 +296,10 @@ ldap_delete_zone2(ldap_instance_t *inst, >>> dns_name_t *name, isc_boolean_t lock); >>> static isc_result_t ldap_pool_create(isc_mem_t *mctx, unsigned int >>> connections, >>> ldap_pool_t **poolp); >>> static void ldap_pool_destroy(ldap_pool_t **poolp); >>> -static ldap_connection_t * ldap_pool_getconnection(ldap_pool_t *pool); >>> +static isc_result_t ldap_pool_getconnection(ldap_pool_t *pool, >>> + ldap_connection_t ** conn); >>> static void ldap_pool_putconnection(ldap_pool_t *pool, >>> - ldap_connection_t *ldap_conn); >>> + ldap_connection_t ** conn); >>> static isc_result_t ldap_pool_connect(ldap_pool_t *pool, >>> ldap_instance_t *ldap_inst); >>> >>> @@ -402,6 +403,10 @@ new_ldap_instance(isc_mem_t *mctx, const char >>> *db_name, >>> ldap_settings[i++].target =&ldap_inst->dyn_update; >>> CHECK(set_settings(ldap_settings, argv)); >>> >>> + /* Set timer for deadlock detection inside semaphore_wait_timed >>> . */ >>> + if (semaphore_wait_timeout.seconds< >>> ldap_inst->timeout+SEM_WAIT_TIMEOUT_ADD) >>> + semaphore_wait_timeout.seconds = >>> ldap_inst->timeout+SEM_WAIT_TIMEOUT_ADD; >> >> I'm affraid such low timeout (12 sec by default) will cause too many >> false >> positives. I recommend to set semaphore timeout to at least 60 seconds. >> >>> /* Validate and check settings. */ >>> str_toupper(ldap_inst->sasl_mech); >>> if (ldap_inst->connections< 1) { >>> @@ -1089,7 +1094,7 @@ isc_result_t >>> refresh_zones_from_ldap(ldap_instance_t *ldap_inst) >>> { >>> isc_result_t result = ISC_R_SUCCESS; >>> - ldap_connection_t *ldap_conn; >>> + ldap_connection_t *ldap_conn = NULL; >>> int zone_count = 0; >>> ldap_entry_t *entry; >>> dns_rbt_t *rbt = NULL; >>> @@ -1114,7 +1119,7 @@ refresh_zones_from_ldap(ldap_instance_t >>> *ldap_inst) >>> >>> log_debug(2, "refreshing list of zones for %s", >>> ldap_inst->db_name); >>> >>> - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); >>> + CHECK(ldap_pool_getconnection(ldap_inst->pool,&ldap_conn)); >>> >>> CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(ldap_inst->base), >>> LDAP_SCOPE_SUBTREE, config_attrs, 0, >>> @@ -1227,7 +1232,7 @@ cleanup: >>> if (invalidate_nodechain) >>> dns_rbtnodechain_invalidate(&chain); >>> >>> - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); >>> + ldap_pool_putconnection(ldap_inst->pool,&ldap_conn); >>> >>> log_debug(2, "finished refreshing list of zones"); >>> >>> @@ -1381,7 +1386,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, >>> ldap_instance_t *ldap_inst, dns_name_t *nam >>> dns_name_t *origin, ldapdb_nodelist_t *nodelist) >>> { >>> isc_result_t result; >>> - ldap_connection_t *ldap_conn; >>> + ldap_connection_t *ldap_conn = NULL; >>> ldap_entry_t *entry; >>> ld_string_t *string = NULL; >>> ldapdb_node_t *node; >>> @@ -1392,7 +1397,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, >>> ldap_instance_t *ldap_inst, dns_name_t *nam >>> REQUIRE(nodelist != NULL); >>> >>> /* RRs aren't in the cache, perform ordinary LDAP query */ >>> - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); >>> + CHECK(ldap_pool_getconnection(ldap_inst->pool,&ldap_conn)); >>> >>> INIT_LIST(*nodelist); >>> CHECK(str_new(mctx,&string)); >>> @@ -1439,7 +1444,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, >>> ldap_instance_t *ldap_inst, dns_name_t *nam >>> result = ISC_R_SUCCESS; >>> >>> cleanup: >>> - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); >>> + ldap_pool_putconnection(ldap_inst->pool,&ldap_conn); >>> str_destroy(&string); >>> >>> return result; >>> @@ -1450,7 +1455,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, >>> ldap_instance_t *ldap_inst, dns_name_t *na >>> dns_name_t *origin, ldapdb_rdatalist_t *rdatalist) >>> { >>> isc_result_t result; >>> - ldap_connection_t *ldap_conn; >>> + ldap_connection_t *ldap_conn = NULL; >>> ldap_entry_t *entry; >>> ld_string_t *string = NULL; >>> ldap_cache_t *cache; >>> @@ -1468,12 +1473,11 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, >>> ldap_instance_t *ldap_inst, dns_name_t *na >>> return result; >>> >>> /* RRs aren't in the cache, perform ordinary LDAP query */ >>> - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); >>> - >>> INIT_LIST(*rdatalist); >>> CHECK(str_new(mctx,&string)); >>> CHECK(dnsname_to_dn(ldap_inst->zone_register, name, string)); >>> >>> + CHECK(ldap_pool_getconnection(ldap_inst->pool,&ldap_conn)); >>> CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(string), >>> LDAP_SCOPE_BASE, NULL, 0, "(objectClass=idnsRecord)")); >>> >>> @@ -1500,7 +1504,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, >>> ldap_instance_t *ldap_inst, dns_name_t *na >>> result = ISC_R_NOTFOUND; >>> >>> cleanup: >>> - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); >>> + ldap_pool_putconnection(ldap_inst->pool,&ldap_conn); >>> str_destroy(&string); >>> >>> if (result != ISC_R_SUCCESS) >>> @@ -2250,7 +2254,7 @@ modify_ldap_common(dns_name_t *owner, >>> ldap_instance_t *ldap_inst, >>> zone_dn += 1; /* skip whitespace */ >>> } >>> >>> - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); >>> + CHECK(ldap_pool_getconnection(ldap_inst->pool,&ldap_conn)); >>> CHECK(ldap_query(ldap_inst, ldap_conn, zone_dn, >>> LDAP_SCOPE_BASE, attrs, 0, >>> >>> "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); >>> @@ -2481,9 +2485,7 @@ modify_ldap_common(dns_name_t *owner, >>> ldap_instance_t *ldap_inst, >>> } >>> >>> cleanup: >>> - if (ldap_conn != NULL) >>> - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); >>> - >>> + ldap_pool_putconnection(ldap_inst->pool,&ldap_conn); >>> str_destroy(&owner_dn_ptr); >>> str_destroy(&owner_dn); >>> free_ldapmod(mctx,&change[0]); >>> @@ -2565,15 +2567,18 @@ ldap_pool_destroy(ldap_pool_t **poolp) >>> MEM_PUT_AND_DETACH(pool); >>> } >>> >>> -static ldap_connection_t * >>> -ldap_pool_getconnection(ldap_pool_t *pool) >>> +static isc_result_t >>> +ldap_pool_getconnection(ldap_pool_t *pool, ldap_connection_t ** conn) >>> { >>> ldap_connection_t *ldap_conn = NULL; >>> unsigned int i; >>> + isc_result_t result; >>> >>> REQUIRE(pool != NULL); >>> + REQUIRE(conn != NULL&& *conn == NULL); >>> + ldap_conn = *conn; >>> >>> - semaphore_wait(&pool->conn_semaphore); >>> + CHECK(semaphore_wait_timed(&pool->conn_semaphore)); >>> for (i = 0; i< pool->connections; i++) { >>> ldap_conn = pool->conns[i]; >>> if (isc_mutex_trylock(&ldap_conn->lock) == ISC_R_SUCCESS) >>> @@ -2583,16 +2588,30 @@ ldap_pool_getconnection(ldap_pool_t *pool) >>> RUNTIME_CHECK(ldap_conn != NULL); >>> >>> INIT_LIST(ldap_conn->ldap_entries); >>> + *conn = ldap_conn; >>> >>> - return ldap_conn; >>> +cleanup: >>> + if (result != ISC_R_SUCCESS) { >>> + log_error("timeout in ldap_pool_getconnection(): try to >>> raise " >>> + "'connections' parameter; potential deadlock?"); >>> + } >>> + return result; >>> } >>> >>> static void >>> -ldap_pool_putconnection(ldap_pool_t *pool, ldap_connection_t >>> *ldap_conn) >>> +ldap_pool_putconnection(ldap_pool_t *pool, ldap_connection_t **conn) >>> { >>> + REQUIRE(conn); >> >> Although this is correct, please use REQUIRE(conn != NULL); >> >>> + ldap_connection_t *ldap_conn = *conn; >>> + >>> + if (ldap_conn == NULL) >>> + return; >>> + >>> ldap_query_free(ldap_conn); >>> UNLOCK(&ldap_conn->lock); >>> semaphore_signal(&pool->conn_semaphore); >>> + >>> + *conn = NULL; >>> } >>> >>> static isc_result_t >>> @@ -2719,7 +2738,7 @@ update_action(isc_task_t *task, isc_event_t >>> *event) >>> ldap_psearchevent_t *pevent = (ldap_psearchevent_t *)event; >>> isc_result_t result ; >>> ldap_instance_t *inst = NULL; >>> - ldap_connection_t *conn; >>> + ldap_connection_t *conn = NULL; >>> ldap_entry_t *entry; >>> isc_boolean_t delete = ISC_TRUE; >>> isc_mem_t *mctx; >>> @@ -2736,7 +2755,7 @@ update_action(isc_task_t *task, isc_event_t >>> *event) >>> if (result != ISC_R_SUCCESS) >>> goto cleanup; >>> >>> - conn = ldap_pool_getconnection(inst->pool); >>> + CHECK(ldap_pool_getconnection(inst->pool,&conn)); >>> >>> CHECK(ldap_query(inst, conn, pevent->dn, >>> LDAP_SCOPE_BASE, attrs, 0, >>> @@ -2754,14 +2773,13 @@ update_action(isc_task_t *task, isc_event_t >>> *event) >>> if (delete) >>> CHECK(ldap_delete_zone(inst, pevent->dn, ISC_TRUE)); >>> >>> - ldap_pool_putconnection(inst->pool, conn); >>> - >>> cleanup: >>> if (result != ISC_R_SUCCESS) >>> log_error("update_action (psearch) failed for %s. " >>> "Zones can be outdated, run `rndc reload`", >>> pevent->dn); >>> >>> + ldap_pool_putconnection(inst->pool,&conn); >>> isc_mem_free(mctx, pevent->dbname); >>> isc_mem_free(mctx, pevent->dn); >>> isc_mem_detach(&mctx); >>> @@ -2790,7 +2808,7 @@ update_config(isc_task_t *task, isc_event_t >>> *event) >>> if (result != ISC_R_SUCCESS) >>> goto cleanup; >>> >>> - conn = ldap_pool_getconnection(inst->pool); >>> + CHECK(ldap_pool_getconnection(inst->pool,&conn)); >>> >>> CHECK(ldap_query(inst, conn, pevent->dn, >>> LDAP_SCOPE_BASE, attrs, 0, >>> @@ -2809,14 +2827,12 @@ update_config(isc_task_t *task, isc_event_t >>> *event) >>> >>> >>> cleanup: >>> - if (conn != NULL) >>> - ldap_pool_putconnection(inst->pool, conn); >>> - >>> if (result != ISC_R_SUCCESS) >>> log_error("update_config (psearch) failed for %s. " >>> "Configuration can be outdated, run `rndc reload`", >>> pevent->dn); >>> >>> + ldap_pool_putconnection(inst->pool,&conn); >>> isc_mem_free(mctx, pevent->dbname); >>> isc_mem_free(mctx, pevent->dn); >>> isc_mem_detach(&mctx); >>> @@ -3079,7 +3095,7 @@ ldap_psearch_watcher(isc_threadarg_t arg) >>> tv.tv_usec = 0; >>> >>> /* Pick connection, one is reserved purely for this thread */ >>> - conn = ldap_pool_getconnection(inst->pool); >>> + CHECK(ldap_pool_getconnection(inst->pool,&conn)); >>> >>> /* Try to connect. */ >>> while (conn->handle == NULL) { >>> @@ -3189,7 +3205,7 @@ soft_err: >>> log_debug(1, "Ending ldap_psearch_watcher"); >>> >>> cleanup: >>> - ldap_pool_putconnection(inst->pool, conn); >>> + ldap_pool_putconnection(inst->pool,&conn); >>> >>> return (isc_threadresult_t)0; >>> } >>> diff --git a/src/semaphore.c b/src/semaphore.c >>> index 41d6a30..76e6aa2 100644 >>> --- a/src/semaphore.c >>> +++ b/src/semaphore.c >>> @@ -27,8 +27,19 @@ >>> #include >>> #include >>> #include >>> +#include >>> >>> #include "semaphore.h" >>> +#include "util.h" >>> + >>> +/* >>> + * Timer setting for deadlock detection. Format: seconds, nanoseconds. >>> + * These values will be overwriten during initialization >>> + * from set_settings() with max(setting+SEM_WAIT_TIMEOUT_ADD, >>> curr_value). >>> + * >>> + * Initial value can be useful in early phases of initialization. >>> + */ >>> +isc_interval_t semaphore_wait_timeout = { 3, 0 }; >>> >>> /* >>> * Initialize a semaphore. >>> @@ -74,20 +85,27 @@ semaphore_destroy(semaphore_t *sem) >>> /* >>> * Wait on semaphore. This operation will try to acquire a lock on >>> the >>> * semaphore. If the semaphore is already acquired as many times >>> at it allows, >>> - * the function will block until someone releases the lock. >>> + * the function will block until someone releases the lock OR >>> timeout expire. >>> + * >>> + * @return ISC_R_SUCCESS or ISC_R_TIMEDOUT or other errors from ISC >>> libs >>> */ >>> -void >>> -semaphore_wait(semaphore_t *sem) >>> +isc_result_t >>> +semaphore_wait_timed(semaphore_t *sem) >>> { >>> + isc_result_t result; >>> + isc_time_t abs_timeout; >>> REQUIRE(sem != NULL); >>> >>> LOCK(&sem->mutex); >>> + >>> CHECK(isc_time_nowplusinterval(&abs_timeout,&semaphore_wait_timeout)); >> >> The isc_time_nowplusinterval doesn't modify shared data, please call >> it before >> LOCK(). >> >>> while (sem->value<= 0) >>> - WAIT(&sem->cond,&sem->mutex); >>> + CHECK(WAITUNTIL(&sem->cond,&sem->mutex,&abs_timeout)); >>> sem->value--; >>> >>> +cleanup: >>> UNLOCK(&sem->mutex); >>> + return result; >>> } >>> >>> /* >>> diff --git a/src/semaphore.h b/src/semaphore.h >>> index 4ca4f65..5aef5d2 100644 >>> --- a/src/semaphore.h >>> +++ b/src/semaphore.h >>> @@ -24,6 +24,10 @@ >>> #include >>> #include >>> >>> +/* How many seconds will be added to user-defined connection >>> parameter 'timeout'. */ >>> +#define SEM_WAIT_TIMEOUT_ADD 2 /* seconds */ >>> +extern isc_interval_t semaphore_wait_timeout; >>> + >>> /* >>> * Semaphore can be "acquired" multiple times. However, it has a >>> maximum >>> * number of times someone can acquire him. If a semaphore is >>> already acquired >>> @@ -40,7 +44,7 @@ typedef struct semaphore semaphore_t; >>> /* Public functions. */ >>> isc_result_t semaphore_init(semaphore_t *sem, int value); >>> void semaphore_destroy(semaphore_t *sem); >>> -void semaphore_wait(semaphore_t *sem); >>> +isc_result_t semaphore_wait_timed(semaphore_t *sem); >>> void semaphore_signal(semaphore_t *sem); >>> >>> #endif /* !_LD_SEMAPHORE_H_ */ >>> -- >>> 1.7.7.6 >>> >> >> > > > bind-dyndb-ldap-pspacek-0018-2-Add-simple-semaphore-deadlock-detection-logic > > > From ff06086e55d64cd6893a6d064d8e101f09a71956 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Tue, 24 Apr 2012 15:09:32 +0200 > Subject: [PATCH] Add simple semaphore deadlock detection logic. > Signed-off-by: Petr Spacek > > --- > src/ldap_helper.c | 78 ++++++++++++++++++++++++++++++++--------------------- > src/semaphore.c | 26 +++++++++++++++--- > src/semaphore.h | 6 +++- > 3 files changed, 74 insertions(+), 36 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index 47c05599cb48d22e172244b2c3229b9320f37d59..304c37296f8f3a428c4c72b45fe4154b2c9e954c 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -296,9 +296,10 @@ ldap_delete_zone2(ldap_instance_t *inst, dns_name_t *name, isc_boolean_t lock); > static isc_result_t ldap_pool_create(isc_mem_t *mctx, unsigned int connections, > ldap_pool_t **poolp); > static void ldap_pool_destroy(ldap_pool_t **poolp); > -static ldap_connection_t * ldap_pool_getconnection(ldap_pool_t *pool); > +static isc_result_t ldap_pool_getconnection(ldap_pool_t *pool, > + ldap_connection_t ** conn); > static void ldap_pool_putconnection(ldap_pool_t *pool, > - ldap_connection_t *ldap_conn); > + ldap_connection_t ** conn); > static isc_result_t ldap_pool_connect(ldap_pool_t *pool, > ldap_instance_t *ldap_inst); > > @@ -402,6 +403,10 @@ new_ldap_instance(isc_mem_t *mctx, const char *db_name, > ldap_settings[i++].target =&ldap_inst->dyn_update; > CHECK(set_settings(ldap_settings, argv)); > > + /* Set timer for deadlock detection inside semaphore_wait_timed . */ > + if (semaphore_wait_timeout.seconds< ldap_inst->timeout*SEM_WAIT_TIMEOUT_MUL) > + semaphore_wait_timeout.seconds = ldap_inst->timeout*SEM_WAIT_TIMEOUT_MUL; > + > /* Validate and check settings. */ > str_toupper(ldap_inst->sasl_mech); > if (ldap_inst->connections< 1) { > @@ -1089,7 +1094,7 @@ isc_result_t > refresh_zones_from_ldap(ldap_instance_t *ldap_inst) > { > isc_result_t result = ISC_R_SUCCESS; > - ldap_connection_t *ldap_conn; > + ldap_connection_t *ldap_conn = NULL; > int zone_count = 0; > ldap_entry_t *entry; > dns_rbt_t *rbt = NULL; > @@ -1114,7 +1119,7 @@ refresh_zones_from_ldap(ldap_instance_t *ldap_inst) > > log_debug(2, "refreshing list of zones for %s", ldap_inst->db_name); > > - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); > + CHECK(ldap_pool_getconnection(ldap_inst->pool,&ldap_conn)); > > CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(ldap_inst->base), > LDAP_SCOPE_SUBTREE, config_attrs, 0, > @@ -1227,7 +1232,7 @@ cleanup: > if (invalidate_nodechain) > dns_rbtnodechain_invalidate(&chain); > > - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); > + ldap_pool_putconnection(ldap_inst->pool,&ldap_conn); > > log_debug(2, "finished refreshing list of zones"); > > @@ -1381,7 +1386,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > dns_name_t *origin, ldapdb_nodelist_t *nodelist) > { > isc_result_t result; > - ldap_connection_t *ldap_conn; > + ldap_connection_t *ldap_conn = NULL; > ldap_entry_t *entry; > ld_string_t *string = NULL; > ldapdb_node_t *node; > @@ -1392,7 +1397,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > REQUIRE(nodelist != NULL); > > /* RRs aren't in the cache, perform ordinary LDAP query */ > - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); > + CHECK(ldap_pool_getconnection(ldap_inst->pool,&ldap_conn)); > > INIT_LIST(*nodelist); > CHECK(str_new(mctx,&string)); > @@ -1439,7 +1444,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > result = ISC_R_SUCCESS; > > cleanup: > - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); > + ldap_pool_putconnection(ldap_inst->pool,&ldap_conn); > str_destroy(&string); > > return result; > @@ -1450,7 +1455,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > dns_name_t *origin, ldapdb_rdatalist_t *rdatalist) > { > isc_result_t result; > - ldap_connection_t *ldap_conn; > + ldap_connection_t *ldap_conn = NULL; > ldap_entry_t *entry; > ld_string_t *string = NULL; > ldap_cache_t *cache; > @@ -1468,12 +1473,11 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > return result; > > /* RRs aren't in the cache, perform ordinary LDAP query */ > - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); > - > INIT_LIST(*rdatalist); > CHECK(str_new(mctx,&string)); > CHECK(dnsname_to_dn(ldap_inst->zone_register, name, string)); > > + CHECK(ldap_pool_getconnection(ldap_inst->pool,&ldap_conn)); > CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(string), > LDAP_SCOPE_BASE, NULL, 0, "(objectClass=idnsRecord)")); > > @@ -1500,7 +1504,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > result = ISC_R_NOTFOUND; > > cleanup: > - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); > + ldap_pool_putconnection(ldap_inst->pool,&ldap_conn); > str_destroy(&string); > > if (result != ISC_R_SUCCESS) > @@ -2250,7 +2254,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > zone_dn += 1; /* skip whitespace */ > } > > - ldap_conn = ldap_pool_getconnection(ldap_inst->pool); > + CHECK(ldap_pool_getconnection(ldap_inst->pool,&ldap_conn)); > CHECK(ldap_query(ldap_inst, ldap_conn, zone_dn, > LDAP_SCOPE_BASE, attrs, 0, > "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); > @@ -2481,9 +2485,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > } > > cleanup: > - if (ldap_conn != NULL) > - ldap_pool_putconnection(ldap_inst->pool, ldap_conn); > - > + ldap_pool_putconnection(ldap_inst->pool,&ldap_conn); > str_destroy(&owner_dn_ptr); > str_destroy(&owner_dn); > free_ldapmod(mctx,&change[0]); > @@ -2565,34 +2567,51 @@ ldap_pool_destroy(ldap_pool_t **poolp) > MEM_PUT_AND_DETACH(pool); > } > > -static ldap_connection_t * > -ldap_pool_getconnection(ldap_pool_t *pool) > +static isc_result_t > +ldap_pool_getconnection(ldap_pool_t *pool, ldap_connection_t ** conn) > { > ldap_connection_t *ldap_conn = NULL; > unsigned int i; > + isc_result_t result; > > REQUIRE(pool != NULL); > + REQUIRE(conn != NULL&& *conn == NULL); > + ldap_conn = *conn; > > - semaphore_wait(&pool->conn_semaphore); > + CHECK(semaphore_wait_timed(&pool->conn_semaphore)); > for (i = 0; i< pool->connections; i++) { > ldap_conn = pool->conns[i]; > if (isc_mutex_trylock(&ldap_conn->lock) == ISC_R_SUCCESS) > break; > } > > RUNTIME_CHECK(ldap_conn != NULL); > > INIT_LIST(ldap_conn->ldap_entries); > + *conn = ldap_conn; > > - return ldap_conn; > +cleanup: > + if (result != ISC_R_SUCCESS) { > + log_error("timeout in ldap_pool_getconnection(): try to raise " > + "'connections' parameter; potential deadlock?"); > + } > + return result; > } > > static void > -ldap_pool_putconnection(ldap_pool_t *pool, ldap_connection_t *ldap_conn) > +ldap_pool_putconnection(ldap_pool_t *pool, ldap_connection_t **conn) > { > + REQUIRE(conn != NULL); > + ldap_connection_t *ldap_conn = *conn; > + > + if (ldap_conn == NULL) > + return; > + > ldap_query_free(ldap_conn); > UNLOCK(&ldap_conn->lock); > semaphore_signal(&pool->conn_semaphore); > + > + *conn = NULL; > } > > static isc_result_t > @@ -2719,7 +2738,7 @@ update_action(isc_task_t *task, isc_event_t *event) > ldap_psearchevent_t *pevent = (ldap_psearchevent_t *)event; > isc_result_t result ; > ldap_instance_t *inst = NULL; > - ldap_connection_t *conn; > + ldap_connection_t *conn = NULL; > ldap_entry_t *entry; > isc_boolean_t delete = ISC_TRUE; > isc_mem_t *mctx; > @@ -2736,7 +2755,7 @@ update_action(isc_task_t *task, isc_event_t *event) > if (result != ISC_R_SUCCESS) > goto cleanup; > > - conn = ldap_pool_getconnection(inst->pool); > + CHECK(ldap_pool_getconnection(inst->pool,&conn)); > > CHECK(ldap_query(inst, conn, pevent->dn, > LDAP_SCOPE_BASE, attrs, 0, > @@ -2754,14 +2773,13 @@ update_action(isc_task_t *task, isc_event_t *event) > if (delete) > CHECK(ldap_delete_zone(inst, pevent->dn, ISC_TRUE)); > > - ldap_pool_putconnection(inst->pool, conn); > - > cleanup: > if (result != ISC_R_SUCCESS) > log_error("update_action (psearch) failed for %s. " > "Zones can be outdated, run `rndc reload`", > pevent->dn); > > + ldap_pool_putconnection(inst->pool,&conn); > isc_mem_free(mctx, pevent->dbname); > isc_mem_free(mctx, pevent->dn); > isc_mem_detach(&mctx); > @@ -2790,7 +2808,7 @@ update_config(isc_task_t *task, isc_event_t *event) > if (result != ISC_R_SUCCESS) > goto cleanup; > > - conn = ldap_pool_getconnection(inst->pool); > + CHECK(ldap_pool_getconnection(inst->pool,&conn)); > > CHECK(ldap_query(inst, conn, pevent->dn, > LDAP_SCOPE_BASE, attrs, 0, > @@ -2809,14 +2827,12 @@ update_config(isc_task_t *task, isc_event_t *event) > > > cleanup: > - if (conn != NULL) > - ldap_pool_putconnection(inst->pool, conn); > - > if (result != ISC_R_SUCCESS) > log_error("update_config (psearch) failed for %s. " > "Configuration can be outdated, run `rndc reload`", > pevent->dn); > > + ldap_pool_putconnection(inst->pool,&conn); > isc_mem_free(mctx, pevent->dbname); > isc_mem_free(mctx, pevent->dn); > isc_mem_detach(&mctx); > @@ -3079,7 +3095,7 @@ ldap_psearch_watcher(isc_threadarg_t arg) > tv.tv_usec = 0; > > /* Pick connection, one is reserved purely for this thread */ > - conn = ldap_pool_getconnection(inst->pool); > + CHECK(ldap_pool_getconnection(inst->pool,&conn)); > > /* Try to connect. */ > while (conn->handle == NULL) { > @@ -3189,7 +3205,7 @@ soft_err: > log_debug(1, "Ending ldap_psearch_watcher"); > > cleanup: > - ldap_pool_putconnection(inst->pool, conn); > + ldap_pool_putconnection(inst->pool,&conn); > > return (isc_threadresult_t)0; > } > diff --git a/src/semaphore.c b/src/semaphore.c > index 41d6a306b76022a35967cd157ff767cdf2f2588d..352219f113a233218b5522beea5520dddbd748e6 100644 > --- a/src/semaphore.c > +++ b/src/semaphore.c > @@ -27,8 +27,19 @@ > #include > #include > #include > +#include > > #include "semaphore.h" > +#include "util.h" > + > +/* > + * Timer setting for deadlock detection. Format: seconds, nanoseconds. > + * These values will be overwriten during initialization > + * from set_settings() with max(setting+SEM_WAIT_TIMEOUT_ADD, curr_value). > + * > + * Initial value can be useful in early phases of initialization. > + */ > +isc_interval_t semaphore_wait_timeout = { 3, 0 }; > > /* > * Initialize a semaphore. > @@ -74,20 +85,27 @@ semaphore_destroy(semaphore_t *sem) > /* > * Wait on semaphore. This operation will try to acquire a lock on the > * semaphore. If the semaphore is already acquired as many times at it allows, > - * the function will block until someone releases the lock. > + * the function will block until someone releases the lock OR timeout expire. > + * > + * @return ISC_R_SUCCESS or ISC_R_TIMEDOUT or other errors from ISC libs > */ > -void > -semaphore_wait(semaphore_t *sem) > +isc_result_t > +semaphore_wait_timed(semaphore_t *sem) > { > + isc_result_t result; > + isc_time_t abs_timeout; > REQUIRE(sem != NULL); > > + CHECK(isc_time_nowplusinterval(&abs_timeout,&semaphore_wait_timeout)); > LOCK(&sem->mutex); > > while (sem->value<= 0) > - WAIT(&sem->cond,&sem->mutex); > + CHECK(WAITUNTIL(&sem->cond,&sem->mutex,&abs_timeout)); > sem->value--; > > +cleanup: > UNLOCK(&sem->mutex); > + return result; > } > > /* > diff --git a/src/semaphore.h b/src/semaphore.h > index 4ca4f652f599b520d759f656f42aa782c4ba9b38..1367747c03b9c022dce82b1b0191a496c5d359af 100644 > --- a/src/semaphore.h > +++ b/src/semaphore.h > @@ -24,6 +24,10 @@ > #include > #include > > +/* Multiplier for to user-defined connection parameter 'timeout'. */ > +#define SEM_WAIT_TIMEOUT_MUL 6 /* times */ > +extern isc_interval_t semaphore_wait_timeout; > + > /* > * Semaphore can be "acquired" multiple times. However, it has a maximum > * number of times someone can acquire him. If a semaphore is already acquired > @@ -40,7 +44,7 @@ typedef struct semaphore semaphore_t; > /* Public functions. */ > isc_result_t semaphore_init(semaphore_t *sem, int value); > void semaphore_destroy(semaphore_t *sem); > -void semaphore_wait(semaphore_t *sem); > +isc_result_t semaphore_wait_timed(semaphore_t *sem); > void semaphore_signal(semaphore_t *sem); > > #endif /* !_LD_SEMAPHORE_H_ */ From pspacek at redhat.com Mon May 7 11:19:03 2012 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 07 May 2012 13:19:03 +0200 Subject: [Freeipa-devel] [PATCH 0018] Deadlock detection logic In-Reply-To: <4FA7A8E4.4010405@redhat.com> References: <4F96A8E4.7010002@redhat.com> <4F96B000.204@redhat.com> <20120503121842.GA14952@redhat.com> <4FA7A58C.80403@redhat.com> <4FA7A8E4.4010405@redhat.com> Message-ID: <4FA7AFA7.3010805@redhat.com> On 05/07/2012 12:50 PM, Adam Tkac wrote: > On 05/07/2012 12:35 PM, Petr Spacek wrote: >> On 05/03/2012 02:18 PM, Adam Tkac wrote: >>> On Tue, Apr 24, 2012 at 03:52:00PM +0200, Petr Spacek wrote: >>>> On 04/24/2012 03:21 PM, Petr Spacek wrote: >>>>> Hello, >>>>> >>>>> this patch adds deadlock detection (based on simple timeout) to current >>>>> code. >>>>> If (probable) deadlock is detected, current action is stopped with proper >>>>> error. >>>>> >>>>> It properly detects "Simo's deadlock" with 'connections' parameter == 1. >>>>> (Described in https://fedorahosted.org/bind-dyndb-ldap/ticket/66) >>>>> >>>>> Deadlock itself will be fixed by separate patch. >>>>> >>>>> Petr^2 Spacek >>>> >>>> Self-NACK :-) >>>> >>>> Second version of the patch is attached. ldap_pool_getconnection() >>>> and ldap_pool_putconnection() now has same interface and more >>>> consistent behaviour. >>>> >>>> Overall functionality is same. >>> >>> Hi, >>> >>> although I'm not fully happy with current design of the detection logic, we >>> can >>> include it before we create something better, for example based on thread IDs >>> (one thread can acquire semaphore only one time). >> I agree, it's far from ideal. Connection and result handling needs redesign >> at first. After that I can modify detection logic to be more accurate. >> >>> >>> Please check my comments inside the patch. >> All done. > > Ack. Please push it to master. Pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/481e350f5848cf01da6743f259a6f12419fc4177 Petr^2 Spacek > Regards, Adam >>> Regards, Adam From mkosek at redhat.com Mon May 7 12:10:14 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 07 May 2012 14:10:14 +0200 Subject: [Freeipa-devel] [PATCH] 0038 Do not use extra command options in the automount plugin In-Reply-To: <4F8E97F6.2070703@redhat.com> References: <4F8E97F6.2070703@redhat.com> Message-ID: <1336392614.29911.13.camel@balmora.brq.redhat.com> On Wed, 2012-04-18 at 12:31 +0200, Petr Viktorin wrote: > Part of the work for https://fedorahosted.org/freeipa/ticket/2509 > > Validation doesn't complain about extra command options, which means > e.g. that misspelling an option's name will cause it to be ignored. This > allows errors in custom RPC clients (such as our test suite) to go > unnoticed. > > > Since I expect the review to take some time, and the master branch is a > moving target, I'll post this in several patches. This patch is for the > automount plugin; a patch for ACI and a patch to wrap up and enable the > validation will follow later. > ACK. Pushed to master. Martin From jcholast at redhat.com Mon May 7 12:48:23 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 07 May 2012 14:48:23 +0200 Subject: [Freeipa-devel] [PATCH] 78 Redo boolean value encoding Message-ID: <4FA7C497.4020003@redhat.com> Hi, this patch changes the way boolean values are encoded to LDAP boolean syntax. The code for encoding boolean values is moved from the Parameter class to the Encoder class, where the rest of LDAP encoding takes place. The patch removes encoding code from the Parameter class altogether, as all LDAP encoding should be done in the Encoder class. Unit tests show no regressions and fixes for related tickets ( and ) seem to be intact. Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-78-redo-boolean-encoding.patch Type: text/x-patch Size: 8207 bytes Desc: not available URL: From pspacek at redhat.com Mon May 7 12:49:07 2012 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 07 May 2012 14:49:07 +0200 Subject: [Freeipa-devel] [PATCH 0020] Separate LDAP result from LDAP connection, fix deadlock. Message-ID: <4FA7C4C3.7020202@redhat.com> Hello, this patch fixes https://fedorahosted.org/bind-dyndb-ldap/ticket/66: Plugin deadlocks during new zone load when connections == 1. It fixes structural problem, when LDAP query result was tied with LDAP connection up. It wasn't possible to release connection and work with query result after that. Described deadlock is consequence of this problematic design. Now LDAP connection is separated from LDAP result. Next planed patch will avoid "manual" connection management, so possibility of deadlock should be next to zero. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0020-Separate-LDAP-result-from-LDAP-connection.patch Type: text/x-patch Size: 17360 bytes Desc: not available URL: From mkosek at redhat.com Mon May 7 15:25:01 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 07 May 2012 17:25:01 +0200 Subject: [Freeipa-devel] [PATCHES] 0042-43 Fix internal server errors with empty options In-Reply-To: <4F9E82F2.1010205@redhat.com> References: <4F9A9DE4.9060500@redhat.com> <4F9E82F2.1010205@redhat.com> Message-ID: <1336404301.29911.42.camel@balmora.brq.redhat.com> On Mon, 2012-04-30 at 14:17 +0200, Petr Viktorin wrote: > On 04/27/2012 03:23 PM, Petr Viktorin wrote: > > Empty values in reverse_member options, and attattr/setattr/delattr, > > caused internal server errors. > > > > We convert empty values to None and bypass normal validation, so they > > need special care. > > > > > > https://fedorahosted.org/freeipa/ticket/2680 > > https://fedorahosted.org/freeipa/ticket/2681 > > > > Fixing the subject, sorry for the noise. > > ACK. Pushed both to master. Martin From pspacek at redhat.com Mon May 7 15:48:01 2012 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 07 May 2012 17:48:01 +0200 Subject: [Freeipa-devel] DNS zone serial number updates [#2554] In-Reply-To: <1334764404.16658.277.camel@willson.li.ssimo.org> References: <4F8D9110.3030703@redhat.com> <1334679211.16658.200.camel@willson.li.ssimo.org> <4F8EC1C8.4090204@redhat.com> <1334757864.16658.258.camel@willson.li.ssimo.org> <4F8EDBE0.7090108@redhat.com> <1334764404.16658.277.camel@willson.li.ssimo.org> Message-ID: <4FA7EEB1.1040909@redhat.com> Hello, I found following reply in unsent mail, so I resending it to complete original thread. It's just "for record", because original idea was obsoleted by recent discussion on Thursday meeting. Petr^2 Spacek On 04/18/2012 05:53 PM, Simo Sorce wrote: > On Wed, 2012-04-18 at 17:21 +0200, Petr Spacek wrote: > >>> If this happens, it is possible that on one of the masters the serial >>> will be updated twice even though no other change was performed on the >>> entry-set. That is not a big deal though, at most it will cause a >>> useless zone transfer, but zone transfer should already be somewhat rate >>> limited anyway, because our zones do change frequently due to DNS >>> updates from clients. >> SOA record has also refresh, retry and expiry fields. These define how often >> zone transfer should happen. It's nicely described in [8]. > > Sure but we have 2 opposing requirements. On one hand we want to avoid > doing too many transfers too often. On the other we want to reflect > dynamic Updates fast enough to avoid stale entries in the slaves. So the > problem is that we need to carefully balance and tradeoff. > >> There is next problem with transfers: Currently we support only full zone >> transfers (AXFR), not incremental updates (IXFR), because there is no "last >> record change"<->SOA# information. For now it's postponed, because nobody >> wanted it. > > We can support IXFR, using entryUSN, all entries that have a entryUSN > higher than the max we recorded at last trabfer time would need to be > transfered. It's not urgent, but we have a way to do it relatively > easily. I think the main issue here would be deleted records. We might > need to access the tombstone for it, or keep some record of deleted > entries somewhere, needs more investigation. Ticket for this enhancement created: https://fedorahosted.org/bind-dyndb-ldap/ticket/64 It definitely needs investigation, but I think we can postpone it for now. >>> You still need to search the whole cache and save additional data. (I >>> sure hope you do not keep in memory the whole ldap object but a parsed >>> version of it, if you keep the whole LDAP object I think we just found >>> another place for enhancement. Wasting all that memory is not a good >>> idea IMO). >> Only DNS records are stored, i.e. parsed objects. > > Excellent. > >> Please, can you explain "You still need to search the whole cache and save >> additional data."? I probably missed some important point. > > I was referring to your proposal to store the modifyTimestamp in the > cache. But I think we agree modifyTimestamp is not the way to go so we > can ignore the rest. Ah, it is misunderstanding. I meant "store only max(modifyTimestamp)", not each separate modifyTimestamp value. It doesn't matter, modifyTimestamp is not the right way. >>>> There are still problems to solve without DS plugin (specifically >>>> mapping/updating NN part from YYYYMMDDNN), but: Sounds this reasonable? >>> >>> Well I am not sure we need to use a YYYYMMDDNN convention to start with. >>> I expect with DYNDNS updates that a 2 digit NN will never be enough, >>> plus it is never updated by a human so we do not need to keep it >>> readable. But I do not care eiither way, as long as the serial can >>> handle thousands of updates per day I am fine (if this is an issue we >>> need to understand how to update the serial in time intervals). >> Current BIND implementation handles overflow in one day gracefully: >> 2012041899 -> 2012041900 >> So SOA# can be in far future, if you changes zone too often :-) > > Sure, but then what's the point of keeping it in date format ? :-) I agree, it doesn't make sense. I consulted this with Adam on IRC: We don't have to follow this syntax. >> AFAIK this format is traditional, but not required by standard, if arithmetic >> works. [9] defines arithmetic for SOA serials, so DS plugin should follow it. >> >> It says "The maximum defined increment is 2147483647 (2^31 - 1)" >> This limit applies inside to one SOA TTL time window (so it shouldn't be a >> problem, I think). I didn't looked into in this RFC deeply. Some practical >> recommendations can be found in [10]. > > Yeah 2^31 is large enough for practical deployments if you start small, > if you start close to the top (2012 is not the far from 2147) then you > have substantially reduced the window. I didn't described it well. (2^31 - 1) is maximal *increment* inside one SOA expiry time window, not maximal value. Time window is typically between days and weeks. SOA serial number range is 0..(2^32 - 1). Defined arithmetic allows to overflow (2^32 - 1) -> 0 gracefully (with proper result = make a transfer to slaves). Only limitation is (2^31 - 1) increment during SOA expiry time window. For 4 weeks SOA expire timer: 2^31/(60*60*24*7*4) = 887 updates per second during 4 weeks So there is no real limitation, we can forget to this problem I think. I created bind-dyndb-ldap ticket for SOA update inside DS: https://fedorahosted.org/bind-dyndb-ldap/ticket/65 Petr^2 Spacek > >> Thanks for your time. > > YW > > Simo. > From mkosek at redhat.com Mon May 7 15:59:38 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 07 May 2012 17:59:38 +0200 Subject: [Freeipa-devel] [PATCH] 78 Redo boolean value encoding In-Reply-To: <4FA7C497.4020003@redhat.com> References: <4FA7C497.4020003@redhat.com> Message-ID: <1336406378.29911.55.camel@balmora.brq.redhat.com> On Mon, 2012-05-07 at 14:48 +0200, Jan Cholasta wrote: > Hi, > > this patch changes the way boolean values are encoded to LDAP boolean > syntax. The code for encoding boolean values is moved from the Parameter > class to the Encoder class, where the rest of LDAP encoding takes place. > The patch removes encoding code from the Parameter class altogether, as > all LDAP encoding should be done in the Encoder class. > > Unit tests show no regressions and fixes for related tickets > ( and > ) seem to be intact. > > Honza The patch looks ok, unit tests pass and I also did not find any regression. I have just one concern - with this patch, encoding is bound to native Python type and not to our Param classes. This means that all Params based on the same native Python type (lets say, str) will be encoded in the same way. Are you sure that nobody would want to implement a str-based param that has a custom LDAP encoding? Martin From jcholast at redhat.com Mon May 7 16:49:35 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 07 May 2012 18:49:35 +0200 Subject: [Freeipa-devel] [PATCH] 78 Redo boolean value encoding In-Reply-To: <1336406378.29911.55.camel@balmora.brq.redhat.com> References: <4FA7C497.4020003@redhat.com> <1336406378.29911.55.camel@balmora.brq.redhat.com> Message-ID: <4FA7FD1F.8050401@redhat.com> On 7.5.2012 17:59, Martin Kosek wrote: > On Mon, 2012-05-07 at 14:48 +0200, Jan Cholasta wrote: >> Hi, >> >> this patch changes the way boolean values are encoded to LDAP boolean >> syntax. The code for encoding boolean values is moved from the Parameter >> class to the Encoder class, where the rest of LDAP encoding takes place. >> The patch removes encoding code from the Parameter class altogether, as >> all LDAP encoding should be done in the Encoder class. >> >> Unit tests show no regressions and fixes for related tickets >> ( and >> ) seem to be intact. >> >> Honza > > The patch looks ok, unit tests pass and I also did not find any > regression. > > I have just one concern - with this patch, encoding is bound to native > Python type and not to our Param classes. This means that all Params > based on the same native Python type (lets say, str) will be encoded in > the same way. Are you sure that nobody would want to implement a > str-based param that has a custom LDAP encoding? > > Martin > I don't think we want to do this, see (especially the ending). Honza -- Jan Cholasta From pspacek at redhat.com Mon May 7 20:29:12 2012 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 07 May 2012 22:29:12 +0200 Subject: [Freeipa-devel] DNS zone serial number updates [#2554]: local SOA approach In-Reply-To: <4FA7EEB1.1040909@redhat.com> References: <4F8D9110.3030703@redhat.com> <1334679211.16658.200.camel@willson.li.ssimo.org> <4F8EC1C8.4090204@redhat.com> <1334757864.16658.258.camel@willson.li.ssimo.org> <4F8EDBE0.7090108@redhat.com> <1334764404.16658.277.camel@willson.li.ssimo.org> <4FA7EEB1.1040909@redhat.com> Message-ID: <4FA83098.4000800@redhat.com> Hello, on the last meeting there was another approach to $SUBJ$ discussed: Each DNS server will maintain its own serial number value independently from other servers. Pros: Should be simpler to implement; no DS plugin required. Cons: Slave DNS servers cannot fall-back to other masters, because of SOA serial inconsistency. Very basic implementation: 1) Do not replicate idnsSoaSerial attribute 2) Use persistent search to watch for incoming changes 3) After each change increment "local" SOA serial number (and write it to LDAP - to survive DNS server restart) There is a problem with database updates if bind-dyndb-ldap plugin is not running, but "its" DS runs. In that case DS can replicate changes from other servers but there is no bind-dyndb-ldap plugin to modify serial number. I think it can happen during startup/shutdown phase or after BIND crash. In this case zone content can be changed without appropriate SOA serial update. Dumb solution: Always increment stored SOA serial number after DNS server start. It causes unnecessary zone transfer, but we are safe. Another solution (IPA+389 specific): Remember max(entryUSN) value computed from all records together with SOA serial. (Principle is the same as with modifyTimestamp described earlier in this thread.) It requires new LDAP object with two attributes: - assigned value of idnsSOASerial - highest entryUSN found DNS server after start can check if max(entryUSN) == recorded max value. If values do not match plugin bumps idnsSOASerial. If entryUSN is not supported by server, we can fall-back to bumping idnsSOASerial on every start-up. Did I miss anything? :-) It requires persistent search, but I resigned already. Petr^2 Spacek P.S.: There is a pretty new RFC about zone transfers with "very nice" simplification: http://tools.ietf.org/html/rfc5936#section-3.1 ... Zones for which it is impractical to list the entire zone for a serial number are not suitable for AXFR retrieval. A typical (but not limiting) description of such a zone is a zone consisting of responses generated via other database lookups ... From edewata at redhat.com Mon May 7 23:47:32 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 07 May 2012 18:47:32 -0500 Subject: [Freeipa-devel] [PATCHES] 124-132 Inconsistent ways to show/change entry status In-Reply-To: <4FA137C4.7000101@redhat.com> References: <4FA137C4.7000101@redhat.com> Message-ID: <4FA85F14.40705@redhat.com> The code works, it's ACKed. I have some comments below: On 5/2/2012 8:33 AM, Petr Vobornik wrote: > This bunch of patches are implementing ticket #2247. They introduce some > new logic and types of internal objects. There might be design issues > (mainly in state evaluation). I would appreciate some opinions on what > might be improved. > > See patch comments for more details. > > What I think might be the main concerns: > > Action list definition: > Now action lists are defined on facet level and facet header takes them > from facet. Would in be better to define action list - the widget and > actions separately. The widget could be defined in header and actions on > facet level. Yes, the header could be considered a widget, so it contain the action list widget. > State evaluation: > The patches are adding support for some kind of state evaluation. The > state is represented by array of string. I'm thinking that the design > might not be robust. Is a string good enough? We might have a problem > with conditions like to "have this particular access right' (#2318). > There are state_evaluators and state_listeners. They are practically the > same but the execution point and parameters are different. Should they > be somehow joined? It might be better to use a map to represent the state. The map could hold any attribute types, not just string flag with the current array. So, the widget itself could be the state because it's also a map. That said, evaluating a map might be difficult to define declaratively, we would end up using a code again. So for now using an array of string is fine, but we just have to know the limitations. > Code placement: > There's a lot of new objects and for some of them it is not clear to > what code file they should be placed. This will continue to change as we add more code. Feel free to create a new file when you see a pattern. We might want to start separating the IPA-specific code from the framework (reusable code). The framework could be moved into a 'lib' folder. > FYI: In close future I would like to address some problems in UI > architecture. I'm in a middle of designing phase, so there is nothing to > present at the moment. The main topics are: > * reduce the need of overriding methods when a new widgets or > capabilities are added > * make it more declarative to enhance extendibility > It may be done by: > * better inheriting model to support events > * build phases (preinit, init, postinit, create, load) to improve spec > object creating and initialization of created object. > * path representation of an event/attribute/model property to support > bindings to various events/attrs from anywhere > * introduce model and model bindings, converters between command > output/model/human readable representation Sounds good! Some comments about the patch: 1. When you open a specific user, there's a default action that's already selected in the action list. The thing is the current default action (i.e. Disable) is probably not something that's regularly used. It might be better to show something like '-- Select Action --' or '-- Action List --' so you'd have to select an action first then click Apply. 2. Suppose we did #1, do we still need the confirmation when applying the action? 3. I think the action list should be available in all details page for all entities. At least it should have the Delete action. 4. Something to think about, do we need an action list in the association page? Entities like group default to an association page instead of details page. So if we don't have an action list in the association page the UI may look inconsistent because you'd have to go to the Settings to execute an action. But if we do have an action list in the association page, the actions could become confusing: do they apply to the associations or to the entry itself? One possible solution is to move the Settings to the left-most position and make it the default page and have the action list in the details page only. 5. Some details pages have status indicator (check sign or minus sign) on the left of the page title, some others don't. This is because not all entities have a concept of status. Would it be better to show the status as a read-only field in the facet content? 6. It might be better to avoid using element ID in the CSS. This would make the CSS more reusable. #content .facet-action-list div[name=apply] a.ui-state-default { ... } 7. In some facet class definitions the no_init parameter is defined separately from the spec object, any particular reason? IPA.facet = function(spec, no_init) { ... }; You can think of spec object as named parameters. So the no_init can be defined in the spec object and later used to determine what operations to be done inside the init method. 8. The declarative control button definitions still contain some code, e.g. the handler function inside the button actions. Maybe the actions should be defined in a separate list and the buttons can refer to them by name. 9. The 'control_buttons' attribute in search facet definitions contains a 'buttons' array. Any plans to create custom control buttons widget? If not maybe the 'control_buttons' itself could be the array. 10. The 'Action List' in ticket #2248 for the password reset is actually different than the action list for Enable/Disable. I was proposing to create a small panel under the 'Account Settings' section, probably something like this: --------------------------------------------------------------- Account Settings +-----------------+ User login: admin | Actions: | Password: | Reset password | Password expiration: +-----------------+ UID: GID: This panel might better be called 'Action Panel' to distinguish from the dropdown 'Action List' on the top. The reason for this panel is that a Password Reset only affects an aspect of the user, not the entire object like Enable/Disable (although that can also be argued differently), so the action should be placed where it's relevant, in this case near the Password field. The same concept will be used for ticket #2250, #2251, #2252. -- Endi S. Dewata From edewata at redhat.com Tue May 8 02:23:54 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 07 May 2012 21:23:54 -0500 Subject: [Freeipa-devel] [PATCHES] 124-132 Inconsistent ways to show/change entry status In-Reply-To: <4FA287A0.2070107@redhat.com> References: <4FA137C4.7000101@redhat.com> <4FA285D0.90301@redhat.com> <4FA287A0.2070107@redhat.com> Message-ID: <4FA883BA.8080803@redhat.com> On 5/3/2012 8:26 AM, Petr Vobornik wrote: > On 05/03/2012 03:19 PM, Petr Vobornik wrote: >> I found that limitation of maximum pkey length in facet header is not >> working well. Attaching patch #134 which actually calculates it. > > I found useless line in the patch. Corrected version attached. Try adding a very long DNS zone, then open the zone. Compare the breadcrumbs in the DNS Resource Records page and in the Settings page, in my case the second one is a bit longer. I think the length of the breadcrumb should be calculated differently. Btw, I just noticed that the facet tab label for DNS Resource Records is changed to 'Search'. Same problem happens in Automount. -- Endi S. Dewata From mkosek at redhat.com Wed May 9 07:44:12 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 09 May 2012 09:44:12 +0200 Subject: [Freeipa-devel] [PATCH] 78 Redo boolean value encoding In-Reply-To: <4FA7FD1F.8050401@redhat.com> References: <4FA7C497.4020003@redhat.com> <1336406378.29911.55.camel@balmora.brq.redhat.com> <4FA7FD1F.8050401@redhat.com> Message-ID: <1336549452.4800.2.camel@balmora.brq.redhat.com> On Mon, 2012-05-07 at 18:49 +0200, Jan Cholasta wrote: > On 7.5.2012 17:59, Martin Kosek wrote: > > On Mon, 2012-05-07 at 14:48 +0200, Jan Cholasta wrote: > >> Hi, > >> > >> this patch changes the way boolean values are encoded to LDAP boolean > >> syntax. The code for encoding boolean values is moved from the Parameter > >> class to the Encoder class, where the rest of LDAP encoding takes place. > >> The patch removes encoding code from the Parameter class altogether, as > >> all LDAP encoding should be done in the Encoder class. > >> > >> Unit tests show no regressions and fixes for related tickets > >> ( and > >> ) seem to be intact. > >> > >> Honza > > > > The patch looks ok, unit tests pass and I also did not find any > > regression. > > > > I have just one concern - with this patch, encoding is bound to native > > Python type and not to our Param classes. This means that all Params > > based on the same native Python type (lets say, str) will be encoded in > > the same way. Are you sure that nobody would want to implement a > > str-based param that has a custom LDAP encoding? > > > > Martin > > > > I don't think we want to do this, see > > (especially the ending). > > Honza > Ok. We may revisit this idea when the the need for custom per-Param encoding is justified and really needed. ACK. Pushed to master. Martin From mkosek at redhat.com Wed May 9 07:55:38 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 09 May 2012 09:55:38 +0200 Subject: [Freeipa-devel] [PATCH] 0046 Don't fail when adding default objectclasses using config-mod In-Reply-To: <4FA2B179.4050108@redhat.com> References: <4FA2B179.4050108@redhat.com> Message-ID: <1336550138.4800.3.camel@balmora.brq.redhat.com> On Thu, 2012-05-03 at 18:25 +0200, Petr Viktorin wrote: > Fix another setattr internal error that QA found. > > https://fedorahosted.org/freeipa/ticket/2706 > ACK. Pushed to master. Martin From mkosek at redhat.com Wed May 9 09:58:59 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 09 May 2012 11:58:59 +0200 Subject: [Freeipa-devel] [PATCH] 0039 Remove duplicate and unused utility code In-Reply-To: <4F8FE9EB.6030602@redhat.com> References: <4F8FE9EB.6030602@redhat.com> Message-ID: <1336557539.4800.5.camel@balmora.brq.redhat.com> On Thu, 2012-04-19 at 12:33 +0200, Petr Viktorin wrote: > IPA has some unused code from abandoned features (Radius, ipa 1.x user > input, command-line tab completion), as well as some duplicate utilities. > This patch cleans up the utility modules. > > https://fedorahosted.org/freeipa/ticket/2650 > The patch looks ok, I didn't find any breakage in unit tests or my smoke tests. So ACK, pushed to master. Martin From atkac at redhat.com Wed May 9 11:24:26 2012 From: atkac at redhat.com (Adam Tkac) Date: Wed, 09 May 2012 13:24:26 +0200 Subject: [Freeipa-devel] [PATCH 0019] Add proper DN escaping before LDAP library calls In-Reply-To: <4FA28C43.6060408@redhat.com> References: <4FA24F17.2000106@redhat.com> <4FA28C43.6060408@redhat.com> Message-ID: <4FAA53EA.1080803@redhat.com> On 05/03/2012 03:46 PM, Petr Spacek wrote: > On 05/03/2012 11:25 AM, Petr Spacek wrote: >> Hello, >> >> this patch adds missing DNS->LDAP escaping conversion. It's necessary to >> prevent (potential) LDAP injection attacks in future. >> >> Code isn't very nice, because DNS users decimal escaping \123, LDAP uses >> hexadecimal escaping \ab and set of escaped characters is smaller in >> DNS than >> in LDAP. >> >> Any improvements are welcome. >> >> Petr^2 Spacek > > Here is second version of the patch. > > Changes: > - comments > - order of [._-] in if() > - function was renamed to dns_to_ldap_dn_escape() > > Escaping logic itself wasn't changed. Hello Peter, please check my comments inside the patch. Regards, Adam > > Petr^2 Spacek > > bind-dyndb-ldap-pspacek-0019-2-Add-proper-DN-escaping-before-LDAP-library-calls.patch > > > From 0291e1b63e077af06b84374c7aa0b15bd71aeb7d Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Mon, 23 Apr 2012 11:38:43 +0200 > Subject: [PATCH] Add proper DN escaping before LDAP library calls. > Signed-off-by: Petr Spacek > > --- > src/ldap_convert.c | 105 ++++++++++++++++++++++++++++++++++++++++++++++++-- > src/zone_register.c | 7 +++ > src/zone_register.h | 3 + > 3 files changed, 110 insertions(+), 5 deletions(-) > > diff --git a/src/ldap_convert.c b/src/ldap_convert.c > index 6405a98fda942cbba10b4c862351c4e3695aba85..1e7c31097b532a8641a34de589ef0926df6bf45f 100644 > --- a/src/ldap_convert.c > +++ b/src/ldap_convert.c > @@ -21,6 +21,7 @@ > #include > #include > #include > +#include > > #include > #include > @@ -32,6 +33,7 @@ > > #include > #include > +#include > > #include "str.h" > #include "ldap_convert.h" > @@ -189,6 +191,94 @@ cleanup: > return result; > } > > +/** > + * Convert a string from DNS escaping to LDAP escaping. > + * The Input string dns_str is expected to be the result of dns_name_tostring(). > + * The DNS label can contain any binary data as described in > + * http://tools.ietf.org/html/rfc2181#section-11 . > + * > + * DNS escaping uses form "\123" = ASCII value 123 (decimal) > + * LDAP escaping users form "\7b" = ASCII value 7b (hexadecimal) > + * > + * Input (DNS escaped) example : _aaa,bbb\255\000ccc.555.ddd-eee > + * Output (LDAP escaped) example: _aaa\2cbbb\ff\00ccc.555.ddd-eee > + * > + * The DNS to text functions from ISC libraries do not convert certain > + * characters (e.g. ","). This function converts \123 form to \7b form in all > + * cases. Other characters (not escaped by ISC libraries) will be additionally > + * converted to the LDAP escape form. > + * Input characters [a-zA-Z0-9._-] are left in raw ASCII form. > + * > + * If dns_str consists only of the characters in the [a-zA-Z0-9._-] set, it > + * will be checked& copied to the output buffer, without any additional escaping. > + */ > +isc_result_t > +dns_to_ldap_dn_escape(isc_mem_t *mctx, const char const * dns_str, char ** ldap_name) { > + isc_result_t result = ISC_R_FAILURE; > + char * esc_name = NULL; > + int idx_symb_first = -1; /* index of first "nice" printable symbol in dns_str */ > + int dns_idx = 0; > + int esc_idx = 0; > + > + REQUIRE(dns_str); Please use REQUIRE(dns_str != NULL); > + REQUIRE(ldap_name != NULL&& *ldap_name == NULL); > + > + int dns_str_len = strlen(dns_str); > + > + /** > + * In worst case each symbol from DNS dns_str will be represented > + * as "\xy" in ldap_name. (xy are hexadecimal digits) > + */ > + CHECKED_MEM_ALLOCATE(mctx, *ldap_name, 3*dns_str_len+1); Please put the space between number/variable and an operator ("3 * dns_str_len + 1" in this case). > + esc_name = *ldap_name; > + > + for (dns_idx = 0; dns_idx< dns_str_len; dns_idx++) { > + if (isalnum(dns_str[dns_idx]) || dns_str[dns_idx] == '.' > + || dns_str[dns_idx] == '-' || dns_str[dns_idx] == '_' ) { > + if (idx_symb_first == -1) > + idx_symb_first = dns_idx; > + continue; > + } else { /* some not very nice symbols */ > + int ascii_val; > + if (idx_symb_first != -1) { /* copy previous nice part */ > + int length_ok = dns_idx-idx_symb_first; ditto > + memcpy(esc_name + esc_idx, dns_str + idx_symb_first, length_ok); > + esc_idx += length_ok; > + idx_symb_first = -1; > + } > + if (dns_str[dns_idx] != '\\') { /* not nice raw value, e.g. ',' */ > + ascii_val = dns_str[dns_idx]; > + } else { /* not nice value in DNS \123 decimal format */ > + /* check if input length<= expected size */ > + if (dns_str_len<= dns_idx+3) { /* this should never happen */ ditto > + log_bug("dns string too short"); > + result = ISC_R_NOSPACE; > + goto cleanup; > + } Simple REQUIRE(dns_str_len <= dns_idx + 3); seems more appropriate here. > + ascii_val = 100*(dns_str[dns_idx+1]-'0') > + + 10*(dns_str[dns_idx+2]-'0') + (dns_str[dns_idx+3]-'0'); ditto > + dns_idx += 3; > + } > + /* LDAP uses \xy escaping. "xy" represent two hexadecimal digits.*/ > + /* TODO: optimize to bit mask& rotate& dec->hex table? */ > + CHECK(isc_string_printf(esc_name+esc_idx, 4, "\\%02x", ascii_val)); ditto > + esc_idx += 3; /* isc_string_printf wrote 4 bytes including '\0' */ > + } > + } > + if (idx_symb_first != -1) { /* copy last nice part */ > + int length_ok = dns_idx-idx_symb_first; > + memcpy(esc_name + esc_idx, dns_str + idx_symb_first, dns_idx-idx_symb_first); ditto > + esc_idx += length_ok; > + idx_symb_first = -1; > + } > + esc_name[esc_idx] = '\0'; > + return ISC_R_SUCCESS; > + > +cleanup: > + if (*ldap_name) isc_mem_free(mctx, *ldap_name); For one-line if () statements please use this format: if (cond) statement; > + return result; > +} > + > static isc_result_t > explode_dn(const char *dn, char ***explodedp, int notypes) > { > @@ -243,11 +333,15 @@ dnsname_to_dn(zone_register_t *zr, dns_name_t *name, ld_string_t *target) > isc_result_t result; > int label_count; > const char *zone_dn = NULL; > + char *dns_str = NULL; > + char *escaped_name = NULL; > > REQUIRE(zr != NULL); > REQUIRE(name != NULL); > REQUIRE(target != NULL); > > + isc_mem_t * mctx = zr_get_mctx(zr); > + > /* Find the DN of the zone we belong to. */ > { > DECLARE_BUFFERED_NAME(zone); > @@ -264,26 +358,26 @@ dnsname_to_dn(zone_register_t *zr, dns_name_t *name, ld_string_t *target) > > str_clear(target); > if (label_count> 0) { > - DECLARE_BUFFER(buffer, DNS_NAME_MAXTEXT); > dns_name_t labels; > > - INIT_BUFFER(buffer); > dns_name_init(&labels, NULL); > - > dns_name_getlabelsequence(name, 0, label_count,&labels); > - CHECK(dns_name_totext(&labels, ISC_TRUE,&buffer)); > + CHECK(dns_name_tostring(&labels,&dns_str, mctx)); > > + CHECK(dns_to_ldap_dn_escape(mctx, dns_str,&escaped_name)); > CHECK(str_cat_char(target, "idnsName=")); > - CHECK(str_cat_isc_buffer(target,&buffer)); > + CHECK(str_cat_char(target, escaped_name)); > /* > * Modification of following line can affect modify_ldap_common(). > * See line with: char *zone_dn = strstr(str_buf(owner_dn),", ") + 1; > */ > CHECK(str_cat_char(target, ", ")); > } > CHECK(str_cat_char(target, zone_dn)); > > cleanup: > + if (dns_str) isc_mem_free(mctx, dns_str); > + if (escaped_name) isc_mem_free(mctx, escaped_name); ditto (one-line if) > return result; > } > > @@ -328,3 +422,4 @@ rdatatype_to_ldap_attribute(dns_rdatatype_t rdtype, const char **target) > > return ISC_R_SUCCESS; > } > + > diff --git a/src/zone_register.c b/src/zone_register.c > index fc6dc076ac91ae2f21ecf934d0d6c837f1581766..81d208fc6e3c0dba6efb72ae10db54a79c336eca 100644 > --- a/src/zone_register.c > +++ b/src/zone_register.c > @@ -61,6 +61,13 @@ zr_get_rbt(zone_register_t *zr) > return zr->rbt; > } > > +isc_mem_t * > +zr_get_mctx(zone_register_t *zr) { > + REQUIRE(zr); > + > + return zr->mctx; > +} > + > /* > * Create a new zone register. > */ > diff --git a/src/zone_register.h b/src/zone_register.h > index e2408cbf8630effc0036c3765535a84381f83117..fa8ef255ef9cf212bca04aaafba0e6450d40a5c6 100644 > --- a/src/zone_register.h > +++ b/src/zone_register.h > @@ -45,4 +45,7 @@ zr_get_zone_ptr(zone_register_t *zr, dns_name_t *name, dns_zone_t **zonep); > dns_rbt_t * > zr_get_rbt(zone_register_t *zr); > > +isc_mem_t * > +zr_get_mctx(zone_register_t *zr); > + > #endif /* !_LD_ZONE_REGISTER_H_ */ From pspacek at redhat.com Wed May 9 12:11:09 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 09 May 2012 14:11:09 +0200 Subject: [Freeipa-devel] [PATCH 0019] Add proper DN escaping before LDAP library calls In-Reply-To: <4FAA53EA.1080803@redhat.com> References: <4FA24F17.2000106@redhat.com> <4FA28C43.6060408@redhat.com> <4FAA53EA.1080803@redhat.com> Message-ID: <4FAA5EDD.5020703@redhat.com> On 05/09/2012 01:24 PM, Adam Tkac wrote: > On 05/03/2012 03:46 PM, Petr Spacek wrote: >> On 05/03/2012 11:25 AM, Petr Spacek wrote: >>> Hello, >>> >>> this patch adds missing DNS->LDAP escaping conversion. It's necessary to >>> prevent (potential) LDAP injection attacks in future. >>> >>> Code isn't very nice, because DNS users decimal escaping \123, LDAP uses >>> hexadecimal escaping \ab and set of escaped characters is smaller in DNS than >>> in LDAP. >>> >>> Any improvements are welcome. >>> >>> Petr^2 Spacek >> >> Here is second version of the patch. >> >> Changes: >> - comments >> - order of [._-] in if() >> - function was renamed to dns_to_ldap_dn_escape() >> >> Escaping logic itself wasn't changed. > > Hello Peter, > > please check my comments inside the patch. Oh, I feel so ashamed. All errors were corrected, see attachment. Petr^2 Spacek > > Regards, Adam > >> >> Petr^2 Spacek >> >> bind-dyndb-ldap-pspacek-0019-2-Add-proper-DN-escaping-before-LDAP-library-calls.patch -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0019-3-Add-proper-DN-escaping-before-LDAP-library-calls.patch Type: text/x-patch Size: 6957 bytes Desc: not available URL: From atkac at redhat.com Wed May 9 12:17:01 2012 From: atkac at redhat.com (Adam Tkac) Date: Wed, 09 May 2012 14:17:01 +0200 Subject: [Freeipa-devel] [PATCH 0019] Add proper DN escaping before LDAP library calls In-Reply-To: <4FAA5EDD.5020703@redhat.com> References: <4FA24F17.2000106@redhat.com> <4FA28C43.6060408@redhat.com> <4FAA53EA.1080803@redhat.com> <4FAA5EDD.5020703@redhat.com> Message-ID: <4FAA603D.2010302@redhat.com> On 05/09/2012 02:11 PM, Petr Spacek wrote: > On 05/09/2012 01:24 PM, Adam Tkac wrote: >> On 05/03/2012 03:46 PM, Petr Spacek wrote: >>> On 05/03/2012 11:25 AM, Petr Spacek wrote: >>>> Hello, >>>> >>>> this patch adds missing DNS->LDAP escaping conversion. It's >>>> necessary to >>>> prevent (potential) LDAP injection attacks in future. >>>> >>>> Code isn't very nice, because DNS users decimal escaping \123, LDAP >>>> uses >>>> hexadecimal escaping \ab and set of escaped characters is smaller >>>> in DNS than >>>> in LDAP. >>>> >>>> Any improvements are welcome. >>>> >>>> Petr^2 Spacek >>> >>> Here is second version of the patch. >>> >>> Changes: >>> - comments >>> - order of [._-] in if() >>> - function was renamed to dns_to_ldap_dn_escape() >>> >>> Escaping logic itself wasn't changed. >> >> Hello Peter, >> >> please check my comments inside the patch. > Oh, I feel so ashamed. All errors were corrected, see attachment. > > Petr^2 Spacek Ack, please push it to master. > >> >> Regards, Adam >> >>> >>> Petr^2 Spacek >>> >>> bind-dyndb-ldap-pspacek-0019-2-Add-proper-DN-escaping-before-LDAP-library-calls.patch >>> > > bind-dyndb-ldap-pspacek-0019-3-Add-proper-DN-escaping-before-LDAP-library-calls.patch > > > From 1db133bf26a3c8bec7da3b045258bad630378d50 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Mon, 23 Apr 2012 11:38:43 +0200 > Subject: [PATCH] Add proper DN escaping before LDAP library calls. > Signed-off-by: Petr Spacek > > --- > src/ldap_convert.c | 109 ++++++++++++++++++++++++++++++++++++++++++++++++-- > src/zone_register.c | 7 +++ > src/zone_register.h | 3 + > 3 files changed, 114 insertions(+), 5 deletions(-) > > diff --git a/src/ldap_convert.c b/src/ldap_convert.c > index 6405a98fda942cbba10b4c862351c4e3695aba85..6cb761db97cff12b999263e2fcb4f31603ccd733 100644 > --- a/src/ldap_convert.c > +++ b/src/ldap_convert.c > @@ -21,6 +21,7 @@ > #include > #include > #include > +#include > > #include > #include > @@ -32,6 +33,7 @@ > > #include > #include > +#include > > #include "str.h" > #include "ldap_convert.h" > @@ -189,6 +191,96 @@ cleanup: > return result; > } > > +/** > + * Convert a string from DNS escaping to LDAP escaping. > + * The Input string dns_str is expected to be the result of dns_name_tostring(). > + * The DNS label can contain any binary data as described in > + * http://tools.ietf.org/html/rfc2181#section-11 . > + * > + * DNS escaping uses form "\123" = ASCII value 123 (decimal) > + * LDAP escaping users form "\7b" = ASCII value 7b (hexadecimal) > + * > + * Input (DNS escaped) example : _aaa,bbb\255\000ccc.555.ddd-eee > + * Output (LDAP escaped) example: _aaa\2cbbb\ff\00ccc.555.ddd-eee > + * > + * The DNS to text functions from ISC libraries do not convert certain > + * characters (e.g. ","). This function converts \123 form to \7b form in all > + * cases. Other characters (not escaped by ISC libraries) will be additionally > + * converted to the LDAP escape form. > + * Input characters [a-zA-Z0-9._-] are left in raw ASCII form. > + * > + * If dns_str consists only of the characters in the [a-zA-Z0-9._-] set, it > + * will be checked& copied to the output buffer, without any additional escaping. > + */ > +isc_result_t > +dns_to_ldap_dn_escape(isc_mem_t *mctx, const char const * dns_str, char ** ldap_name) { > + isc_result_t result = ISC_R_FAILURE; > + char * esc_name = NULL; > + int idx_symb_first = -1; /* index of first "nice" printable symbol in dns_str */ > + int dns_idx = 0; > + int esc_idx = 0; > + > + REQUIRE(dns_str != NULL); > + REQUIRE(ldap_name != NULL&& *ldap_name == NULL); > + > + int dns_str_len = strlen(dns_str); > + > + /** > + * In worst case each symbol from DNS dns_str will be represented > + * as "\xy" in ldap_name. (xy are hexadecimal digits) > + */ > + CHECKED_MEM_ALLOCATE(mctx, *ldap_name, 3 * dns_str_len + 1); > + esc_name = *ldap_name; > + > + for (dns_idx = 0; dns_idx< dns_str_len; dns_idx++) { > + if (isalnum(dns_str[dns_idx]) || dns_str[dns_idx] == '.' > + || dns_str[dns_idx] == '-' || dns_str[dns_idx] == '_' ) { > + if (idx_symb_first == -1) > + idx_symb_first = dns_idx; > + continue; > + } else { /* some not very nice symbols */ > + int ascii_val; > + if (idx_symb_first != -1) { /* copy previous nice part */ > + int length_ok = dns_idx - idx_symb_first; > + memcpy(esc_name + esc_idx, dns_str + idx_symb_first, length_ok); > + esc_idx += length_ok; > + idx_symb_first = -1; > + } > + if (dns_str[dns_idx] != '\\') { /* not nice raw value, e.g. ',' */ > + ascii_val = dns_str[dns_idx]; > + } else { /* not nice value in DNS \123 decimal format */ > + /* check if input length<= expected size */ > + if (dns_str_len<= dns_idx + 3) { /* this should never happen */ > + log_bug("dns string too short"); > + result = ISC_R_NOSPACE; > + goto cleanup; > + } > + ascii_val = 100 * (dns_str[dns_idx + 1] - '0') > + + 10 * (dns_str[dns_idx + 2] - '0') > + + (dns_str[dns_idx + 3] - '0'); > + dns_idx += 3; > + } > + /* LDAP uses \xy escaping. "xy" represent two hexadecimal digits.*/ > + /* TODO: optimize to bit mask& rotate& dec->hex table? */ > + CHECK(isc_string_printf(esc_name + esc_idx, 4, "\\%02x", ascii_val)); > + esc_idx += 3; /* isc_string_printf wrote 4 bytes including '\0' */ > + } > + } > + if (idx_symb_first != -1) { /* copy last nice part */ > + int length_ok = dns_idx - idx_symb_first; > + memcpy(esc_name + esc_idx, dns_str + idx_symb_first, dns_idx - idx_symb_first); > + esc_idx += length_ok; > + idx_symb_first = -1; > + } > + esc_name[esc_idx] = '\0'; > + return ISC_R_SUCCESS; > + > +cleanup: > + if (*ldap_name) > + isc_mem_free(mctx, *ldap_name); > + return result; > +} > + > static isc_result_t > explode_dn(const char *dn, char ***explodedp, int notypes) > { > @@ -243,11 +335,15 @@ dnsname_to_dn(zone_register_t *zr, dns_name_t *name, ld_string_t *target) > isc_result_t result; > int label_count; > const char *zone_dn = NULL; > + char *dns_str = NULL; > + char *escaped_name = NULL; > > REQUIRE(zr != NULL); > REQUIRE(name != NULL); > REQUIRE(target != NULL); > > + isc_mem_t * mctx = zr_get_mctx(zr); > + > /* Find the DN of the zone we belong to. */ > { > DECLARE_BUFFERED_NAME(zone); > @@ -264,26 +360,28 @@ dnsname_to_dn(zone_register_t *zr, dns_name_t *name, ld_string_t *target) > > str_clear(target); > if (label_count> 0) { > - DECLARE_BUFFER(buffer, DNS_NAME_MAXTEXT); > dns_name_t labels; > > - INIT_BUFFER(buffer); > dns_name_init(&labels, NULL); > - > dns_name_getlabelsequence(name, 0, label_count,&labels); > - CHECK(dns_name_totext(&labels, ISC_TRUE,&buffer)); > + CHECK(dns_name_tostring(&labels,&dns_str, mctx)); > > + CHECK(dns_to_ldap_dn_escape(mctx, dns_str,&escaped_name)); > CHECK(str_cat_char(target, "idnsName=")); > - CHECK(str_cat_isc_buffer(target,&buffer)); > + CHECK(str_cat_char(target, escaped_name)); > /* > * Modification of following line can affect modify_ldap_common(). > * See line with: char *zone_dn = strstr(str_buf(owner_dn),", ") + 1; > */ > CHECK(str_cat_char(target, ", ")); > } > CHECK(str_cat_char(target, zone_dn)); > > cleanup: > + if (dns_str) > + isc_mem_free(mctx, dns_str); > + if (escaped_name) > + isc_mem_free(mctx, escaped_name); > return result; > } > > @@ -328,3 +426,4 @@ rdatatype_to_ldap_attribute(dns_rdatatype_t rdtype, const char **target) > > return ISC_R_SUCCESS; > } > + > diff --git a/src/zone_register.c b/src/zone_register.c > index fc6dc076ac91ae2f21ecf934d0d6c837f1581766..81d208fc6e3c0dba6efb72ae10db54a79c336eca 100644 > --- a/src/zone_register.c > +++ b/src/zone_register.c > @@ -61,6 +61,13 @@ zr_get_rbt(zone_register_t *zr) > return zr->rbt; > } > > +isc_mem_t * > +zr_get_mctx(zone_register_t *zr) { > + REQUIRE(zr); > + > + return zr->mctx; > +} > + > /* > * Create a new zone register. > */ > diff --git a/src/zone_register.h b/src/zone_register.h > index e2408cbf8630effc0036c3765535a84381f83117..fa8ef255ef9cf212bca04aaafba0e6450d40a5c6 100644 > --- a/src/zone_register.h > +++ b/src/zone_register.h > @@ -45,4 +45,7 @@ zr_get_zone_ptr(zone_register_t *zr, dns_name_t *name, dns_zone_t **zonep); > dns_rbt_t * > zr_get_rbt(zone_register_t *zr); > > +isc_mem_t * > +zr_get_mctx(zone_register_t *zr); > + > #endif /* !_LD_ZONE_REGISTER_H_ */ From pspacek at redhat.com Wed May 9 12:37:16 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 09 May 2012 14:37:16 +0200 Subject: [Freeipa-devel] [PATCH 0019] Add proper DN escaping before LDAP library calls In-Reply-To: <4FAA603D.2010302@redhat.com> References: <4FA24F17.2000106@redhat.com> <4FA28C43.6060408@redhat.com> <4FAA53EA.1080803@redhat.com> <4FAA5EDD.5020703@redhat.com> <4FAA603D.2010302@redhat.com> Message-ID: <4FAA64FC.8090307@redhat.com> On 05/09/2012 02:17 PM, Adam Tkac wrote: > On 05/09/2012 02:11 PM, Petr Spacek wrote: >> On 05/09/2012 01:24 PM, Adam Tkac wrote: >>> On 05/03/2012 03:46 PM, Petr Spacek wrote: >>>> On 05/03/2012 11:25 AM, Petr Spacek wrote: >>>>> Hello, >>>>> >>>>> this patch adds missing DNS->LDAP escaping conversion. It's necessary to >>>>> prevent (potential) LDAP injection attacks in future. >>>>> >>>>> Code isn't very nice, because DNS users decimal escaping \123, LDAP uses >>>>> hexadecimal escaping \ab and set of escaped characters is smaller in DNS >>>>> than >>>>> in LDAP. >>>>> >>>>> Any improvements are welcome. >>>>> >>>>> Petr^2 Spacek >>>> >>>> Here is second version of the patch. >>>> >>>> Changes: >>>> - comments >>>> - order of [._-] in if() >>>> - function was renamed to dns_to_ldap_dn_escape() >>>> >>>> Escaping logic itself wasn't changed. >>> >>> Hello Peter, >>> >>> please check my comments inside the patch. >> Oh, I feel so ashamed. All errors were corrected, see attachment. >> >> Petr^2 Spacek > Ack, please push it to master. Pushed with minor change as discussed on IRC: log_error() was substituted by REQUIRE(). https://fedorahosted.org/bind-dyndb-ldap/changeset/3d43fd66aa68ef275855391a94e47e9d2f30309d Petr^2 Spacek >> >>> >>> Regards, Adam >>> >>>> >>>> Petr^2 Spacek >>>> >>>> bind-dyndb-ldap-pspacek-0019-2-Add-proper-DN-escaping-before-LDAP-library-calls.patch >>>> >> >> bind-dyndb-ldap-pspacek-0019-3-Add-proper-DN-escaping-before-LDAP-library-calls.patch >> -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0019-4-Add-proper-DN-escaping-before-LDAP-library-calls.patch Type: text/x-patch Size: 6872 bytes Desc: not available URL: From ohamada at redhat.com Wed May 9 12:42:23 2012 From: ohamada at redhat.com (Ondrej Hamada) Date: Wed, 09 May 2012 14:42:23 +0200 Subject: [Freeipa-devel] [PATCH] 257 Fix python Requires in Fedora 17 build In-Reply-To: <1336146344.2581.18.camel@balmora.brq.redhat.com> References: <1336146344.2581.18.camel@balmora.brq.redhat.com> Message-ID: <4FAA662F.7010106@redhat.com> On 05/04/2012 05:45 PM, Martin Kosek wrote: > This one actually took me some time to track it down (details are in a > patch description). To check the result, simply build freeipa on Fedora > 17 with "make rpms", install rpms on the machine and check Requires of > freeipa-admintools package: > > $ rpm -qR freeipa-admintools > > Before the patch, there was a requirement for "/bin/python" which > effectively blocked an update of python package until freeipa packages > were removed. > > With this patch, there should be a correct requirement for > "/usr/bin/python" and python updates will work again - yay. > > Our newest freeipa package on F-17 (2.2.0-1) is not affected, koji F-17 > build root may have a different $PATH which translates "python" to > "/usr/bin/python" and not "/bin/python". > > Martin > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel works as proposed, ACK -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- An HTML attachment was scrubbed... URL: From pviktori at redhat.com Wed May 9 14:49:46 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 09 May 2012 16:49:46 +0200 Subject: [Freeipa-devel] [PATCH] 0044 Validate externalhost (when added by --addattr/--setattr) In-Reply-To: <4FA3BC9C.2010309@redhat.com> References: <4F9E81E7.3030005@redhat.com> <4FA3BC9C.2010309@redhat.com> Message-ID: <4FAA840A.6070203@redhat.com> On 05/04/2012 01:25 PM, Ondrej Hamada wrote: > On 04/30/2012 02:13 PM, Petr Viktorin wrote: >> >> Change the externalhost attribute of hbacrule, netgroup >> and sudorule into a full-fledged Parameter, and attach >> a validator to it. >> >> RFC 1123 specifies that only [-a-z0-9] are allowed, but apparently >> Windows and some phones also use underscores in hostnames. >> So the new validator allows the underscore. >> >> >> https://fedorahosted.org/freeipa/ticket/2649 >> >> >> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > 1) Current validation of external hostnames does not require them to be > fully qualified, but you do. It's inconsistent. > > 2) one test case failed: > FAIL: Test adding an invalid external host to Sudo rule using > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest > self.test(*self.arg) > File "/home/ohamada/2649/tests/test_xmlrpc/test_sudorule_plugin.py", > line 500, in test_a_sudorule_mod_externalhost_invalid_addattr > "character") > AssertionError > Thanks. Attaching updated patch. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0044-02-Validate-externalhost-when-added-by-addattr-setattr.patch Type: text/x-patch Size: 9078 bytes Desc: not available URL: From mkosek at redhat.com Thu May 10 07:31:44 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 10 May 2012 09:31:44 +0200 Subject: [Freeipa-devel] [PATCH] 258 Remove LDAP limits from DNS service Message-ID: <1336635104.27129.7.camel@balmora.brq.redhat.com> bind-dyndb-ldap persistent search queries LDAP for all DNS records. The LDAP connection must have no size or time limits to work properly. This patch updates limits both for existing service principal on updated machine and for new service principals added as a part of DNS installation. https://fedorahosted.org/freeipa/ticket/2531 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-258-remove-ldap-limits-from-dns-service.patch Type: text/x-patch Size: 5412 bytes Desc: not available URL: From mkosek at redhat.com Thu May 10 09:02:25 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 10 May 2012 11:02:25 +0200 Subject: [Freeipa-devel] [PATCH] 259 Remove ipa-server-install LDAP update errors Message-ID: <1336640545.27129.12.camel@balmora.brq.redhat.com> LDAP addEntry method raises an exception when a parent entry of the entry being added does not exist. This may not be an error, for example NIS entries are only added when NIS is enabled and thus the NIS entry container exists. This patch adds an appropriate check so that we rather add a debug message to ipaupgrade.log instead of raising a user visible error. https://fedorahosted.org/freeipa/ticket/2743 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-259-remove-ipa-server-install-ldap-update-errors.patch Type: text/x-patch Size: 2226 bytes Desc: not available URL: From ohamada at redhat.com Thu May 10 10:05:45 2012 From: ohamada at redhat.com (Ondrej Hamada) Date: Thu, 10 May 2012 12:05:45 +0200 Subject: [Freeipa-devel] [PATCH] 0044 Validate externalhost (when added by --addattr/--setattr) In-Reply-To: <4FAA840A.6070203@redhat.com> References: <4F9E81E7.3030005@redhat.com> <4FA3BC9C.2010309@redhat.com> <4FAA840A.6070203@redhat.com> Message-ID: <4FAB92F9.3010505@redhat.com> On 05/09/2012 04:49 PM, Petr Viktorin wrote: > On 05/04/2012 01:25 PM, Ondrej Hamada wrote: >> On 04/30/2012 02:13 PM, Petr Viktorin wrote: >>> >>> Change the externalhost attribute of hbacrule, netgroup >>> and sudorule into a full-fledged Parameter, and attach >>> a validator to it. >>> >>> RFC 1123 specifies that only [-a-z0-9] are allowed, but apparently >>> Windows and some phones also use underscores in hostnames. >>> So the new validator allows the underscore. >>> >>> >>> https://fedorahosted.org/freeipa/ticket/2649 >>> >>> >>> >>> _______________________________________________ >>> Freeipa-devel mailing list >>> Freeipa-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/freeipa-devel >> 1) Current validation of external hostnames does not require them to be >> fully qualified, but you do. It's inconsistent. >> >> 2) one test case failed: >> FAIL: Test adding an invalid external host to Sudo rule using >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in >> runTest >> self.test(*self.arg) >> File "/home/ohamada/2649/tests/test_xmlrpc/test_sudorule_plugin.py", >> line 500, in test_a_sudorule_mod_externalhost_invalid_addattr >> "character") >> AssertionError >> > > Thanks. Attaching updated patch. > > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel Suggestion: you can use ipalib.utils.validate_hostname function with check_fqdn param set to False. Sorry for not mentioning it before. Otherwise ACK -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- An HTML attachment was scrubbed... URL: From pvoborni at redhat.com Thu May 10 10:29:32 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 10 May 2012 12:29:32 +0200 Subject: [Freeipa-devel] [PATCH] 136 Correction of nested search facets tab labels Message-ID: <4FAB988C.5020905@redhat.com> Nested search facets were using 'search' tab label instead of their nested entity name. This patch is fixing that regression. https://fedorahosted.org/freeipa/ticket/2744 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0136-Correction-of-nested-search-facets-tab-labels.patch Type: text/x-patch Size: 2037 bytes Desc: not available URL: From pviktori at redhat.com Thu May 10 11:07:22 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 10 May 2012 13:07:22 +0200 Subject: [Freeipa-devel] [PATCH] 0047 Do not use extra command options in ACI, permission, selfservice In-Reply-To: <1336392614.29911.13.camel@balmora.brq.redhat.com> References: <4F8E97F6.2070703@redhat.com> <1336392614.29911.13.camel@balmora.brq.redhat.com> Message-ID: <4FABA16A.10802@redhat.com> This is the second and likely the next-to-last part of disabling extra command options (after this it's just test fixes and turning the checking on). Part of the work for https://fedorahosted.org/freeipa/ticket/2509 -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0047-Do-not-use-extra-command-options-in-ACI-permission-s.patch Type: text/x-patch Size: 10166 bytes Desc: not available URL: From pviktori at redhat.com Thu May 10 11:39:12 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 10 May 2012 13:39:12 +0200 Subject: [Freeipa-devel] [PATCH] 0044 Validate externalhost (when added by --addattr/--setattr) In-Reply-To: <4FAB92F9.3010505@redhat.com> References: <4F9E81E7.3030005@redhat.com> <4FA3BC9C.2010309@redhat.com> <4FAA840A.6070203@redhat.com> <4FAB92F9.3010505@redhat.com> Message-ID: <4FABA8E0.2010105@redhat.com> On 05/10/2012 12:05 PM, Ondrej Hamada wrote: > On 05/09/2012 04:49 PM, Petr Viktorin wrote: >> On 05/04/2012 01:25 PM, Ondrej Hamada wrote: >>> On 04/30/2012 02:13 PM, Petr Viktorin wrote: >>>> >>>> Change the externalhost attribute of hbacrule, netgroup >>>> and sudorule into a full-fledged Parameter, and attach >>>> a validator to it. >>>> >>>> RFC 1123 specifies that only [-a-z0-9] are allowed, but apparently >>>> Windows and some phones also use underscores in hostnames. >>>> So the new validator allows the underscore. >>>> >>>> >>>> https://fedorahosted.org/freeipa/ticket/2649 >>>> >>>> >>>> >>>> _______________________________________________ >>>> Freeipa-devel mailing list >>>> Freeipa-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/freeipa-devel >>> 1) Current validation of external hostnames does not require them to be >>> fully qualified, but you do. It's inconsistent. >>> >>> 2) one test case failed: >>> FAIL: Test adding an invalid external host to Sudo rule using >>> ---------------------------------------------------------------------- >>> Traceback (most recent call last): >>> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in >>> runTest >>> self.test(*self.arg) >>> File "/home/ohamada/2649/tests/test_xmlrpc/test_sudorule_plugin.py", >>> line 500, in test_a_sudorule_mod_externalhost_invalid_addattr >>> "character") >>> AssertionError >>> >> >> Thanks. Attaching updated patch. >> >> >> >> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > Suggestion: you can use ipalib.utils.validate_hostname function with > check_fqdn param set to False. Sorry for not mentioning it before. > > Otherwise ACK > Attached patch uses your suggestion. Thanks. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0044-03-Validate-externalhost-when-added-by-addattr-setattr.patch Type: text/x-patch Size: 8790 bytes Desc: not available URL: From pviktori at redhat.com Thu May 10 11:40:19 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 10 May 2012 13:40:19 +0200 Subject: [Freeipa-devel] [PATCH] 0044 Validate externalhost (when added by --addattr/--setattr) In-Reply-To: <4FABA8E0.2010105@redhat.com> References: <4FABA8E0.2010105@redhat.com> Message-ID: <4FABA923.8010009@redhat.com> On 05/10/2012 12:05 PM, Ondrej Hamada wrote: > On 05/09/2012 04:49 PM, Petr Viktorin wrote: >> On 05/04/2012 01:25 PM, Ondrej Hamada wrote: >>> On 04/30/2012 02:13 PM, Petr Viktorin wrote: >>>> >>>> Change the externalhost attribute of hbacrule, netgroup >>>> and sudorule into a full-fledged Parameter, and attach >>>> a validator to it. >>>> >>>> RFC 1123 specifies that only [-a-z0-9] are allowed, but apparently >>>> Windows and some phones also use underscores in hostnames. >>>> So the new validator allows the underscore. >>>> >>>> >>>> https://fedorahosted.org/freeipa/ticket/2649 >>>> >>>> >>>> >>>> _______________________________________________ >>>> Freeipa-devel mailing list >>>> Freeipa-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/freeipa-devel >>> 1) Current validation of external hostnames does not require them to be >>> fully qualified, but you do. It's inconsistent. >>> >>> 2) one test case failed: >>> FAIL: Test adding an invalid external host to Sudo rule using >>> ---------------------------------------------------------------------- >>> Traceback (most recent call last): >>> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in >>> runTest >>> self.test(*self.arg) >>> File "/home/ohamada/2649/tests/test_xmlrpc/test_sudorule_plugin.py", >>> line 500, in test_a_sudorule_mod_externalhost_invalid_addattr >>> "character") >>> AssertionError >>> >> >> Thanks. Attaching updated patch. >> >> >> >> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > Suggestion: you can use ipalib.utils.validate_hostname function with > check_fqdn param set to False. Sorry for not mentioning it before. > > Otherwise ACK > Attached patch uses your suggestion. Thanks. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0044-03-Validate-externalhost-when-added-by-addattr-setattr.patch Type: text/x-patch Size: 8790 bytes Desc: not available URL: From pvoborni at redhat.com Thu May 10 12:19:05 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 10 May 2012 14:19:05 +0200 Subject: [Freeipa-devel] [PATCHES] 124-132 Inconsistent ways to show/change entry status In-Reply-To: <4FA883BA.8080803@redhat.com> References: <4FA137C4.7000101@redhat.com> <4FA285D0.90301@redhat.com> <4FA287A0.2070107@redhat.com> <4FA883BA.8080803@redhat.com> Message-ID: <4FABB239.2040500@redhat.com> Updated patch attached. See comments below. On 05/08/2012 04:23 AM, Endi Sukma Dewata wrote: > On 5/3/2012 8:26 AM, Petr Vobornik wrote: >> On 05/03/2012 03:19 PM, Petr Vobornik wrote: >>> I found that limitation of maximum pkey length in facet header is not >>> working well. Attaching patch #134 which actually calculates it. >> >> I found useless line in the patch. Corrected version attached. > > Try adding a very long DNS zone, then open the zone. Compare the > breadcrumbs in the DNS Resource Records page and in the Settings page, > in my case the second one is a bit longer. I think the length of the > breadcrumb should be calculated differently. I remade how breadcrumb is limited. Now it tries to use as much space available with keeping low complexity level of calculation. It still uses an average but now the average is calculated in two phases in order to neglect a presence of short keys. > > Btw, I just noticed that the facet tab label for DNS Resource Records is > changed to 'Search'. Same problem happens in Automount. > New ticket: #2744, patch on the list. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0134-2-Improved-calculation-of-max-pkey-length-in-facet-hea.patch Type: text/x-patch Size: 4988 bytes Desc: not available URL: From pviktori at redhat.com Thu May 10 12:20:28 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 10 May 2012 14:20:28 +0200 Subject: [Freeipa-devel] [PATCH] 0048 Rework the CallbackInterface Message-ID: <4FABB28C.90208@redhat.com> While investigating ticket 2674, I found several problems with our implementation of the CallbackInterface ?? it required complicated calling code, and would subtly break if command classes were instantiated in different ways than they are currently. Here's my fix. See commit message for details. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0048-Rework-the-CallbackInterface.patch Type: text/x-patch Size: 28596 bytes Desc: not available URL: From pviktori at redhat.com Thu May 10 13:19:50 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 10 May 2012 15:19:50 +0200 Subject: [Freeipa-devel] [RANT] --setattr validation is a minefield. In-Reply-To: <1334080383.2282.9.camel@priserak> References: <4F843D0F.5060906@redhat.com> <4F844BC4.2020109@redhat.com> <1334077635.16034.20.camel@balmora.brq.redhat.com> <4F846CF8.3020708@redhat.com> <1334080383.2282.9.camel@priserak> Message-ID: <4FABC076.7020308@redhat.com> On 04/10/2012 07:53 PM, Martin Kosek wrote: > On Tue, 2012-04-10 at 19:25 +0200, Petr Viktorin wrote: >> On 04/10/2012 07:07 PM, Martin Kosek wrote: >>> On Tue, 2012-04-10 at 17:03 +0200, Jan Cholasta wrote: >>>> On 10.4.2012 16:00, Petr Viktorin wrote: > [snip] >>>> Like you said above, we should either not validate --{set,add,del}attr >>>> or don't allow them on known attributes. >>> >>> IMHO, validating attributes we manage in the same way for both --setattr >>> and standard attrs is not that wrong. It is a good precaution, because >>> if we let an unvalidated value in, it can make even a bigger mess later >>> in our pre_callbacks or post_callbacks where we assume that at this >>> point everything is valid. >> >> Then we should validate *exactly* the same way, including not allowing >> no_update attributes to be updated. > > That makes some sense, I could agree with that. > Now that I have a ticket on this (https://fedorahosted.org/freeipa/ticket/2580), I would like to get some wider agreement here. The no_update/no_create attributes are mainly "enabled" flags (ipaenabledflag, nsaccountlock, idnszoneactive), administrative (krbprincipalname, ipauniqueid, ipacertificatesubjectbase), DNS record type and data, and various virtual attributes. If setattr etc. is disabled for all of these, it will mainly matter for the "enabled" flags. To be honest I don't know why we only allow modifying those through special commands. If there's some security reason for that, then setattr etc. should be disabled for them; otherwise I think they should be changeable through xyz-mod. Either way, setattr etc. should honor the no_update flags. Any objections? -- Petr? PS. For reference, our no_create/no_update params are: automember automemberdefaultgroup: no_create no_search no_update key: no_create no_search no_update automountkey description: no_create no_output no_search no_update config ipacertificatesubjectbase: no_update dnsrecord dnsrecords: no_create no_search no_update dnstype: no_create no_search no_update dnsdata: no_create no_search no_update a_extra_create_reverse: dnsrecord_extra no_update virtual_attribute aaaa_extra_create_reverse: dnsrecord_extra no_update virtual_attribute dnszone idnszoneactive: no_create no_update entitle uuid: no_create no_update ipaentitlementid: no_create no_update hbacrule ipaenabledflag: no_create no_search no_update memberuser_user: no_create no_search no_update memberuser_group: no_create no_search no_update memberhost_host: no_create no_search no_update memberhost_hostgroup: no_create no_search no_update sourcehost_host: no_create no_search no_update sourcehost_hostgroup: no_create no_search no_update memberservice_hbacsvc: no_create no_search no_update memberservice_hbacsvcgroup: no_create no_search no_update host randompassword: no_create no_search no_update virtual_attribute krbprincipalname: no_create no_search no_update sshpubkeyfp: no_create no_search no_update virtual_attribute netgroup ipauniqueid: no_create no_update selinuxusermap ipaenabledflag: no_create no_search no_update memberuser_user: no_create no_search no_update memberuser_group: no_create no_search no_update memberhost_host: no_create no_search no_update memberhost_hostgroup: no_create no_search no_update sudocmdgroup membercmd_sudocmd: no_create no_search no_update membercmd_sudocmdgroup: no_create no_search no_update sudorule ipaenabledflag: no_create no_search no_update memberuser_user: no_create no_search no_update memberuser_group: no_create no_search no_update memberhost_host: no_create no_search no_update memberhost_hostgroup: no_create no_search no_update memberallowcmd_sudocmd: no_create no_search no_update memberdenycmd_sudocmd: no_create no_search no_update memberallowcmd_sudocmdgroup: no_create no_search no_update memberdenycmd_sudocmdgroup: no_create no_search no_update ipasudorunas_user: no_create no_search no_update ipasudorunas_group: no_create no_search no_update ipasudoopt: no_create no_search no_update ipasudorunasgroup_group: no_create no_search no_update user krbprincipalname: no_update randompassword: no_create no_search no_update virtual_attribute nsaccountlock: no_create no_search no_update sshpubkeyfp: no_create no_search no_update virtual_attribute From rcritten at redhat.com Thu May 10 15:06:20 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 10 May 2012 11:06:20 -0400 Subject: [Freeipa-devel] [PATCH] 1010 Unblock referral mods in libipa_uuid Message-ID: <4FABD96C.5050100@redhat.com> The ipa_uuid plugin was blocking mods to referral objects due to the way it was retrieving the LDAP entry. It would retrieve the entry and if the result was non-zero, such as LDAP_REFERRAL, it would raise it as an error, short-circuiting the mod process. Instead check to see if we got a referral and if so, exit more gracefully. Testing information is in the associated BZ. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1010-referral.patch Type: text/x-diff Size: 1488 bytes Desc: not available URL: From ohamada at redhat.com Thu May 10 16:22:57 2012 From: ohamada at redhat.com (Ondrej Hamada) Date: Thu, 10 May 2012 18:22:57 +0200 Subject: [Freeipa-devel] [PATCH] 0044 Validate externalhost (when added by --addattr/--setattr) In-Reply-To: <4FABA923.8010009@redhat.com> References: <4FABA8E0.2010105@redhat.com> <4FABA923.8010009@redhat.com> Message-ID: <4FABEB61.7000607@redhat.com> On 05/10/2012 01:40 PM, Petr Viktorin wrote: > On 05/10/2012 12:05 PM, Ondrej Hamada wrote: >> On 05/09/2012 04:49 PM, Petr Viktorin wrote: >>> On 05/04/2012 01:25 PM, Ondrej Hamada wrote: >>>> On 04/30/2012 02:13 PM, Petr Viktorin wrote: >>>>> >>>>> Change the externalhost attribute of hbacrule, netgroup >>>>> and sudorule into a full-fledged Parameter, and attach >>>>> a validator to it. >>>>> >>>>> RFC 1123 specifies that only [-a-z0-9] are allowed, but apparently >>>>> Windows and some phones also use underscores in hostnames. >>>>> So the new validator allows the underscore. >>>>> >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/2649 >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Freeipa-devel mailing list >>>>> Freeipa-devel at redhat.com >>>>> https://www.redhat.com/mailman/listinfo/freeipa-devel >>>> 1) Current validation of external hostnames does not require them >>>> to be >>>> fully qualified, but you do. It's inconsistent. >>>> >>>> 2) one test case failed: >>>> FAIL: Test adding an invalid external host to Sudo rule using >>>> ---------------------------------------------------------------------- >>>> Traceback (most recent call last): >>>> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in >>>> runTest >>>> self.test(*self.arg) >>>> File "/home/ohamada/2649/tests/test_xmlrpc/test_sudorule_plugin.py", >>>> line 500, in test_a_sudorule_mod_externalhost_invalid_addattr >>>> "character") >>>> AssertionError >>>> >>> >>> Thanks. Attaching updated patch. >>> >>> >>> >>> >>> _______________________________________________ >>> Freeipa-devel mailing list >>> Freeipa-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/freeipa-devel >> Suggestion: you can use ipalib.utils.validate_hostname function with >> check_fqdn param set to False. Sorry for not mentioning it before. >> >> Otherwise ACK >> > > Attached patch uses your suggestion. Thanks. > > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel ACK -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- An HTML attachment was scrubbed... URL: From simo at redhat.com Thu May 10 16:23:18 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 10 May 2012 12:23:18 -0400 Subject: [Freeipa-devel] [PATCH] 1010 Unblock referral mods in libipa_uuid In-Reply-To: <4FABD96C.5050100@redhat.com> References: <4FABD96C.5050100@redhat.com> Message-ID: <1336666998.5722.223.camel@willson.li.ssimo.org> On Thu, 2012-05-10 at 11:06 -0400, Rob Crittenden wrote: > The ipa_uuid plugin was blocking mods to referral objects due to the > way > it was retrieving the LDAP entry. It would retrieve the entry and if > the > result was non-zero, such as LDAP_REFERRAL, it would raise it as an > error, short-circuiting the mod process. > > Instead check to see if we got a referral and if so, exit more > gracefully. > > Testing information is in the associated BZ. > ACK Simo. -- Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Fri May 11 06:36:34 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 11 May 2012 08:36:34 +0200 Subject: [Freeipa-devel] [PATCH] 0044 Validate externalhost (when added by --addattr/--setattr) In-Reply-To: <4FABEB61.7000607@redhat.com> References: <4FABA8E0.2010105@redhat.com> <4FABA923.8010009@redhat.com> <4FABEB61.7000607@redhat.com> Message-ID: <1336718194.3067.9.camel@priserak> On Thu, 2012-05-10 at 18:22 +0200, Ondrej Hamada wrote: > On 05/10/2012 01:40 PM, Petr Viktorin wrote: > > On 05/10/2012 12:05 PM, Ondrej Hamada wrote: > > > On 05/09/2012 04:49 PM, Petr Viktorin wrote: > > > > On 05/04/2012 01:25 PM, Ondrej Hamada wrote: > > > > > On 04/30/2012 02:13 PM, Petr Viktorin wrote: > > > > > > > > > > > > Change the externalhost attribute of hbacrule, netgroup > > > > > > and sudorule into a full-fledged Parameter, and attach > > > > > > a validator to it. > > > > > > > > > > > > RFC 1123 specifies that only [-a-z0-9] are allowed, but > > > > > > apparently > > > > > > Windows and some phones also use underscores in hostnames. > > > > > > So the new validator allows the underscore. > > > > > > > > > > > > > > > > > > https://fedorahosted.org/freeipa/ticket/2649 > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > Freeipa-devel mailing list > > > > > > Freeipa-devel at redhat.com > > > > > > https://www.redhat.com/mailman/listinfo/freeipa-devel > > > > > 1) Current validation of external hostnames does not require > > > > > them to be > > > > > fully qualified, but you do. It's inconsistent. > > > > > > > > > > 2) one test case failed: > > > > > FAIL: Test adding an invalid external host to Sudo rule using > > > > > ---------------------------------------------------------------------- > > > > > Traceback (most recent call last): > > > > > File "/usr/lib/python2.7/site-packages/nose/case.py", line > > > > > 197, in > > > > > runTest > > > > > self.test(*self.arg) > > > > > File > > > > > "/home/ohamada/2649/tests/test_xmlrpc/test_sudorule_plugin.py", > > > > > line 500, in test_a_sudorule_mod_externalhost_invalid_addattr > > > > > "character") > > > > > AssertionError > > > > > > > > > > > > > Thanks. Attaching updated patch. > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > Freeipa-devel mailing list > > > > Freeipa-devel at redhat.com > > > > https://www.redhat.com/mailman/listinfo/freeipa-devel > > > Suggestion: you can use ipalib.utils.validate_hostname function > > > with > > > check_fqdn param set to False. Sorry for not mentioning it > > > before. > > > > > > Otherwise ACK > > > > > > > Attached patch uses your suggestion. Thanks. > > > > > > > > > > _______________________________________________ > > Freeipa-devel mailing list > > Freeipa-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/freeipa-devel > ACK > Pushed to master. Martin From mkosek at redhat.com Fri May 11 06:41:18 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 11 May 2012 08:41:18 +0200 Subject: [Freeipa-devel] [PATCH] 1010 Unblock referral mods in libipa_uuid In-Reply-To: <1336666998.5722.223.camel@willson.li.ssimo.org> References: <4FABD96C.5050100@redhat.com> <1336666998.5722.223.camel@willson.li.ssimo.org> Message-ID: <1336718478.3067.10.camel@priserak> On Thu, 2012-05-10 at 12:23 -0400, Simo Sorce wrote: > On Thu, 2012-05-10 at 11:06 -0400, Rob Crittenden wrote: > > The ipa_uuid plugin was blocking mods to referral objects due to the > > way > > it was retrieving the LDAP entry. It would retrieve the entry and if > > the > > result was non-zero, such as LDAP_REFERRAL, it would raise it as an > > error, short-circuiting the mod process. > > > > Instead check to see if we got a referral and if so, exit more > > gracefully. > > > > Testing information is in the associated BZ. > > > ACK > > Simo. > Pushed to master. Martin From atkac at redhat.com Fri May 11 10:26:55 2012 From: atkac at redhat.com (Adam Tkac) Date: Fri, 11 May 2012 12:26:55 +0200 Subject: [Freeipa-devel] [PATCH 0020] Separate LDAP result from LDAP connection, fix deadlock. In-Reply-To: <4FA7C4C3.7020202@redhat.com> References: <4FA7C4C3.7020202@redhat.com> Message-ID: <20120511102654.GA2785@redhat.com> On Mon, May 07, 2012 at 02:49:07PM +0200, Petr Spacek wrote: > Hello, > > this patch fixes https://fedorahosted.org/bind-dyndb-ldap/ticket/66: > Plugin deadlocks during new zone load when connections == 1. > > It fixes structural problem, when LDAP query result was tied with > LDAP connection up. It wasn't possible to release connection and > work with query result after that. > Described deadlock is consequence of this problematic design. > > Now LDAP connection is separated from LDAP result. Next planed patch > will avoid "manual" connection management, so possibility of > deadlock should be next to zero. > > Petr^2 Spacek Hello Peter, good work, please check my comments below. Regards, Adam > From 8ee1fd607531ef71369e99c9228456baea45b65d Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Mon, 7 May 2012 12:51:09 +0200 > Subject: [PATCH] Separate LDAP result from LDAP connection, fix deadlock. > https://fedorahosted.org/bind-dyndb-ldap/ticket/66 > Signed-off-by: Petr Spacek > > --- > src/ldap_helper.c | 191 ++++++++++++++++++++++++++++++----------------------- > 1 files changed, 109 insertions(+), 82 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index 304c37296f8f3a428c4c72b45fe4154b2c9e954c..298d8e64cc9f6a0bafac6408f199276ac0e45f0d 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -108,6 +108,7 @@ > * must acquire the semaphore and the lock. > */ > > +typedef struct ldap_qresult ldap_qresult_t; > typedef struct ldap_connection ldap_connection_t; > typedef struct ldap_pool ldap_pool_t; > typedef struct ldap_auth_pair ldap_auth_pair_t; > @@ -188,28 +189,28 @@ struct ldap_connection { > ld_string_t *query_string; > > LDAP *handle; > - LDAPMessage *result; > LDAPControl *serverctrls[2]; /* psearch/NULL or NULL/NULL */ > int msgid; > > /* Parsing. */ > isc_lex_t *lex; > isc_buffer_t rdata_target; > unsigned char *rdata_target_mem; > > - /* Cache. */ > - ldap_entrylist_t ldap_entries; > - > /* For reconnection logic. */ > isc_time_t next_reconnect; > unsigned int tries; > +}; > > - /* Temporary stuff. */ > - LDAPMessage *entry; > - BerElement *ber; > - char *attribute; > - char **values; > - char *dn; > +/** > + * Result from single LDAP query. > + */ > +struct ldap_qresult { > + isc_mem_t *mctx; > + LDAPMessage *result; > + > + /* Cache. */ > + ldap_entrylist_t ldap_entries; > }; > > /* > @@ -271,9 +272,10 @@ static int handle_connection_error(ldap_instance_t *ldap_inst, > ldap_connection_t *ldap_conn, isc_boolean_t force, > isc_result_t *result); > static isc_result_t ldap_query(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn, > - const char *base, > - int scope, char **attrs, int attrsonly, const char *filter, ...); > -static void ldap_query_free(ldap_connection_t *ldap_conn); > + ldap_qresult_t **ldap_qresult, const char *base, int scope, char **attrs, When naming "**param" output parameters, convention is to append "p" (i.e. **ldap_qresultp) in this case. > + int attrsonly, const char *filter, ...); > +static void ldap_query_free(ldap_qresult_t **ldap_qresult); Ditto. > + > > /* Functions for writing to LDAP. */ > static isc_result_t ldap_modify_do(ldap_connection_t *ldap_conn, const char *dn, > @@ -1095,6 +1097,8 @@ refresh_zones_from_ldap(ldap_instance_t *ldap_inst) > { > isc_result_t result = ISC_R_SUCCESS; > ldap_connection_t *ldap_conn = NULL; > + ldap_qresult_t *ldap_config_qresult = NULL; > + ldap_qresult_t *ldap_zones_qresult = NULL; > int zone_count = 0; > ldap_entry_t *entry; > dns_rbt_t *rbt = NULL; > @@ -1119,31 +1123,31 @@ refresh_zones_from_ldap(ldap_instance_t *ldap_inst) > > log_debug(2, "refreshing list of zones for %s", ldap_inst->db_name); > > + /* Query configuration and zones from LDAP and release LDAP connection I think "Query for configuration..." is more appropriate here. > + * before processing them. It prevents deadlock in situations where > + * ldap_parse_zoneentry() requests another connection. */ > CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); > - > - CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(ldap_inst->base), > + CHECK(ldap_query(ldap_inst, ldap_conn, &ldap_config_qresult, str_buf(ldap_inst->base), > LDAP_SCOPE_SUBTREE, config_attrs, 0, > "(objectClass=idnsConfigObject)")); > - > - for (entry = HEAD(ldap_conn->ldap_entries); > - entry != NULL; > - entry = NEXT(entry, link)) { > - CHECK(ldap_parse_configentry(entry, ldap_inst)); > - } > - > - ldap_query_free(ldap_conn); > - > - CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(ldap_inst->base), > + CHECK(ldap_query(ldap_inst, ldap_conn, &ldap_zones_qresult, str_buf(ldap_inst->base), > LDAP_SCOPE_SUBTREE, zone_attrs, 0, > "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); > + ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > + > + for (entry = HEAD(ldap_config_qresult->ldap_entries); > + entry != NULL; > + entry = NEXT(entry, link)) { > + CHECK(ldap_parse_configentry(entry, ldap_inst)); > + } > > /* > * Create RB-tree with all zones stored in LDAP for cross check > * with registered zones in plugin. > */ > CHECK(dns_rbt_create(ldap_inst->mctx, NULL, NULL, &rbt)); > > - for (entry = HEAD(ldap_conn->ldap_entries); > + for (entry = HEAD(ldap_zones_qresult->ldap_entries); > entry != NULL; > entry = NEXT(entry, link)) { > > @@ -1232,6 +1236,8 @@ cleanup: > if (invalidate_nodechain) > dns_rbtnodechain_invalidate(&chain); > > + ldap_query_free(&ldap_config_qresult); > + ldap_query_free(&ldap_zones_qresult); > ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > > log_debug(2, "finished refreshing list of zones"); > @@ -1387,6 +1393,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > { > isc_result_t result; > ldap_connection_t *ldap_conn = NULL; > + ldap_qresult_t *ldap_qresult = NULL; > ldap_entry_t *entry; > ld_string_t *string = NULL; > ldapdb_node_t *node; > @@ -1403,15 +1410,15 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > CHECK(str_new(mctx, &string)); > CHECK(dnsname_to_dn(ldap_inst->zone_register, name, string)); > > - CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(string), > + CHECK(ldap_query(ldap_inst, ldap_conn, &ldap_qresult, str_buf(string), > LDAP_SCOPE_SUBTREE, NULL, 0, "(objectClass=idnsRecord)")); > > - if (EMPTY(ldap_conn->ldap_entries)) { > + if (EMPTY(ldap_qresult->ldap_entries)) { > result = ISC_R_NOTFOUND; > goto cleanup; > } > > - for (entry = HEAD(ldap_conn->ldap_entries); > + for (entry = HEAD(ldap_qresult->ldap_entries); > entry != NULL; > entry = NEXT(entry, link)) { > node = NULL; > @@ -1444,6 +1451,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > result = ISC_R_SUCCESS; > > cleanup: > + ldap_query_free(&ldap_qresult); > ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > str_destroy(&string); > > @@ -1456,6 +1464,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > { > isc_result_t result; > ldap_connection_t *ldap_conn = NULL; > + ldap_qresult_t *ldap_qresult = NULL; > ldap_entry_t *entry; > ld_string_t *string = NULL; > ldap_cache_t *cache; > @@ -1478,15 +1487,15 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > CHECK(dnsname_to_dn(ldap_inst->zone_register, name, string)); > > CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); > - CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(string), > + CHECK(ldap_query(ldap_inst, ldap_conn, &ldap_qresult, str_buf(string), > LDAP_SCOPE_BASE, NULL, 0, "(objectClass=idnsRecord)")); > > - if (EMPTY(ldap_conn->ldap_entries)) { > + if (EMPTY(ldap_qresult->ldap_entries)) { > result = ISC_R_NOTFOUND; > goto cleanup; > } > > - for (entry = HEAD(ldap_conn->ldap_entries); > + for (entry = HEAD(ldap_qresult->ldap_entries); > entry != NULL; > entry = NEXT(entry, link)) { > if (ldap_parse_rrentry(mctx, entry, ldap_conn, > @@ -1504,6 +1513,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > result = ISC_R_NOTFOUND; > > cleanup: > + ldap_query_free(&ldap_qresult); > ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > str_destroy(&string); > > @@ -1601,16 +1611,23 @@ cleanup: > return result; > } > > +/** > + * @param ldap_conn A LDAP connection structure obtained via ldap_get_connection(). > + * @param ldap_qresult New ldap_qresult structure will be allocated and pointer > + * to it will be returned through this parameter. The result > + * has to be freed by caller via ldap_query_free(). > + */ > static isc_result_t > ldap_query(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn, > - const char *base, int scope, char **attrs, > + ldap_qresult_t **ldap_qresult, const char *base, int scope, char **attrs, s/ldap_qresult/ldap_qresultp/ > int attrsonly, const char *filter, ...) > { > va_list ap; > isc_result_t result; > int cnt; > > REQUIRE(ldap_conn != NULL); > + REQUIRE(ldap_qresult != NULL && *ldap_qresult == NULL); > > va_start(ap, filter); > str_vsprintf(ldap_conn->query_string, filter, ap); > @@ -1629,62 +1646,59 @@ ldap_query(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn, > return result; > } > > + CHECKED_MEM_GET_PTR(ldap_inst->mctx, *ldap_qresult); > + (*ldap_qresult)->mctx = ldap_inst->mctx; > + INIT_LIST((*ldap_qresult)->ldap_entries); This is not so good. When ldap_query() doesn't return success, it shouldn't modify it's **ldap_qresultp parameter, as other functions do. Good way how to handle this situations is (pseudocode): function(**outputp) { *output; REQUIRE(outputp != NULL && *outputp == NULL); output = mem_get(); modify_output(output); if (some_failure) goto cleanup; *outputp = output; return success; cleanup: mem_put(output); return failure; } > + > do { > int ret; > > ret = ldap_search_ext_s(ldap_conn->handle, base, scope, > str_buf(ldap_conn->query_string), > attrs, attrsonly, NULL, NULL, NULL, > - LDAP_NO_LIMIT, &ldap_conn->result); > + LDAP_NO_LIMIT, &((*ldap_qresult)->result)); > if (ret == 0) { > ldap_conn->tries = 0; > - cnt = ldap_count_entries(ldap_conn->handle, ldap_conn->result); > + cnt = ldap_count_entries(ldap_conn->handle, (*ldap_qresult)->result); > log_debug(2, "entry count: %d", cnt); > > result = ldap_entrylist_create(ldap_conn->mctx, > ldap_conn->handle, > - ldap_conn->result, > - &ldap_conn->ldap_entries); > + (*ldap_qresult)->result, > + &(*ldap_qresult)->ldap_entries); > if (result != ISC_R_SUCCESS) { > log_error("failed to save LDAP query results"); > return result; > } > > return ISC_R_SUCCESS; > } > } while (handle_connection_error(ldap_inst, ldap_conn, ISC_FALSE, &result)); > > +cleanup: Previously allocated ldap_qresult should be free()-ed here. > return result; > } > > static void > -ldap_query_free(ldap_connection_t *ldap_conn) > +ldap_query_free(ldap_qresult_t **ldap_qresult) > { > - if (ldap_conn == NULL) > + ldap_qresult_t *qresult; > + REQUIRE(ldap_qresult != NULL); > + > + qresult = *ldap_qresult; > + > + if (qresult == NULL) > return; > > - if (ldap_conn->dn) { > - ldap_memfree(ldap_conn->dn); > - ldap_conn->dn = NULL; > - } > - if (ldap_conn->values) { > - ldap_value_free(ldap_conn->values); > - ldap_conn->values = NULL; > - } > - if (ldap_conn->attribute) { > - ldap_memfree(ldap_conn->attribute); > - ldap_conn->attribute = NULL; > - } > - if (ldap_conn->ber) { > - ber_free(ldap_conn->ber, 0); > - ldap_conn->ber = NULL; > - } > - if (ldap_conn->result) { > - ldap_msgfree(ldap_conn->result); > - ldap_conn->result = NULL; > + if (qresult->result) { > + ldap_msgfree(qresult->result); > + qresult->result = NULL; > } > > - ldap_entrylist_destroy(ldap_conn->mctx, &ldap_conn->ldap_entries); > + ldap_entrylist_destroy(qresult->mctx, &qresult->ldap_entries); > + > + SAFE_MEM_PUT_PTR(qresult->mctx, qresult); > + *ldap_qresult = NULL; > } > > /* FIXME: Tested with SASL/GSSAPI/KRB5 only */ > @@ -2229,6 +2243,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > isc_result_t result; > isc_mem_t *mctx = ldap_inst->mctx; > ldap_connection_t *ldap_conn = NULL; > + ldap_qresult_t *ldap_qresult = NULL; > ld_string_t *owner_dn = NULL; > LDAPMod *change[3] = { NULL }; > LDAPMod *change_ptr = NULL; > @@ -2255,12 +2270,12 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > } > > CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); > - CHECK(ldap_query(ldap_inst, ldap_conn, zone_dn, > + CHECK(ldap_query(ldap_inst, ldap_conn, &ldap_qresult, zone_dn, > LDAP_SCOPE_BASE, attrs, 0, > "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); > > /* only 0 or 1 active zone can be returned from query */ > - entry = HEAD(ldap_conn->ldap_entries); > + entry = HEAD(ldap_qresult->ldap_entries); > if (entry == NULL) { > log_debug(3, "Active zone %s not found", zone_dn); > result = ISC_R_NOTFOUND; > @@ -2392,14 +2407,14 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > char *owner_zone_dn_ptr = strstr(str_buf(owner_dn_ptr),", ") + 1; > > /* Get attribute "idnsAllowDynUpdate" for reverse zone or use default. */ > - ldap_query_free(ldap_conn); > + ldap_query_free(&ldap_qresult); > zone_dyn_update = ldap_inst->dyn_update; > - CHECK(ldap_query(ldap_inst, ldap_conn, owner_zone_dn_ptr, > + CHECK(ldap_query(ldap_inst, ldap_conn, &ldap_qresult, owner_zone_dn_ptr, > LDAP_SCOPE_BASE, attrs, 0, > "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); > > /* Only 0 or 1 active zone can be returned from query. */ > - entry = HEAD(ldap_conn->ldap_entries); > + entry = HEAD(ldap_qresult->ldap_entries); > if (entry == NULL) { > log_debug(3, "Active zone %s not found", zone_dn); > result = ISC_R_NOTFOUND; > @@ -2485,6 +2500,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > } > > cleanup: > + ldap_query_free(&ldap_qresult); > ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > str_destroy(&owner_dn_ptr); > str_destroy(&owner_dn); > @@ -2587,7 +2603,6 @@ ldap_pool_getconnection(ldap_pool_t *pool, ldap_connection_t ** conn) > > RUNTIME_CHECK(ldap_conn != NULL); > > - INIT_LIST(ldap_conn->ldap_entries); > *conn = ldap_conn; > > cleanup: > @@ -2607,7 +2622,6 @@ ldap_pool_putconnection(ldap_pool_t *pool, ldap_connection_t **conn) > if (ldap_conn == NULL) > return; > > - ldap_query_free(ldap_conn); > UNLOCK(&ldap_conn->lock); > semaphore_signal(&pool->conn_semaphore); > > @@ -2739,6 +2753,7 @@ update_action(isc_task_t *task, isc_event_t *event) > isc_result_t result ; > ldap_instance_t *inst = NULL; > ldap_connection_t *conn = NULL; > + ldap_qresult_t *ldap_qresult = NULL; > ldap_entry_t *entry; > isc_boolean_t delete = ISC_TRUE; > isc_mem_t *mctx; > @@ -2757,11 +2772,11 @@ update_action(isc_task_t *task, isc_event_t *event) > > CHECK(ldap_pool_getconnection(inst->pool, &conn)); > > - CHECK(ldap_query(inst, conn, pevent->dn, > + CHECK(ldap_query(inst, conn, &ldap_qresult, pevent->dn, > LDAP_SCOPE_BASE, attrs, 0, > "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); > > - for (entry = HEAD(conn->ldap_entries); > + for (entry = HEAD(ldap_qresult->ldap_entries); > entry != NULL; > entry = NEXT(entry, link)) { > delete = ISC_FALSE; > @@ -2779,6 +2794,7 @@ cleanup: > "Zones can be outdated, run `rndc reload`", > pevent->dn); > > + ldap_query_free(&ldap_qresult); > ldap_pool_putconnection(inst->pool, &conn); > isc_mem_free(mctx, pevent->dbname); > isc_mem_free(mctx, pevent->dn); > @@ -2793,6 +2809,7 @@ update_config(isc_task_t *task, isc_event_t *event) > isc_result_t result ; > ldap_instance_t *inst = NULL; > ldap_connection_t *conn = NULL; > + ldap_qresult_t *ldap_qresult = NULL; > ldap_entry_t *entry; > isc_mem_t *mctx; > char *attrs[] = { > @@ -2810,14 +2827,14 @@ update_config(isc_task_t *task, isc_event_t *event) > > CHECK(ldap_pool_getconnection(inst->pool, &conn)); > > - CHECK(ldap_query(inst, conn, pevent->dn, > + CHECK(ldap_query(inst, conn, &ldap_qresult, pevent->dn, > LDAP_SCOPE_BASE, attrs, 0, > "(objectClass=idnsConfigObject)")); > > - if (EMPTY(conn->ldap_entries)) > - log_error("Config object can not be empty"); > + if (EMPTY(ldap_qresult->ldap_entries)) > + log_error("Config object can not be empty"); // WHY? > > - for (entry = HEAD(conn->ldap_entries); > + for (entry = HEAD(ldap_qresult->ldap_entries); > entry != NULL; > entry = NEXT(entry, link)) { > result = ldap_parse_configentry(entry, inst); > @@ -2832,6 +2849,7 @@ cleanup: > "Configuration can be outdated, run `rndc reload`", > pevent->dn); > > + ldap_query_free(&ldap_qresult); > ldap_pool_putconnection(inst->pool, &conn); > isc_mem_free(mctx, pevent->dbname); > isc_mem_free(mctx, pevent->dn); > @@ -3071,6 +3089,7 @@ ldap_psearch_watcher(isc_threadarg_t arg) > { > ldap_instance_t *inst = (ldap_instance_t *)arg; > ldap_connection_t *conn; > + ldap_qresult_t *ldap_qresult = NULL; > struct timeval tv; > int ret, cnt; > isc_result_t result; > @@ -3108,6 +3127,11 @@ ldap_psearch_watcher(isc_threadarg_t arg) > ldap_connect(inst, conn, ISC_TRUE); > } > > + CHECKED_MEM_GET_PTR(conn->mctx, ldap_qresult); > + ldap_qresult->mctx = conn->mctx; > + ldap_qresult->result = NULL; > + INIT_LIST(ldap_qresult->ldap_entries); This piece of code is on two places. What about ldap_query_create(ldap_qresult_t **qresultp) function? It will be better because we can add/remove members of the ldap_qresult_t structure in the future. > + > restart: > /* Perform initial lookup */ > if (inst->psearch) { > @@ -3133,8 +3157,9 @@ restart: > } > > while (!inst->exiting) { > + > ret = ldap_result(conn->handle, conn->msgid, 0, &tv, > - &conn->result); > + &ldap_qresult->result); > > if (ret <= 0) { > int ok; > @@ -3158,14 +3183,14 @@ restart: > } > > conn->tries = 0; > - cnt = ldap_count_entries(conn->handle, conn->result); > + cnt = ldap_count_entries(conn->handle, ldap_qresult->result); > > if (cnt > 0) { > log_debug(3, "Got psearch updates (%d)", cnt); > - result = ldap_entrylist_append(conn->mctx, > + result = ldap_entrylist_append(ldap_qresult->mctx, > conn->handle, > - conn->result, > - &conn->ldap_entries); > + ldap_qresult->result, > + &ldap_qresult->ldap_entries); > if (result != ISC_R_SUCCESS) { > /* > * Error means inconsistency of our zones > @@ -3177,7 +3202,7 @@ restart: > } > > ldap_entry_t *entry; > - for (entry = HEAD(conn->ldap_entries); > + for (entry = HEAD(ldap_qresult->ldap_entries); > entry != NULL; > entry = NEXT(entry, link)) { > LDAPControl **ctrls = NULL; > @@ -3195,16 +3220,18 @@ restart: > psearch_update(inst, entry, ctrls); > } > soft_err: > - > - ldap_msgfree(conn->result); > - ldap_entrylist_destroy(conn->mctx, > - &conn->ldap_entries); > + ; Empty label "soft_err:" is useless, please remove it and use "continue;" on appropriate places; > } > + ldap_msgfree(ldap_qresult->result); > + ldap_qresult->result = NULL; > + ldap_entrylist_destroy(ldap_qresult->mctx, &ldap_qresult->ldap_entries); > + INIT_LIST(ldap_qresult->ldap_entries); I would prefer to add another parameter to the ldap_query_free() which will indicate if the ldap_qresult() should be free()-ed or just results should be cleared. Reason is same as above - ldap_qresult_t can be changed and we will have to change both ldap_query_free() and this piece of code. > } > > log_debug(1, "Ending ldap_psearch_watcher"); > > cleanup: > + ldap_query_free(&ldap_qresult); > ldap_pool_putconnection(inst->pool, &conn); > > return (isc_threadresult_t)0; > -- > 1.7.7.6 > -- Adam Tkac, Red Hat, Inc. From pvoborni at redhat.com Fri May 11 11:37:54 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 11 May 2012 13:37:54 +0200 Subject: [Freeipa-devel] [PATCH] 137 Instructions to generate cert use certutil instead of openssl Message-ID: <4FACFA12.80402@redhat.com> Instructions to generate certificate were changed. Now they use certutil instead of openssl. In the example is also used option for specifying key size. https://fedorahosted.org/freeipa/ticket/2725 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0137-Instructions-to-generate-cert-use-certutil-instead-o.patch Type: text/x-patch Size: 4293 bytes Desc: not available URL: From pviktori at redhat.com Fri May 11 14:55:59 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 11 May 2012 16:55:59 +0200 Subject: [Freeipa-devel] [PATCH] 0049 Disallow '<' and non-ASCII characters in the DM password Message-ID: <4FAD287F.8000000@redhat.com> https://fedorahosted.org/freeipa/ticket/2675 I've tested all ASCII non-alphanumeric characters that weren't blocked already. With all except for '<' I've succeeded. Non-ASCII characters also don't work in passwords. (Not that it'd be a good idea to use those.) -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0049-Disallow-and-non-ASCII-characters-in-the-DM-password.patch Type: text/x-patch Size: 1757 bytes Desc: not available URL: From mkosek at redhat.com Fri May 11 15:10:23 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 11 May 2012 17:10:23 +0200 Subject: [Freeipa-devel] [PATCH] 259 Remove ipa-server-install LDAP update errors In-Reply-To: <1336640545.27129.12.camel@balmora.brq.redhat.com> References: <1336640545.27129.12.camel@balmora.brq.redhat.com> Message-ID: <1336749023.3067.13.camel@priserak> On Thu, 2012-05-10 at 11:02 +0200, Martin Kosek wrote: > LDAP addEntry method raises an exception when a parent entry of > the entry being added does not exist. This may not be an error, > for example NIS entries are only added when NIS is enabled and > thus the NIS entry container exists. > > This patch adds an appropriate check so that we rather add > a debug message to ipaupgrade.log instead of raising a user > visible error. > > https://fedorahosted.org/freeipa/ticket/2743 > I got inspired in ticket #2520 and prepared a better solution which fixes both the incorrect exception processing in ipaldap + handles gracefully the missing parent entry situation without emitting extra LDAP query. Patch is attached. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-259-2-remove-ipa-server-install-ldap-update-errors.patch Type: text/x-patch Size: 2862 bytes Desc: not available URL: From pvoborni at redhat.com Fri May 11 16:36:21 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 11 May 2012 18:36:21 +0200 Subject: [Freeipa-devel] [PATCHES] 124-132 Inconsistent ways to show/change entry status In-Reply-To: <4FA85F14.40705@redhat.com> References: <4FA137C4.7000101@redhat.com> <4FA85F14.40705@redhat.com> Message-ID: <4FAD4005.2060807@redhat.com> On 05/08/2012 01:47 AM, Endi Sukma Dewata wrote: > The code works, it's ACKed. Patches 124-132 pushed to master. > I have some comments below: I'll address some points in separate patches. > > On 5/2/2012 8:33 AM, Petr Vobornik wrote: >> This bunch of patches are implementing ticket #2247. They introduce some >> new logic and types of internal objects. There might be design issues >> (mainly in state evaluation). I would appreciate some opinions on what >> might be improved. >> >> See patch comments for more details. >> >> What I think might be the main concerns: >> >> Action list definition: >> Now action lists are defined on facet level and facet header takes them >> from facet. Would in be better to define action list - the widget and >> actions separately. The widget could be defined in header and actions on >> facet level. > > Yes, the header could be considered a widget, so it contain the action > list widget. > >> State evaluation: >> The patches are adding support for some kind of state evaluation. The >> state is represented by array of string. I'm thinking that the design >> might not be robust. Is a string good enough? We might have a problem >> with conditions like to "have this particular access right' (#2318). >> There are state_evaluators and state_listeners. They are practically the >> same but the execution point and parameters are different. Should they >> be somehow joined? > > It might be better to use a map to represent the state. The map could > hold any attribute types, not just string flag with the current array. > So, the widget itself could be the state because it's also a map. That > said, evaluating a map might be difficult to define declaratively, we > would end up using a code again. So for now using an array of string is > fine, but we just have to know the limitations. > >> Code placement: >> There's a lot of new objects and for some of them it is not clear to >> what code file they should be placed. > > This will continue to change as we add more code. Feel free to create a > new file when you see a pattern. We might want to start separating the > IPA-specific code from the framework (reusable code). The framework > could be moved into a 'lib' folder. > >> FYI: In close future I would like to address some problems in UI >> architecture. I'm in a middle of designing phase, so there is nothing to >> present at the moment. The main topics are: >> * reduce the need of overriding methods when a new widgets or >> capabilities are added >> * make it more declarative to enhance extendibility >> It may be done by: >> * better inheriting model to support events >> * build phases (preinit, init, postinit, create, load) to improve spec >> object creating and initialization of created object. >> * path representation of an event/attribute/model property to support >> bindings to various events/attrs from anywhere >> * introduce model and model bindings, converters between command >> output/model/human readable representation > > Sounds good! Some comments about the patch: > > 1. When you open a specific user, there's a default action that's > already selected in the action list. The thing is the current default > action (i.e. Disable) is probably not something that's regularly used. > It might be better to show something like '-- Select Action --' or '-- > Action List --' so you'd have to select an action first then click Apply. > > 2. Suppose we did #1, do we still need the confirmation when applying > the action? We don't. > > 3. I think the action list should be available in all details page for > all entities. At least it should have the Delete action. #1,#2,#3: Sounds reasonable, will do. Delete button can be also created as control button and we have ticket for it. I'll do it as action in action list, though. > > 4. Something to think about, do we need an action list in the > association page? Entities like group default to an association page > instead of details page. So if we don't have an action list in the > association page the UI may look inconsistent because you'd have to go > to the Settings to execute an action. But if we do have an action list > in the association page, the actions could become confusing: do they > apply to the associations or to the entry itself? One possible solution > is to move the Settings to the left-most position and make it the > default page and have the action list in the details page only. I don't think there is an optimal solution. For now I prefer current implementation. I agree with your thoughts. Reasons why would I leave it as is: * if association facet is the first facet it usually means that the details facet is less important and so the actions are not so important (frequently used) to be offered in assoc. facet. User can always switch to settings facet. Nice example is dnszone entity or other nested facets. * I would show action list in association facets only for actions related to associations even though the association is related to the object. > > 5. Some details pages have status indicator (check sign or minus sign) > on the left of the page title, some others don't. This is because not > all entities have a concept of status. Would it be better to show the > status as a read-only field in the facet content? I just want to show read only field as an addition to the icon or to replace the icon? I have nothing against showing it as an addition. > > 6. It might be better to avoid using element ID in the CSS. This would > make the CSS more reusable. > > #content .facet-action-list div[name=apply] a.ui-state-default { > ... > } I'll try to avoid it. I'm also thinking about using OOCSS principals. They seems very useful for the kind of application which IPA Web UI is. > > 7. In some facet class definitions the no_init parameter is defined > separately from the spec object, any particular reason? > > IPA.facet = function(spec, no_init) { > ... > }; > > You can think of spec object as named parameters. So the no_init can be > defined in the spec object and later used to determine what operations > to be done inside the init method. It is to suppress initialization of ancestor classed before the process of inheriting is completed. If I would alter the spec it one class I would have to remember the state before calling ancestor constructor because offspring classes wouldn't be able to suppress this class init method. Example where current class wants to disable init method in ancestors builder function: var no_init = spec.no_init; //remember offspring will spec.no_init = true; //disable init in ancestor - IPA.facet var that = IPA.facet(spec); if (!no_init) that.init(); //my init Current implementation seemed easier. I already have a concept which can replace this. http://pvoborni.fedorapeople.org/patches/wip-freeipa-pvoborni-0123-IFW-base-classes-builders.patch http://pvoborni.fedorapeople.org/patches/wip-freeipa-pvoborni-0124-Events.patch > > 8. The declarative control button definitions still contain some code, > e.g. the handler function inside the button actions. Maybe the actions > should be defined in a separate list and the buttons can refer to them > by name. This separation sounds interesting. We will be able to define some list of actions which applies to that object and then action_list, control_buttons and action_panels can pick whatever they want. But the code you are referring to is just a shortcut to safe some lines of code - lets say anonymous action. > > 9. The 'control_buttons' attribute in search facet definitions contains > a 'buttons' array. Any plans to create custom control buttons widget? If > not maybe the 'control_buttons' itself could be the array. No plans but in 'control_buttons' definition can be also added state_listeners. > > 10. The 'Action List' in ticket #2248 for the password reset is actually > different than the action list for Enable/Disable. I was proposing to > create a small panel under the 'Account Settings' section, probably > something like this: > > --------------------------------------------------------------- > Account Settings > +-----------------+ > User login: admin | Actions: | > Password: | Reset password | > Password expiration: +-----------------+ > UID: > GID: > > This panel might better be called 'Action Panel' to distinguish from the > dropdown 'Action List' on the top. The reason for this panel is that a > Password Reset only affects an aspect of the user, not the entire object > like Enable/Disable (although that can also be argued differently), so > the action should be placed where it's relevant, in this case near the > Password field. The same concept will be used for ticket #2250, #2251, > #2252. > I'll consider my patch #133 NACKed and first create the action panels. -- Petr Vobornik From JR.Aquino at citrix.com Fri May 11 16:46:11 2012 From: JR.Aquino at citrix.com (JR Aquino) Date: Fri, 11 May 2012 16:46:11 +0000 Subject: [Freeipa-devel] [PATCH] 135 Host page fixed to work with disabled DNS support In-Reply-To: <4FA3C92F.7070206@redhat.com> References: <4FA3C92F.7070206@redhat.com> Message-ID: On May 4, 2012, at 5:18 AM, Petr Vobornik wrote: > When DNS support was disabled there were following errors in Web UI: > 1) Host details page was not filled with data > 2) Host adder dialog was broken -> unusable > 3) DNS tab was displayed in navigation > > The bugs were fixed by: > > 1) Was caused by entity_link_widget. The widget was modified to do not show link if other_entity (in this case dnsrecord) is not present. > > 2) Was caused by host_fqdn_widget. The widget is unusable because without DNS support it doesn't have access to DNS zone entity. The section with this widget was removed. Also IP address field was removed because it shouln't be used without DNS support. New 'fqdn' text box was added for specifying hostname. > > 3) New DNS config entity was initialized but it wasn't shown because it caused some JavaScript error. The dnsconfig's init method was modified to throw expected exception. Now no dns entity is initialized and therefore DNS tab in navigation is not displayed. > > https://fedorahosted.org/freeipa/ticket/2728 > -- > Petr Vobornik > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel Patch tested and works as advertised! ACK From mkosek at redhat.com Fri May 11 16:52:10 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 11 May 2012 18:52:10 +0200 Subject: [Freeipa-devel] [PATCH] 260 Replace DNS client based on acutil with python-dns Message-ID: <1336755130.19268.12.camel@priserak> python-dns is very feature-rich and it can help us a lot with our DNS related code. This patch does the first step, i.e. replaces acutil use with python-dns, which is more convenient to use as you will see in the patch. More integration will follow in the future. I send this patch rather early, so that I can get responses to this patch early and also so that we are able to catch issues in a safe distance from the next release. --- IPA client and server tool set used authconfig acutil module to for client DNS operations. This is not optimal DNS interface for several reasons: - does not provide native Python object oriented interface but but rather C-like interface based on functions and structures which is not easy to use and extend - acutil is not meant to be used by third parties besides authconfig and thus can break without notice Replace the acutil with python-dns package which has a feature rich interface for dealing with all different aspects of DNS including DNSSEC. The main target of this patch is to replace all uses of acutil DNS library with a use python-dns. In most cases, even though the larger parts of the code are changed, the actual functionality is changed only in the following cases: - redundant DNS checks were removed from verify_fqdn function in installutils to make the whole DNS check simpler and less error-prone. Logging was improves for the remaining checks - improved logging for ipa-client-install DNS discovery https://fedorahosted.org/freeipa/ticket/2730 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-260-replace-dns-client-based-on-acutil-with-python-dns.patch Type: text/x-patch Size: 40601 bytes Desc: not available URL: From rcritten at redhat.com Fri May 11 19:31:06 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 11 May 2012 15:31:06 -0400 Subject: [Freeipa-devel] [PATCH] 1011 fix permission-find Message-ID: <4FAD68FA.4070101@redhat.com> The permission-find command was failing for two reasons. The first was an overlap in the --name option and the primary key. The second was that aci's use a different attribute for name, aciname, so cn wasn't matching anything returning all entries. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1011-permission.patch Type: text/x-patch Size: 3307 bytes Desc: not available URL: From sgallagh at redhat.com Fri May 11 19:31:42 2012 From: sgallagh at redhat.com (Stephen Gallagher) Date: Fri, 11 May 2012 15:31:42 -0400 Subject: [Freeipa-devel] Announcing SSSD 1.9.0 beta 1 Message-ID: <1336764702.3038.23.camel@sgallagh520.sgallagh.bos.redhat.com> The SSSD team is proud to announce the first beta of our upcoming 1.9.0 release. We plan to have three beta releases, the first today, the second in mid-June and the last at the end of July. Each beta release will provide a set of new enhancements (mostly revolving around Kerberos cross-realm trust support and Active Directory integration). As always, you can download the latest sources at https://fedorahosted.org/sssd/ == Highlights == * Add native support for autofs to the IPA provider * Support for ID-mapping when connecting to Active Directory * Support for handling very large (> 1500 users) groups in Active Directory * Support for sub-domains (will be used for dealing with trust relationships) * Add a new fast in-memory cache to speed up lookups of cached data on repeated requests == Tickets Fixed == https://fedorahosted.org/sssd/ticket/357 SSSD should provide fast in memory cache to provide similar functionality as NSCD currently provides https://fedorahosted.org/sssd/ticket/783 Support range retrievals https://fedorahosted.org/sssd/ticket/887 Implement mechanism to fetch and store domain info https://fedorahosted.org/sssd/ticket/917 Document sss_tools better https://fedorahosted.org/sssd/ticket/949 Filter out inappropriate IP addresses from IPA dynamic DNS update https://fedorahosted.org/sssd/ticket/996 RFE: Allow Constructing uid from Active Directory objectSid https://fedorahosted.org/sssd/ticket/1031 [RFE] Implement "AD friendly" schema mapping https://fedorahosted.org/sssd/ticket/1064 Sub-Domains: define new get_domains method https://fedorahosted.org/sssd/ticket/1065 Sub-Domains: implement new get_domains method in IPA provider https://fedorahosted.org/sssd/ticket/1067 Sub-Domains: add new get_domains method to responders https://fedorahosted.org/sssd/ticket/1114 get_uid_from_pid() perfoms an improper read https://fedorahosted.org/sssd/ticket/1119 Monitor SIGKILL time should be configurable https://fedorahosted.org/sssd/ticket/1140 RFE Request for including pam_pwd_expiration_warning = 0 in sssd.conf https://fedorahosted.org/sssd/ticket/1170 sss_cache should support invalidating services and autofs maps https://fedorahosted.org/sssd/ticket/1172 Bad check for id_provider=local and access_provider=permit https://fedorahosted.org/sssd/ticket/1174 sssd.conf has wrong defaults for the "command" parameter https://fedorahosted.org/sssd/ticket/1176 SSH: Add dp_get_host_send to common responder code https://fedorahosted.org/sssd/ticket/1181 Typos in sssd manual https://fedorahosted.org/sssd/ticket/1203 Hash the hostname/port information in the known_hosts file. https://fedorahosted.org/sssd/ticket/1209 Convert all read and write loops to use atomic I/O function https://fedorahosted.org/sssd/ticket/1233 Memory leak in sss_sudo_send_recv_generic https://fedorahosted.org/sssd/ticket/1250 Add default home directory mapping https://fedorahosted.org/sssd/ticket/1271 Stop using HTML_FOOTER_DESCRIPTION in doxygen docs https://fedorahosted.org/sssd/ticket/1281 Add unit test for compatibility of ldap options between schemas https://fedorahosted.org/sssd/ticket/1289 Create a way to define a default shell for cases when there no shell https://fedorahosted.org/sssd/ticket/1297 Use keytab to select etypes for krb5_get_init_creds_keytab() https://fedorahosted.org/sssd/ticket/1298 Invalid cache file created when canoning principals during krb5_get_init_creds_keytab() https://fedorahosted.org/sssd/ticket/1301 sss_cache does nothing when executed without any options. https://fedorahosted.org/sssd/ticket/1305 sss_cache should return a warning/error while validating unknown user/group https://fedorahosted.org/sssd/ticket/1306 sss_cache should return an error, when executed against inactive domains https://fedorahosted.org/sssd/ticket/1313 exec_child, execv and friends don't return success https://fedorahosted.org/sssd/ticket/1316 kpasswd server status set to working when Kerberos auth succeeds == Detailed Changelog == Ariel Barria (1): * Bad check for id_provider=local and access_provider=permit Jakub Hrozek (105): * Fix SSH compilation on RHEL5 * AUTOFS: IPA provider * Two sssd-ldap manual pages fixes * Fix group enumeration * Only fetch SELinux string if the user is found * Remove setent structure when callback is called * Allocate setent structure on state, not on the client context * Fix memory hierarchy when processing nested group memberships * Fix case insensitive service lookups * Include the fd_limit configuration option * End request if ldap_parse_result fails * remove unused function * Save errno value before calling DEBUG * libnl: fix the path to phy80211 subdirectory * AUTOFS: Invoke implicit setautomntent if needed * AUTOFS: Search all search bases for automounter map entries * AUTOFS: speed up the client by requesting multiple entries at once * Use proper errno code * Only do one cycle when resolving a server * krb5_child: set debugging sooner * Search netgroups by alias, too * Detect cycle in the fail over on subsequent resolve requests only * Autofs: operate on contents of double-pointer, not address * Only free returned values on success * Save original name into the in-memory cache * Handle errors from lookup_netgr_step gracefully * Fix nested groups processing * Fix netgroup error handling * Handle empty elements in proxy netgroups: * Fix uninitialized variable * Free entry found in negative cache * Make the string_equal() function public * Save alias of the primary name, too * NSS: Look for services with correct case when cache is updated * AUTOFS: fix copy-and-paste bug in the autofs client * LDAP services: Keep the protocol around * Silence Coverity warning in the autofs test tool * Return correct resolv_status on resolver timeout * Add sss_get_cased_name_list utility function * LDAP services: Save lowercased protocol names in case-insensitive domains * Proxy services: Save lowercased protocol names and aliases in case-insensitive domains * Fix off-by-one error in principal selection * Catch cases where D-Bus connection is NULL * Use HTML_TIMESTAMP instead of HTML_FOOTER_DESCRIPTION * Fix regression in SSSDConfig.py * netlink integration: ensure that interface name is NULL-terminated * Remove forgotten DEBUG message * autofs: load the correct option * man: document that referral chasing might bring performance penalty * Prevent printing NULL from DEBUG messages * Do not call sdap_auth if not needed * pam_sss: improve error handling in SELinux code * Remove the "command" option from documentation * Add sysdb_set_service_attr and sysdb_set_autofsmap_attr * sss_cache: support invalidating services and autofs maps * autofs: Raise the maximum key length to PATH_MAX * sss_cache: Better error reporting * MAN: timeout can be specified for services, too * MAN: document the hostid and autofs providers * proxy: Canonicalize user and group names * proxy: new option proxy_fast_alias * Free controls in sdap_rebind_proc * Make the monitor SIGKILL time configurable * sdap_check_aliases must not error when detects the same user * sss_atomic_io: Do not fail reads with EPIPE if there is not enough data to read * Move atomic io function to a separate module * Convert read and write operations to sss_atomic_read * Document sss_tools better * Warn on 'make update-po' if there are manpages not listed in po4a.cfg * Test RFC2307bis and RFC2307 option maps * Get the RootDSE after binding if not successfull before * Lowercase group members in case-insensitive domains * NSS: Only return data from initgroups once * SUDO: Return ret, not EOK * SYSDB: return EOK if empty message is passed into get_rm_msg * SYSDB: check return value * SSH: return NULL on error in ssh_host_pubkeys_format_known_host_plain * SERVER: use the correct return code of sss_atomic_write_s * LDAP: check return value of sysdb_attrs_get_el * RESPONDER: check return value from confdb_get_int * PYHBAC: Return NULL on failure * PAM_SSS: report error code if write fails * NSS: Check return code of sss_mmap_cache_gr_store * IPA netgroups: return EOK when there are no netgroups to process * ipa_get_config_send: remove unused assignment * HBAC: Prevent NULL dereference in hbac_evaluate * DP: return correct error message when subdomains back end target is not configured * NSS: fix returning group from cache * SSS_DEBUGLEVEL: silence analyzer warnings * PROXY: return correct return codes * IPA: Check return values * AUTOFS: remove unused assignments * Rename split_service_name_filter * SSH: Add dp_get_host_send to common responder code * Read sysdb attribute name, not LDAP attribute map name * Kerberos locator: Include the correct krb5.h header file * Special-case LDAP_SIZELIMIT_EXCEEDED * krb5 locator: Do not leak addrinfo * Only reset kpasswd server status when performing a chpass operation * Try all KDCs when getting TGT for LDAP * Send the correct enumeration request * subdomains: Fix error handling in Data Provider * Filter out IP addresses inappropriate for DNS forward records * sysdb: return proper error code from sysdb_sudo_purge_all * SYSDB: Handle user and group renames better Jan Cholasta (22): * Add methods for activating and deactivating services to SSSDConfig * Add ssh service to sssd.api.conf * SSH: Verify that names received from client are valid UTF-8 in responder * SSH: Build man pages conditionally * SSH: Save SSH host name aliases * SSH: Refactor responder and client common code * UTIL: Add function for atomic I/O * SSH: Continue connecting to SSH server even when SSSD is not running in sss_ssh_knownhostsproxy * SSH: Manage global known_hosts file in the responder * SSH: Don't abort known_hosts update when host search fails * SSH: Add more debugging messages * SSH: Add missing break statements to sss_ssh_format_pubkey * SSH: Use fchmod instead of chmod on known_hosts file * SSH: Replace blocking getaddrinfo call in the responder with asynchronous resolver code * SSH: Remove unused --file option of sss_ssh_knownhostsproxy * SSH: Update sss_ssh_knownhostsproxy manual page * Include missing source files to the list of source files which contain translatable strings * SSH: Allow clients to explicitly specify host alias * SSH: Canonicalize host name and do reverse DNS lookup in sss_ssh_knownhostsproxy * SSH: Fix infinite loop in sss_ssh_knownhostsproxy * UTIL: Add HMAC-SHA-1 function * SSH: Add support for hashed known_hosts Jan Engelhardt (1): * build: resolve link failure Jan Zeleny (34): * Fixed issue with netgroup update in IPA provider * Don't give memory context in confdb where not needed * IPA hosts refactoring * SELinux related attributes added to config API * Delete missing attributes from netgroups to be stored * Modifications to simplify list_missing_attrs * Fix the script path * Fixed uninitialized pointer in SSH known host proxy * Fixed uninitialized pointer in SSH authorized keys client * Add umask before mkstemp() call in SSH responder * Fixed resource leak in ssh client code * Removed a block of dead code in sdap_async_groups.c * Removed unused block of code is sdap_fill_memberships() * Removed unused function sysdb_attrs_users_from_ldb_vals() * Fixed memory context in sdap_fill_memberships() * Fixed minor memory leak in ldap provider * Sysdb routines for subdomains * Add some utility functions for subdomains * Add conn_name to allow different names for domains and connections * Responder part of the subdomain retrieval work * Modified responder_get_domain() * Retrieve subdomains if there is a request for fully qualified user * Ask for subdomains in responder in the first request after startup * New config option for subdomains * Moved expand_homedir_template() from NSS responder to utility code * Add ID operations in subdomains * Send PAM requests for subdomains to the right provider * Basic support for subdomains in auth provider * Carry sysdb context and domain info in be_req structure * Accept be_req instead if be_ctx in LDAP access provider * Detect subdomain request in IPA access provider * Utilize sysdb context within be_req in HBAC * Two fixes in responder subdomain code * Modify behavior of pam_pwd_expiration_warning Marco Pizzoli (1): * Two manual pages fixes Pavel B?ezina (16): * Improve debug messages in sysdb_sudo_check_time() * SUDO responder: check if the input is a UTF-8 string * Refactor sss_result into sss_sudo_result * Redesign purging of the sudo cache * Honor case_sensitive option in sudo responder * Move sudo_dom_ctx.user to local variable * Hide --debug option in sss_debuglevel * Two memory leaks in sss_sudo_get_values * Missing debug message if sdap_sudo_refresh_set_timer fails * Use of unininitialized value in sudosrv_cache_set_entry and sudosrv_cache_lookup_internal * Use of unininitialized value in sss_sudo_parse_response * Potential NULL-dereference in sudosrv_cmd_get_sudorules * sudo api: check sss_status instead of errnop in sss_sudo_send_recv_generic() * Install and uninstall all documentation * fix copy and paste error in comment * Fix typo in debug message Simo Sorce (11): * nss_group: Cache the result from sssd when the glibc provided buffer is too small. * pam_sss: keep selinux optional * Use the correct hash table for pending requests * util: Helper headers for shared memory cache * nsssrv: shared memory cache server initialization * nsssrv: Add memory cache record handling utils * nsssrv: add handling of memory cache passwd map * sss_client: Add common shared memory cache utils * sss_client: shared memory cache passwd map support * nsssrv: add handling of memory cache group map * sss_client: shared memory cache group map support Stef Walter (6): * Fix erronous reference to the 'allow' access_provider * execv, excvp and exec_child never return EOK * If canon'ing principals, write ccache with updated default principal * Remove erroneous failure message in find_principal_in_keytab * Limit krb5_get_init_creds_keytab() to etypes in keytab * Clearer documentation for use_fully_qualified_names Stephen Gallagher (96): * Set version to 1.9dev * Updating translatable strings for string freeze * Updating translations * Remove dead code * Fix missing NULL check after malloc * Avoid uninitialized value comparison * Add missing breaks to switch statements * Fix uninitialized in_transaction * Fix bad failure handling in be_sudo_handler() * Check for failure in sss_packet_grow() * Fix uninitialized value error in proxy provider * Ensure NULL-termination in get_uid_from_pid() * Move sss_ssh_* binaries to the main 'sssd' package * Always include all manpage XML files in the distribution tarball * Fix missing %endif in sssd.spec.in * NSS: Always return the same protocol that was requested * LDAP: Ignore group member users that do not have name attributes * RESPONDERS: Allow increasing the file-descriptor limit * RESPONDERS: Make the fd_limit setting configurable * Add tool to convert debug levels * IPA: Add ipa_parse_search_base() * LDAP: Properly assign orig_dn * LDAP: Only use paging control on requests for multiple entries * LDAP: Remove unnecessary filter sanitize * Eliminate build-time requirement for nscd * PAM: Don't send PAM_SYSTEM_INFO message if module unset * Fix typo in autofs option description * Include the debug_level upgrade tool in the tarball * Include new manpages in translations * Fix typo in script name * Handle cases where UID is -1 * IPA: Set the DNS discovery domain to match ipa_domain * IPA: Fix segfault with srchost functionality enabled * DP: Reorganize memory hierarchy of requests * Prune python provides correctly * Make RPM spec more explicit * Build experimental features by default in RPMs * Properly terminate GIT_CHECKOUT * LDAP: Make sdap_access_send/recv public * IPA: Check nsAccountLock during PAM_ACCT_MGMT * PROXY: Create fake user entries for group lookups * SSH: Fix missing semicolon * IPA: Initialize hbac_ctx to NULL * i18n: Remove empty translations * LDAP: Add AD 2008r2 schema * IPA: Allow service lookups * SYSDB: Save only lowercased aliases in case-insensitive domains * LDAP: Errors retrieving the RootDSE should not be fatal * NSS: Fix debug message * Start SSSD earlier and stop it later * LDAP: Add better error logging when ldap_result() fails * LDAP: Fix memory leaks in synchronous_tls_setup * BUILDSYS: Create common libs for LDAP and KRB5 sources * Put dp_option maps in their own file * Add terminator for dp_option * Add better dp_option tests * Add terminator for sdap_attr_map * Add better tests for sdap_attr compability * Remove old compatibility tests * Fix building manpages in parallel build dirs * Clean up log messages about keytab_name * MAN: Improve ldap_disable_paging documentation * MAN: Add ldap_sasl_minssf to the manpage * Fix linker issue with pam_sss * murmurhash: Relax inline requirement * Handle endianness issues on older systems * SYSDB: Handle upgrade script failures better * LDAP: Add objectSID config option * LDAP: Add id-mapping option * SYSDB: Add sysdb routines for ID-mapping * LDAP: Add helper routines for ID-mapping * LDAP: Add ID mapping range settings * LDAP: Initialize ID mapping when configured * LDAP: Enable looking up ID-mapped users by name * LDAP: Add autorid compatibility mode * LDAP: Allow setting a default domain for id-mapping slice 0 * LDAP: Add routine to extract domain SID from an object SID * LDAP: Allow automatically-provisioning a domain and range * LDAP: Enable looking up id-mapped users by UID * LDAP: Allow looking up ID-mapped groups by name * LDAP: Enable looking up id-mapped groups by GID * LDAP: Map the user's primaryGroupID * LDAP: Add helper routine to convert LDAP blob to SID string * LDAP: Do not remove uidNumber and gidNumber attributes when saving id-mapped entries * LDAP: Add helper function to map IDs * LDAP: Treat groups with unmappable SIDs as non-POSIX groups * MAN: Add manpage for ID mapping * LDAP: Add support for enumeration of ID-mapped users and groups * SSSDConfigAPI: Fix missing option in tests * NSS: Add fallback_homedir option * NSS: Add default_shell option * SYSDB: Add better error logging to sysdb_set_entry_attr() * LDAP: Add attr_count return value to build_attrs_from_map() * LDAP: Handle very large Active Directory groups * Updating translations for 1.9.0 beta 1 release * Bumping version to 1.8.91 for 1.9.0 beta 1 release Sumit Bose (13): * Use curly braces in pkgconfig metadata file * Keep sysdb context in domain info struct * Remove sysdb_get_ctx_from_list() * Always initialize the returned data in sss_krb5_princ_realm() * Add idmap library * Check sub-domains in nss_cmd_get{pwuid|grgid}_search() * data provider: added subdomains * IPA: Add get-domains target * Add domain name to get_account_info request * Add s2n extended operation * Allow different SID representations in libidmap * Fix typo in spec file * Fix endian issue in SID conversion Yuri Chornoivan (2): * fix typos in manual * Fix typo: retreiving->retrieving -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From rcritten at redhat.com Fri May 11 20:03:16 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 11 May 2012 16:03:16 -0400 Subject: [Freeipa-devel] [PATCH] 1012 validate domain in installer Message-ID: <4FAD7084.1090905@redhat.com> Use our domain validator to validate the domain name we get in the installer. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1012-validator.patch Type: text/x-patch Size: 1961 bytes Desc: not available URL: From rcritten at redhat.com Fri May 11 20:34:51 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 11 May 2012 16:34:51 -0400 Subject: [Freeipa-devel] [PATCH] 1013 implement permission/aci find by subtree Message-ID: <4FAD77EB.3040709@redhat.com> permission-find --subtree wasn't implemented so always returned all entries (the option was ignored). rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1013-subtree.patch Type: text/x-patch Size: 3745 bytes Desc: not available URL: From mkosek at redhat.com Mon May 14 07:36:26 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 14 May 2012 09:36:26 +0200 Subject: [Freeipa-devel] [RANT] --setattr validation is a minefield. In-Reply-To: <4FABC076.7020308@redhat.com> References: <4F843D0F.5060906@redhat.com> <4F844BC4.2020109@redhat.com> <1334077635.16034.20.camel@balmora.brq.redhat.com> <4F846CF8.3020708@redhat.com> <1334080383.2282.9.camel@priserak> <4FABC076.7020308@redhat.com> Message-ID: <1336980986.4344.24.camel@balmora.brq.redhat.com> On Thu, 2012-05-10 at 15:19 +0200, Petr Viktorin wrote: > On 04/10/2012 07:53 PM, Martin Kosek wrote: > > On Tue, 2012-04-10 at 19:25 +0200, Petr Viktorin wrote: > >> On 04/10/2012 07:07 PM, Martin Kosek wrote: > >>> On Tue, 2012-04-10 at 17:03 +0200, Jan Cholasta wrote: > >>>> On 10.4.2012 16:00, Petr Viktorin wrote: > > [snip] > >>>> Like you said above, we should either not validate --{set,add,del}attr > >>>> or don't allow them on known attributes. > >>> > >>> IMHO, validating attributes we manage in the same way for both --setattr > >>> and standard attrs is not that wrong. It is a good precaution, because > >>> if we let an unvalidated value in, it can make even a bigger mess later > >>> in our pre_callbacks or post_callbacks where we assume that at this > >>> point everything is valid. > >> > >> Then we should validate *exactly* the same way, including not allowing > >> no_update attributes to be updated. > > > > That makes some sense, I could agree with that. > > > > Now that I have a ticket on this > (https://fedorahosted.org/freeipa/ticket/2580), I would like to get some > wider agreement here. > > The no_update/no_create attributes are mainly "enabled" flags > (ipaenabledflag, nsaccountlock, idnszoneactive), administrative > (krbprincipalname, ipauniqueid, ipacertificatesubjectbase), DNS record > type and data, and various virtual attributes. > > If setattr etc. is disabled for all of these, it will mainly matter for > the "enabled" flags. To be honest I don't know why we only allow > modifying those through special commands. > If there's some security reason for that, then setattr etc. should be > disabled for them; otherwise I think they should be changeable through > xyz-mod. I am not aware of any security reasons why we use special commands for enabling/disabling objects. I assume this is to make it different from standard object attribute changes and make sure that user does not disable the object "by accident" when doing a mod operation. Rob, maybe you remember the reason for this interface.... But since we already have this approach, we should keep it and implement missing "xyz-enable" and "xyz-disable" command so that user's using *attr interface to play with enabled/disabled attributes can switch to the specialized commands. So far, I noticed that only DNS zone object misses the enable/disable commands, I created a ticket to fix that: https://fedorahosted.org/freeipa/ticket/2754 > Either way, setattr etc. should honor the no_update flags. Any objections? > Nope - as long as ticket 2754 is fixed. Martin From mkosek at redhat.com Mon May 14 08:00:56 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 14 May 2012 10:00:56 +0200 Subject: [Freeipa-devel] [PATCH] 0047 Do not use extra command options in ACI, permission, selfservice In-Reply-To: <4FABA16A.10802@redhat.com> References: <4F8E97F6.2070703@redhat.com> <1336392614.29911.13.camel@balmora.brq.redhat.com> <4FABA16A.10802@redhat.com> Message-ID: <1336982456.4344.28.camel@balmora.brq.redhat.com> On Thu, 2012-05-10 at 13:07 +0200, Petr Viktorin wrote: > This is the second and likely the next-to-last part of disabling extra > command options (after this it's just test fixes and turning the > checking on). > > Part of the work for https://fedorahosted.org/freeipa/ticket/2509 > This patch looks and works OK. I just think you missed a pkey_only attribute for aci_find command. pkey_only is being passed to aci_find command by selfservice_find and delegation_find commands and it would fail in the hardened tests because it is not defined in aci_find command as it is not based on LDAPSearch class but crud.Search. It just needs to be added to aci_find command and it should be fine. Martin From mkosek at redhat.com Mon May 14 08:08:48 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 14 May 2012 10:08:48 +0200 Subject: [Freeipa-devel] [PATCH] 1011 fix permission-find In-Reply-To: <4FAD68FA.4070101@redhat.com> References: <4FAD68FA.4070101@redhat.com> Message-ID: <1336982928.4344.29.camel@balmora.brq.redhat.com> On Fri, 2012-05-11 at 15:31 -0400, Rob Crittenden wrote: > The permission-find command was failing for two reasons. The first was > an overlap in the --name option and the primary key. The second was that > aci's use a different attribute for name, aciname, so cn wasn't matching > anything returning all entries. > > rob ACK. Pushed to master. Martin From mkosek at redhat.com Mon May 14 08:20:29 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 14 May 2012 10:20:29 +0200 Subject: [Freeipa-devel] [RANT] --setattr validation is a minefield. In-Reply-To: <1336980986.4344.24.camel@balmora.brq.redhat.com> References: <4F843D0F.5060906@redhat.com> <4F844BC4.2020109@redhat.com> <1334077635.16034.20.camel@balmora.brq.redhat.com> <4F846CF8.3020708@redhat.com> <1334080383.2282.9.camel@priserak> <4FABC076.7020308@redhat.com> <1336980986.4344.24.camel@balmora.brq.redhat.com> Message-ID: <1336983629.4344.31.camel@balmora.brq.redhat.com> On Mon, 2012-05-14 at 09:36 +0200, Martin Kosek wrote: > On Thu, 2012-05-10 at 15:19 +0200, Petr Viktorin wrote: > > On 04/10/2012 07:53 PM, Martin Kosek wrote: > > > On Tue, 2012-04-10 at 19:25 +0200, Petr Viktorin wrote: > > >> On 04/10/2012 07:07 PM, Martin Kosek wrote: > > >>> On Tue, 2012-04-10 at 17:03 +0200, Jan Cholasta wrote: > > >>>> On 10.4.2012 16:00, Petr Viktorin wrote: > > > [snip] > > >>>> Like you said above, we should either not validate --{set,add,del}attr > > >>>> or don't allow them on known attributes. > > >>> > > >>> IMHO, validating attributes we manage in the same way for both --setattr > > >>> and standard attrs is not that wrong. It is a good precaution, because > > >>> if we let an unvalidated value in, it can make even a bigger mess later > > >>> in our pre_callbacks or post_callbacks where we assume that at this > > >>> point everything is valid. > > >> > > >> Then we should validate *exactly* the same way, including not allowing > > >> no_update attributes to be updated. > > > > > > That makes some sense, I could agree with that. > > > > > > > Now that I have a ticket on this > > (https://fedorahosted.org/freeipa/ticket/2580), I would like to get some > > wider agreement here. > > > > The no_update/no_create attributes are mainly "enabled" flags > > (ipaenabledflag, nsaccountlock, idnszoneactive), administrative > > (krbprincipalname, ipauniqueid, ipacertificatesubjectbase), DNS record > > type and data, and various virtual attributes. > > > > If setattr etc. is disabled for all of these, it will mainly matter for > > the "enabled" flags. To be honest I don't know why we only allow > > modifying those through special commands. > > If there's some security reason for that, then setattr etc. should be > > disabled for them; otherwise I think they should be changeable through > > xyz-mod. > > I am not aware of any security reasons why we use special commands for > enabling/disabling objects. I assume this is to make it different from > standard object attribute changes and make sure that user does not > disable the object "by accident" when doing a mod operation. Rob, maybe > you remember the reason for this interface.... > > But since we already have this approach, we should keep it and implement > missing "xyz-enable" and "xyz-disable" command so that user's using > *attr interface to play with enabled/disabled attributes can switch to > the specialized commands. > > So far, I noticed that only DNS zone object misses the enable/disable > commands, I created a ticket to fix that: > > https://fedorahosted.org/freeipa/ticket/2754 > > > Either way, setattr etc. should honor the no_update flags. Any objections? > > > > Nope - as long as ticket 2754 is fixed. > I just noticed we have the enable/disable command for DNS zone as well, I somehow managed to overlook it... Sorry for noise, closing the ticket 2754. Martin From mkosek at redhat.com Mon May 14 08:36:36 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 14 May 2012 10:36:36 +0200 Subject: [Freeipa-devel] [PATCH] 1012 validate domain in installer In-Reply-To: <4FAD7084.1090905@redhat.com> References: <4FAD7084.1090905@redhat.com> Message-ID: <1336984596.4344.41.camel@balmora.brq.redhat.com> On Fri, 2012-05-11 at 16:03 -0400, Rob Crittenden wrote: > Use our domain validator to validate the domain name we get in the > installer. > > rob I found few issues with the patch: 1) The "unexpected error" is not very user friendly error message: # ipa-server-install ... Server host name [vm-109.idm.lab.bos.redhat.com]: The domain name has been calculated based on the host name. Please confirm the domain name [idm.lab.bos.redhat.com]: bar-.com Unexpected error - see ipaserver-install.log for details: only letters, numbers, and - are allowed. DNS label may not start or end with - I think we should make it consistent with hostname validation error message format: # ipa-server-install ... Server host name [vm-109.idm.lab.bos.redhat.com]: foo- Invalid hostname 'foo-', must be fully-qualified. 2) The error is also confusing when domain is passed as an option, user won't know what actually failed: # ipa-server-install --domain .. ... Server host name [vm-109.idm.lab.bos.redhat.com]: Unexpected error - see ipaserver-install.log for details: empty DNS label As for value passed via option, I think we should quit immediately and not in second or third interactive step - like we do for --zonemgr validation: # ipa-server-install --zonemgr=foo Usage: ipa-server-install [options] ipa-server-install: error: invalid zonemgr: missing address domain This applies both for --hostname and --domain options. Martin From mkosek at redhat.com Mon May 14 08:40:17 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 14 May 2012 10:40:17 +0200 Subject: [Freeipa-devel] [PATCH] 0047 Do not use extra command options in ACI, permission, selfservice In-Reply-To: <1336982456.4344.28.camel@balmora.brq.redhat.com> References: <4F8E97F6.2070703@redhat.com> <1336392614.29911.13.camel@balmora.brq.redhat.com> <4FABA16A.10802@redhat.com> <1336982456.4344.28.camel@balmora.brq.redhat.com> Message-ID: <1336984817.4344.43.camel@balmora.brq.redhat.com> On Mon, 2012-05-14 at 10:00 +0200, Martin Kosek wrote: > On Thu, 2012-05-10 at 13:07 +0200, Petr Viktorin wrote: > > This is the second and likely the next-to-last part of disabling extra > > command options (after this it's just test fixes and turning the > > checking on). > > > > Part of the work for https://fedorahosted.org/freeipa/ticket/2509 > > > > This patch looks and works OK. I just think you missed a pkey_only > attribute for aci_find command. pkey_only is being passed to aci_find > command by selfservice_find and delegation_find commands and it would > fail in the hardened tests because it is not defined in aci_find command > as it is not based on LDAPSearch class but crud.Search. > > It just needs to be added to aci_find command and it should be fine. > > Martin Petr? noticed that pkey_only was already explicitly added to takes_options of aci_find command. The patch is OK then. ACK. Pushed to master. Martin From mkosek at redhat.com Mon May 14 09:04:45 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 14 May 2012 11:04:45 +0200 Subject: [Freeipa-devel] [PATCH] 1013 implement permission/aci find by subtree In-Reply-To: <4FAD77EB.3040709@redhat.com> References: <4FAD77EB.3040709@redhat.com> Message-ID: <1336986285.4344.52.camel@balmora.brq.redhat.com> On Fri, 2012-05-11 at 16:34 -0400, Rob Crittenden wrote: > permission-find --subtree wasn't implemented so always returned all > entries (the option was ignored). > > rob I found the following 2 issues: 1) The following piece of code is over-complicated: + found = False + if kw['subtree'] == target: + found = True + if not found: + try: + results.remove(a) + except ValueError: + pass This would simpler and more readable: + if kw['subtree'] != target: + try: + results.remove(a) + except ValueError: + pass 2) I know we don't validate the subtree expression, but I wonder if we shouldn't make the subtree comparing at least case insensitive as DN is also not case sensitive. Martin From dan at crasman.fi Mon May 14 10:11:49 2012 From: dan at crasman.fi (Dan Airinen) Date: Mon, 14 May 2012 13:11:49 +0300 Subject: [Freeipa-devel] Build problems with FreeIPA v2.2.0 Message-ID: <1336990309.3266.3.camel@echelon> Hi, when i try to build FreeIPA v2.2.0 rpm's on Scientific Linux v6.2, i get the following errors (during make-lint phase): ipaserver/plugins/ldap2.py:176: [E1121, IPASimpleLDAPObject.rename_s] Too many positional arguments for function call ipa-client/ipa-install/ipa-client-install:742: [E1101, configure_sssd_conf] Instance of 'SSSDConfig' has no 'activate_service' member ipapython/nsslib.py:280: [E1121, NSSConnection.endheaders] Too many positional arguments for function call What might cause these errors ?. Br, -- -- Dan Airinen dan at crasman.fi Crasman Co Ltd - http://www.crasman.fi From jhrozek at redhat.com Mon May 14 10:30:57 2012 From: jhrozek at redhat.com (Jakub Hrozek) Date: Mon, 14 May 2012 12:30:57 +0200 Subject: [Freeipa-devel] Build problems with FreeIPA v2.2.0 In-Reply-To: <1336990309.3266.3.camel@echelon> References: <1336990309.3266.3.camel@echelon> Message-ID: <20120514102843.GA27728@unused-4-110.brq.redhat.com> On Mon, May 14, 2012 at 01:11:49PM +0300, Dan Airinen wrote: > ipa-client/ipa-install/ipa-client-install:742: [E1101, > configure_sssd_conf] Instance of 'SSSDConfig' has no 'activate_service' > member What version of the SSSD are you running? The activate_service method was added into the SSSD during the 1.8 beta phase. I suggest to also go with SSSD 1.8, it's going to be present in RHEL 6.3 anyway. That said, ipa-client should require SSSD-1.8 if it uses newly added features. The other errors seem to be unrelated. From dan at crasman.fi Mon May 14 11:59:56 2012 From: dan at crasman.fi (Dan Airinen) Date: Mon, 14 May 2012 14:59:56 +0300 Subject: [Freeipa-devel] Build problems with FreeIPA v2.2.0 In-Reply-To: <20120514102843.GA27728@unused-4-110.brq.redhat.com> References: <1336990309.3266.3.camel@echelon> <20120514102843.GA27728@unused-4-110.brq.redhat.com> Message-ID: <1336996796.3266.13.camel@echelon> Thanks Jakub!, manually updated sssd RPM's to 1.8.3 version. And that error was now fixed. Still problems with the other two errors tho. On ma, 2012-05-14 at 12:30 +0200, Jakub Hrozek wrote: > On Mon, May 14, 2012 at 01:11:49PM +0300, Dan Airinen wrote: > > ipa-client/ipa-install/ipa-client-install:742: [E1101, > > configure_sssd_conf] Instance of 'SSSDConfig' has no 'activate_service' > > member > > What version of the SSSD are you running? The activate_service method was > added into the SSSD during the 1.8 beta phase. I suggest to also go with > SSSD 1.8, it's going to be present in RHEL 6.3 anyway. > > That said, ipa-client should require SSSD-1.8 if it uses newly added features. > > The other errors seem to be unrelated. > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- -- Dan Airinen dan at crasman.fi Crasman Co Ltd - http://www.crasman.fi From pviktori at redhat.com Mon May 14 12:47:23 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 14 May 2012 14:47:23 +0200 Subject: [Freeipa-devel] [PATCH] 0050 Fail on unknown Command options In-Reply-To: <1336984817.4344.43.camel@balmora.brq.redhat.com> References: <4F8E97F6.2070703@redhat.com> <1336392614.29911.13.camel@balmora.brq.redhat.com> <4FABA16A.10802@redhat.com> <1336982456.4344.28.camel@balmora.brq.redhat.com> <1336984817.4344.43.camel@balmora.brq.redhat.com> Message-ID: <4FB0FEDB.4020806@redhat.com> The final part of rejecting unknown Command arguments: enable the validation, add tests. Also fix up things that were changed since the previous patches. https://fedorahosted.org/freeipa/ticket/2509 -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0050-Fail-on-unknown-Command-options.patch Type: text/x-patch Size: 12114 bytes Desc: not available URL: From pviktori at redhat.com Mon May 14 13:22:28 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 14 May 2012 15:22:28 +0200 Subject: [Freeipa-devel] [PATCH] 0051 Check for empty/single value parameters before calling callbacks Message-ID: <4FB10714.6010706@redhat.com> Pre-callbacks were called before a few validation steps, leading to internal errors if the pre-callback relied on valid data. https://fedorahosted.org/freeipa/ticket/2701 Regression test included. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0051-Check-for-empty-parameters-before-calling-callbacks.patch Type: text/x-patch Size: 2645 bytes Desc: not available URL: From rcritten at redhat.com Mon May 14 14:00:31 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 14 May 2012 10:00:31 -0400 Subject: [Freeipa-devel] [RANT] --setattr validation is a minefield. In-Reply-To: <1336980986.4344.24.camel@balmora.brq.redhat.com> References: <4F843D0F.5060906@redhat.com> <4F844BC4.2020109@redhat.com> <1334077635.16034.20.camel@balmora.brq.redhat.com> <4F846CF8.3020708@redhat.com> <1334080383.2282.9.camel@priserak> <4FABC076.7020308@redhat.com> <1336980986.4344.24.camel@balmora.brq.redhat.com> Message-ID: <4FB10FFF.4050900@redhat.com> Martin Kosek wrote: > On Thu, 2012-05-10 at 15:19 +0200, Petr Viktorin wrote: >> On 04/10/2012 07:53 PM, Martin Kosek wrote: >>> On Tue, 2012-04-10 at 19:25 +0200, Petr Viktorin wrote: >>>> On 04/10/2012 07:07 PM, Martin Kosek wrote: >>>>> On Tue, 2012-04-10 at 17:03 +0200, Jan Cholasta wrote: >>>>>> On 10.4.2012 16:00, Petr Viktorin wrote: >>> [snip] >>>>>> Like you said above, we should either not validate --{set,add,del}attr >>>>>> or don't allow them on known attributes. >>>>> >>>>> IMHO, validating attributes we manage in the same way for both --setattr >>>>> and standard attrs is not that wrong. It is a good precaution, because >>>>> if we let an unvalidated value in, it can make even a bigger mess later >>>>> in our pre_callbacks or post_callbacks where we assume that at this >>>>> point everything is valid. >>>> >>>> Then we should validate *exactly* the same way, including not allowing >>>> no_update attributes to be updated. >>> >>> That makes some sense, I could agree with that. >>> >> >> Now that I have a ticket on this >> (https://fedorahosted.org/freeipa/ticket/2580), I would like to get some >> wider agreement here. >> >> The no_update/no_create attributes are mainly "enabled" flags >> (ipaenabledflag, nsaccountlock, idnszoneactive), administrative >> (krbprincipalname, ipauniqueid, ipacertificatesubjectbase), DNS record >> type and data, and various virtual attributes. >> >> If setattr etc. is disabled for all of these, it will mainly matter for >> the "enabled" flags. To be honest I don't know why we only allow >> modifying those through special commands. >> If there's some security reason for that, then setattr etc. should be >> disabled for them; otherwise I think they should be changeable through >> xyz-mod. > > I am not aware of any security reasons why we use special commands for > enabling/disabling objects. I assume this is to make it different from > standard object attribute changes and make sure that user does not > disable the object "by accident" when doing a mod operation. Rob, maybe > you remember the reason for this interface.... > > But since we already have this approach, we should keep it and implement > missing "xyz-enable" and "xyz-disable" command so that user's using > *attr interface to play with enabled/disabled attributes can switch to > the specialized commands. > > So far, I noticed that only DNS zone object misses the enable/disable > commands, I created a ticket to fix that: > > https://fedorahosted.org/freeipa/ticket/2754 > >> Either way, setattr etc. should honor the no_update flags. Any objections? >> > > Nope - as long as ticket 2754 is fixed. > > Martin > I think those are there so they don't show up for the -mod command since we have a separate interface for doing it. rob From rcritten at redhat.com Mon May 14 14:06:35 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 14 May 2012 10:06:35 -0400 Subject: [Freeipa-devel] Build problems with FreeIPA v2.2.0 In-Reply-To: <1336996796.3266.13.camel@echelon> References: <1336990309.3266.3.camel@echelon> <20120514102843.GA27728@unused-4-110.brq.redhat.com> <1336996796.3266.13.camel@echelon> Message-ID: <4FB1116B.4020701@redhat.com> Dan Airinen wrote: > Thanks Jakub!, > > manually updated sssd RPM's to 1.8.3 version. > And that error was now fixed. > > Still problems with the other two errors tho. What version of python-nss and python-ldap do you have installed? rob From pspacek at redhat.com Mon May 14 14:44:42 2012 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 14 May 2012 16:44:42 +0200 Subject: [Freeipa-devel] [PATCH 0020] Separate LDAP result from LDAP connection, fix deadlock. In-Reply-To: <20120511102654.GA2785@redhat.com> References: <4FA7C4C3.7020202@redhat.com> <20120511102654.GA2785@redhat.com> Message-ID: <4FB11A5A.8000804@redhat.com> On 05/11/2012 12:26 PM, Adam Tkac wrote: > On Mon, May 07, 2012 at 02:49:07PM +0200, Petr Spacek wrote: >> Hello, >> >> this patch fixes https://fedorahosted.org/bind-dyndb-ldap/ticket/66: >> Plugin deadlocks during new zone load when connections == 1. >> >> It fixes structural problem, when LDAP query result was tied with >> LDAP connection up. It wasn't possible to release connection and >> work with query result after that. >> Described deadlock is consequence of this problematic design. >> >> Now LDAP connection is separated from LDAP result. Next planed patch >> will avoid "manual" connection management, so possibility of >> deadlock should be next to zero. >> >> Petr^2 Spacek > > Hello Peter, > > good work, please check my comments below. > > Regards, Adam > >> From 8ee1fd607531ef71369e99c9228456baea45b65d Mon Sep 17 00:00:00 2001 >> From: Petr Spacek >> Date: Mon, 7 May 2012 12:51:09 +0200 >> Subject: [PATCH] Separate LDAP result from LDAP connection, fix deadlock. >> https://fedorahosted.org/bind-dyndb-ldap/ticket/66 >> Signed-off-by: Petr Spacek Hello Adam, thanks for ideas/improvements! Reworked patch is attached. I did all proposed changes except this one: @ ldap_psearch_watcher: >> restart: (... snip ...) >> soft_err: >> - >> - ldap_msgfree(conn->result); >> - ldap_entrylist_destroy(conn->mctx, >> - &conn->ldap_entries); >> + ; > > Empty label "soft_err:" is useless, please remove it and use "continue;" on > appropriate places; I think "continue" in this place can lead to memory leak, so I removed soft_err by other way. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0020-2-Separate-LDAP-result-from-LDAP-connection-fix-deadlo.patch Type: text/x-patch Size: 19841 bytes Desc: not available URL: From jdennis at redhat.com Mon May 14 15:53:12 2012 From: jdennis at redhat.com (John Dennis) Date: Mon, 14 May 2012 11:53:12 -0400 Subject: [Freeipa-devel] Build problems with FreeIPA v2.2.0 In-Reply-To: <1336990309.3266.3.camel@echelon> References: <1336990309.3266.3.camel@echelon> Message-ID: <4FB12A68.3000302@redhat.com> On 05/14/2012 06:11 AM, Dan Airinen wrote: > Hi, > > when i try to build FreeIPA v2.2.0 rpm's on Scientific Linux v6.2, i get > the following errors (during make-lint phase): > > ipaserver/plugins/ldap2.py:176: [E1121, IPASimpleLDAPObject.rename_s] > Too many positional arguments for function call > ipa-client/ipa-install/ipa-client-install:742: [E1101, > configure_sssd_conf] Instance of 'SSSDConfig' has no 'activate_service' > member > ipapython/nsslib.py:280: [E1121, NSSConnection.endheaders] Too many > positional arguments for function call > > What might cause these errors ?. incompatible versions, what version of python do you have, what version of python-ldap? -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From edewata at redhat.com Mon May 14 17:08:21 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 14 May 2012 12:08:21 -0500 Subject: [Freeipa-devel] [PATCHES] 124-132 Inconsistent ways to show/change entry status In-Reply-To: <4FABB239.2040500@redhat.com> References: <4FA137C4.7000101@redhat.com> <4FA285D0.90301@redhat.com> <4FA287A0.2070107@redhat.com> <4FA883BA.8080803@redhat.com> <4FABB239.2040500@redhat.com> Message-ID: <4FB13C05.7090005@redhat.com> On 5/10/2012 7:19 AM, Petr Vobornik wrote: > Updated patch attached. See comments below. > > On 05/08/2012 04:23 AM, Endi Sukma Dewata wrote: >> Try adding a very long DNS zone, then open the zone. Compare the >> breadcrumbs in the DNS Resource Records page and in the Settings page, >> in my case the second one is a bit longer. I think the length of the >> breadcrumb should be calculated differently. > > I remade how breadcrumb is limited. Now it tries to use as much space > available with keeping low complexity level of calculation. It still > uses an average but now the average is calculated in two phases in order > to neglect a presence of short keys. ACK. -- Endi S. Dewata From edewata at redhat.com Mon May 14 17:09:08 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 14 May 2012 12:09:08 -0500 Subject: [Freeipa-devel] [PATCH] 136 Correction of nested search facets tab labels In-Reply-To: <4FAB988C.5020905@redhat.com> References: <4FAB988C.5020905@redhat.com> Message-ID: <4FB13C34.6040007@redhat.com> On 5/10/2012 5:29 AM, Petr Vobornik wrote: > Nested search facets were using 'search' tab label instead of their > nested entity name. > > This patch is fixing that regression. > > https://fedorahosted.org/freeipa/ticket/2744 ACK. -- Endi S. Dewata From edewata at redhat.com Mon May 14 17:12:41 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 14 May 2012 12:12:41 -0500 Subject: [Freeipa-devel] [PATCH] 133 Action list for user password In-Reply-To: <4FA248D9.50406@redhat.com> References: <4FA248D9.50406@redhat.com> Message-ID: <4FB13D09.7010505@redhat.com> On 5/3/2012 3:59 AM, Petr Vobornik wrote: > Currently the user password is shown as follows in the details page: > Password: Reset Password > > This is inconsistent with the rest of the page because the 'Reset > Password' is an action, not the value of the password. > > Now password is shown as follows: > Password: ******* (if set) > Password: (if not set) > > Reset password action was moved to Action List. > > https://fedorahosted.org/freeipa/ticket/2248 > > Note: I will address enabling/disabling 'reset password' option in > action list in ticket #2318. Just for the record: On 5/11/2012 11:36 AM, Petr Vobornik wrote: > On 05/08/2012 01:47 AM, Endi Sukma Dewata wrote: >> 10. The 'Action List' in ticket #2248 for the password reset is actually >> different than the action list for Enable/Disable. I was proposing to >> create a small panel under the 'Account Settings' section, probably >> something like this: >> >> --------------------------------------------------------------- >> Account Settings >> +-----------------+ >> User login: admin | Actions: | >> Password: | Reset password | >> Password expiration: +-----------------+ >> UID: >> GID: >> >> This panel might better be called 'Action Panel' to distinguish from the >> dropdown 'Action List' on the top. The reason for this panel is that a >> Password Reset only affects an aspect of the user, not the entire object >> like Enable/Disable (although that can also be argued differently), so >> the action should be placed where it's relevant, in this case near the >> Password field. The same concept will be used for ticket #2250, #2251, >> #2252. >> > > I'll consider my patch #133 NACKed and first create the action panels. -- Endi S. Dewata From rcritten at redhat.com Mon May 14 21:29:38 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 14 May 2012 17:29:38 -0400 Subject: [Freeipa-devel] [PATCH] 1012 validate domain in installer In-Reply-To: <1336984596.4344.41.camel@balmora.brq.redhat.com> References: <4FAD7084.1090905@redhat.com> <1336984596.4344.41.camel@balmora.brq.redhat.com> Message-ID: <4FB17942.2050605@redhat.com> Martin Kosek wrote: > On Fri, 2012-05-11 at 16:03 -0400, Rob Crittenden wrote: >> Use our domain validator to validate the domain name we get in the >> installer. >> >> rob > > I found few issues with the patch: > > 1) The "unexpected error" is not very user friendly error message: > > # ipa-server-install > ... > Server host name [vm-109.idm.lab.bos.redhat.com]: > > The domain name has been calculated based on the host name. > > Please confirm the domain name [idm.lab.bos.redhat.com]: bar-.com > > Unexpected error - see ipaserver-install.log for details: > only letters, numbers, and - are allowed. DNS label may not start or > end with - > > > I think we should make it consistent with hostname validation error > message format: > > # ipa-server-install > ... > Server host name [vm-109.idm.lab.bos.redhat.com]: foo- > > Invalid hostname 'foo-', must be fully-qualified. > > > 2) The error is also confusing when domain is passed as an option, user > won't know what actually failed: > > # ipa-server-install --domain .. > ... > Server host name [vm-109.idm.lab.bos.redhat.com]: > > Unexpected error - see ipaserver-install.log for details: > empty DNS label > > As for value passed via option, I think we should quit immediately and > not in second or third interactive step - like we do for --zonemgr > validation: > > # ipa-server-install --zonemgr=foo > Usage: ipa-server-install [options] > > ipa-server-install: error: invalid zonemgr: missing address domain > > This applies both for --hostname and --domain options. > > Martin > Done. There is a separate ticket to validate hostname in the parser. It is a bit more complicated since it depends on other options so I punted on that. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1012-2-validator.patch Type: text/x-diff Size: 3219 bytes Desc: not available URL: From rcritten at redhat.com Mon May 14 21:45:59 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 14 May 2012 17:45:59 -0400 Subject: [Freeipa-devel] [PATCH] 1013 implement permission/aci find by subtree In-Reply-To: <1336986285.4344.52.camel@balmora.brq.redhat.com> References: <4FAD77EB.3040709@redhat.com> <1336986285.4344.52.camel@balmora.brq.redhat.com> Message-ID: <4FB17D17.6020602@redhat.com> Martin Kosek wrote: > On Fri, 2012-05-11 at 16:34 -0400, Rob Crittenden wrote: >> permission-find --subtree wasn't implemented so always returned all >> entries (the option was ignored). >> >> rob > > I found the following 2 issues: > > 1) The following piece of code is over-complicated: > > + found = False > + if kw['subtree'] == target: > + found = True > + if not found: > + try: > + results.remove(a) > + except ValueError: > + pass > > This would simpler and more readable: > + if kw['subtree'] != target: > + try: > + results.remove(a) > + except ValueError: > + pass > > 2) I know we don't validate the subtree expression, but I wonder if we > shouldn't make the subtree comparing at least case insensitive as DN is > also not case sensitive. > > Martin > Yeah, much simpler. Fixed both. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1013-2-subtree.patch Type: text/x-diff Size: 3663 bytes Desc: not available URL: From william at firstyear.id.au Mon May 14 23:45:40 2012 From: william at firstyear.id.au (William Brown) Date: Tue, 15 May 2012 09:15:40 +0930 Subject: [Freeipa-devel] Adding indices and permissions to FreeIPA Message-ID: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> Hi, I am currently working on adding DHCP support, so that FreeIPA can control an ISC-DHCP server. As part of this, I need to add a number of indices to 389ds, as well as a number of permissions (ACIs) and groups to manage these. Is there a specific way to add these? Should they be added as part of the DHCP feature installation process, or should they be part of the base server install? Sincerely, William Brown pgp.mit.edu http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mgregg at redhat.com Tue May 15 01:55:47 2012 From: mgregg at redhat.com (Michael Gregg) Date: Mon, 14 May 2012 18:55:47 -0700 Subject: [Freeipa-devel] New beaker server in mountain view Message-ID: <4FB1B7A3.8030706@redhat.com> At long last, a beaker server is now up and running here in Mountain View. The beaker server is able to control nearly all of the machines here at the mountain view location. Login to it here: http://hammer1.dsdev.sjc.redhat.com/bkr/ You should be able to login with your kerberos username and password. If you would like to use beaker through the cli, follow these steps: 1. make sure you have beaker client installed and configured for use with the wesford lab controller. 2. Download and install this RPM: https://engineering.redhat.com/trac/ipa-tests/browser/trunk/ipa-tests/beaker/ipa-server/shared/bkr-hammer-1.0-1.noarch.rpm 3. you should be able to use "bkr-hammer" just as you would use the bkr command. I may need to add you to some beaker groups to allow you to use all of the machines here. There is only a small number of tests in this beaker server. I tried to add all of the ipa tasks. Feel free to add any missing tasks that you need. Or, as me to add them, and I'm move them into this lab controller. There is also a reduced set of rhel builds as there isn't enough bandwidth here to sync the nightlies here. I'll keep this server syned up with the latest snapshots. Look at the currently available distros here: http://hammer1.dsdev.sjc.redhat.com/bkr/distros/ Give it a try, feedback is welcome. Michael- From mkosek at redhat.com Tue May 15 06:38:01 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 May 2012 08:38:01 +0200 Subject: [Freeipa-devel] [PATCH] 1012 validate domain in installer In-Reply-To: <4FB17942.2050605@redhat.com> References: <4FAD7084.1090905@redhat.com> <1336984596.4344.41.camel@balmora.brq.redhat.com> <4FB17942.2050605@redhat.com> Message-ID: <1337063881.10688.4.camel@balmora.brq.redhat.com> On Mon, 2012-05-14 at 17:29 -0400, Rob Crittenden wrote: > Martin Kosek wrote: > > On Fri, 2012-05-11 at 16:03 -0400, Rob Crittenden wrote: > >> Use our domain validator to validate the domain name we get in the > >> installer. > >> > >> rob > > > > I found few issues with the patch: > > > > 1) The "unexpected error" is not very user friendly error message: > > > > # ipa-server-install > > ... > > Server host name [vm-109.idm.lab.bos.redhat.com]: > > > > The domain name has been calculated based on the host name. > > > > Please confirm the domain name [idm.lab.bos.redhat.com]: bar-.com > > > > Unexpected error - see ipaserver-install.log for details: > > only letters, numbers, and - are allowed. DNS label may not start or > > end with - > > > > > > I think we should make it consistent with hostname validation error > > message format: > > > > # ipa-server-install > > ... > > Server host name [vm-109.idm.lab.bos.redhat.com]: foo- > > > > Invalid hostname 'foo-', must be fully-qualified. > > > > > > 2) The error is also confusing when domain is passed as an option, user > > won't know what actually failed: > > > > # ipa-server-install --domain .. > > ... > > Server host name [vm-109.idm.lab.bos.redhat.com]: > > > > Unexpected error - see ipaserver-install.log for details: > > empty DNS label > > > > As for value passed via option, I think we should quit immediately and > > not in second or third interactive step - like we do for --zonemgr > > validation: > > > > # ipa-server-install --zonemgr=foo > > Usage: ipa-server-install [options] > > > > ipa-server-install: error: invalid zonemgr: missing address domain > > > > This applies both for --hostname and --domain options. > > > > Martin > > > > Done. > > There is a separate ticket to validate hostname in the parser. It is a > bit more complicated since it depends on other options so I punted on that. > > rob Nack. Domain name passed via option is not used. I assume this is the reason: +def domain_callback(option, opt_str, value, parser): ... + + parser.values.zonemgr = value + Btw. callback is not necessary for domain validation, it may be done right in parse_options() in ipa-server-install with other checks. Martin From mkosek at redhat.com Tue May 15 06:51:33 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 May 2012 08:51:33 +0200 Subject: [Freeipa-devel] Adding indices and permissions to FreeIPA In-Reply-To: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> References: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> Message-ID: <1337064693.10688.15.camel@balmora.brq.redhat.com> On Tue, 2012-05-15 at 09:15 +0930, William Brown wrote: > Hi, > > > I am currently working on adding DHCP support, so that FreeIPA can > control an ISC-DHCP server. > > > As part of this, I need to add a number of indices to 389ds, as well > as a number of permissions (ACIs) and groups to manage these. > > > Is there a specific way to add these? Should they be added as part of > the DHCP feature installation process, or should they be part of the > base server install? Hello William, in FreeIPA there are 2 common ways to add indices to the DS: 1) LDIFs in the installation process (ipa-server-install) You can see for example install/share/replica-s4u2proxy.ldif in our git repo. In ipaserver/install/dsinstance.py shows how it is sent to LDAP. 2) LDAP update files that are used to update an already installed IPA server when freeipa-server package is being updated. These update files are created when there are changes to the LDIFs that were used in standard IPA installation. An example: install/updates/30-s4u2proxy.update Since you are implementing a new feature that is not present on already installed IPA servers, I think the best approach would be to implement an install script "ipa-dhcp-install" (analogous to install/tools/ipa-dns-install) which could be used to optionally install this feature to running IPA server. This script would do all the needed set up and add the necessary DS indices via LDIFs as I described in case 1). HTH, Martin From mkosek at redhat.com Tue May 15 06:56:38 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 May 2012 08:56:38 +0200 Subject: [Freeipa-devel] [PATCH] 1013 implement permission/aci find by subtree In-Reply-To: <4FB17D17.6020602@redhat.com> References: <4FAD77EB.3040709@redhat.com> <1336986285.4344.52.camel@balmora.brq.redhat.com> <4FB17D17.6020602@redhat.com> Message-ID: <1337064998.10688.16.camel@balmora.brq.redhat.com> On Mon, 2012-05-14 at 17:45 -0400, Rob Crittenden wrote: > Martin Kosek wrote: > > On Fri, 2012-05-11 at 16:34 -0400, Rob Crittenden wrote: > >> permission-find --subtree wasn't implemented so always returned all > >> entries (the option was ignored). > >> > >> rob > > > > I found the following 2 issues: > > > > 1) The following piece of code is over-complicated: > > > > + found = False > > + if kw['subtree'] == target: > > + found = True > > + if not found: > > + try: > > + results.remove(a) > > + except ValueError: > > + pass > > > > This would simpler and more readable: > > + if kw['subtree'] != target: > > + try: > > + results.remove(a) > > + except ValueError: > > + pass > > > > 2) I know we don't validate the subtree expression, but I wonder if we > > shouldn't make the subtree comparing at least case insensitive as DN is > > also not case sensitive. > > > > Martin > > > > Yeah, much simpler. Fixed both. > > rob ACK. Pushed to master. Martin From william at firstyear.id.au Tue May 15 07:07:38 2012 From: william at firstyear.id.au (William Brown) Date: Tue, 15 May 2012 16:37:38 +0930 Subject: [Freeipa-devel] Adding indices and permissions to FreeIPA In-Reply-To: <1337064693.10688.15.camel@balmora.brq.redhat.com> References: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> <1337064693.10688.15.camel@balmora.brq.redhat.com> Message-ID: <4FB200BA.70803@firstyear.id.au> On 15/05/12 16:21, Martin Kosek wrote: I think the best approach would be to implement > an install script "ipa-dhcp-install" (analogous to > install/tools/ipa-dns-install) which could be used to optionally install > this feature to running IPA server. This script would do all the needed > set up and add the necessary DS indices via LDIFs as I described in case > 1). I have already created this script, and was planning to do as you say. I'll add the index creations into this, and just make note of this. Additionally, would you use the same approach for adding aci's and groups into cn=pbac for this feature? -- Sincerely, William Brown pgp.mit.edu http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 945 bytes Desc: OpenPGP digital signature URL: From mkosek at redhat.com Tue May 15 07:20:29 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 May 2012 09:20:29 +0200 Subject: [Freeipa-devel] Adding indices and permissions to FreeIPA In-Reply-To: <4FB200BA.70803@firstyear.id.au> References: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> <1337064693.10688.15.camel@balmora.brq.redhat.com> <4FB200BA.70803@firstyear.id.au> Message-ID: <1337066429.10688.17.camel@balmora.brq.redhat.com> On Tue, 2012-05-15 at 16:37 +0930, William Brown wrote: > On 15/05/12 16:21, Martin Kosek wrote: > I think the best approach would be to implement > > an install script "ipa-dhcp-install" (analogous to > > install/tools/ipa-dns-install) which could be used to optionally install > > this feature to running IPA server. This script would do all the needed > > set up and add the necessary DS indices via LDIFs as I described in case > > 1). > > I have already created this script, and was planning to do as you say. > I'll add the index creations into this, and just make note of this. Great! > > Additionally, would you use the same approach for adding aci's and > groups into cn=pbac for this feature? > I would. ipa-dns-install takes the same approach in install/share/dns.ldif. Martin From mkosek at redhat.com Tue May 15 07:55:55 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 May 2012 09:55:55 +0200 Subject: [Freeipa-devel] [PATCH] 0050 Fail on unknown Command options In-Reply-To: <4FB0FEDB.4020806@redhat.com> References: <4F8E97F6.2070703@redhat.com> <1336392614.29911.13.camel@balmora.brq.redhat.com> <4FABA16A.10802@redhat.com> <1336982456.4344.28.camel@balmora.brq.redhat.com> <1336984817.4344.43.camel@balmora.brq.redhat.com> <4FB0FEDB.4020806@redhat.com> Message-ID: <1337068555.10688.20.camel@balmora.brq.redhat.com> On Mon, 2012-05-14 at 14:47 +0200, Petr Viktorin wrote: > The final part of rejecting unknown Command arguments: enable the > validation, add tests. > Also fix up things that were changed since the previous patches. > > https://fedorahosted.org/freeipa/ticket/2509 > The patch looks OK so far. I just found an error in permission/aci plugin - --subtree does not work when it matches a result: # ipa permission-find --subtree=foo --------------------- 0 permissions matched --------------------- ---------------------------- Number of entries returned 0 ---------------------------- ipa permission-find --subtree='ldap:///ipauniqueid=*,cn=hbac,dc=idm,dc=lab,dc=bos,dc=redhat,dc=Com' ipa: ERROR: Unknown option: subtree We should not pass **options to aci_show, it is too risky. There may be other places where we don't use an option-safe approach that we want to have fixed. Martin From mkosek at redhat.com Tue May 15 08:03:56 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 May 2012 10:03:56 +0200 Subject: [Freeipa-devel] [PATCH] 0051 Check for empty/single value parameters before calling callbacks In-Reply-To: <4FB10714.6010706@redhat.com> References: <4FB10714.6010706@redhat.com> Message-ID: <1337069036.10688.21.camel@balmora.brq.redhat.com> On Mon, 2012-05-14 at 15:22 +0200, Petr Viktorin wrote: > Pre-callbacks were called before a few validation steps, leading to > internal errors if the pre-callback relied on valid data. > > https://fedorahosted.org/freeipa/ticket/2701 > > Regression test included. > ACK. Pushed to master. Martin From pviktori at redhat.com Tue May 15 08:25:56 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 15 May 2012 10:25:56 +0200 Subject: [Freeipa-devel] [PATCH] 0048 Rework the CallbackInterface In-Reply-To: <4FABB28C.90208@redhat.com> References: <4FABB28C.90208@redhat.com> Message-ID: <4FB21314.6040205@redhat.com> On 05/10/2012 02:20 PM, Petr Viktorin wrote: > While investigating ticket 2674, I found several problems with our > implementation of the CallbackInterface ?? it required complicated > calling code, and would subtly break if command classes were > instantiated in different ways than they are currently. > > Here's my fix. See commit message for details. > Rebased to current master -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0048-02-Rework-the-CallbackInterface.patch Type: text/x-patch Size: 28689 bytes Desc: not available URL: From mkosek at redhat.com Tue May 15 08:29:57 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 May 2012 10:29:57 +0200 Subject: [Freeipa-devel] [PATCH] 0049 Disallow '<' and non-ASCII characters in the DM password In-Reply-To: <4FAD287F.8000000@redhat.com> References: <4FAD287F.8000000@redhat.com> Message-ID: <1337070597.10688.22.camel@balmora.brq.redhat.com> On Fri, 2012-05-11 at 16:55 +0200, Petr Viktorin wrote: > https://fedorahosted.org/freeipa/ticket/2675 > > I've tested all ASCII non-alphanumeric characters that weren't blocked > already. With all except for '<' I've succeeded. > Non-ASCII characters also don't work in passwords. (Not that it'd be a > good idea to use those.) > > ACK. Pushed to master. Martin From mkosek at redhat.com Tue May 15 08:39:39 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 May 2012 10:39:39 +0200 Subject: [Freeipa-devel] [PATCH] 137 Instructions to generate cert use certutil instead of openssl In-Reply-To: <4FACFA12.80402@redhat.com> References: <4FACFA12.80402@redhat.com> Message-ID: <1337071179.10688.24.camel@balmora.brq.redhat.com> On Fri, 2012-05-11 at 13:37 +0200, Petr Vobornik wrote: > Instructions to generate certificate were changed. Now they use certutil > instead of openssl. In the example is also used option for specifying > key size. > > https://fedorahosted.org/freeipa/ticket/2725 ACK, the new procedure works fine. Pushed to master. Martin From mkosek at redhat.com Tue May 15 08:45:28 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 15 May 2012 10:45:28 +0200 Subject: [Freeipa-devel] [PATCH] 257 Fix python Requires in Fedora 17 build In-Reply-To: <4FAA662F.7010106@redhat.com> References: <1336146344.2581.18.camel@balmora.brq.redhat.com> <4FAA662F.7010106@redhat.com> Message-ID: <1337071528.10688.25.camel@balmora.brq.redhat.com> On Wed, 2012-05-09 at 14:42 +0200, Ondrej Hamada wrote: > On 05/04/2012 05:45 PM, Martin Kosek wrote: > > This one actually took me some time to track it down (details are in a > > patch description). To check the result, simply build freeipa on Fedora > > 17 with "make rpms", install rpms on the machine and check Requires of > > freeipa-admintools package: > > > > $ rpm -qR freeipa-admintools > > > > Before the patch, there was a requirement for "/bin/python" which > > effectively blocked an update of python package until freeipa packages > > were removed. > > > > With this patch, there should be a correct requirement for > > "/usr/bin/python" and python updates will work again - yay. > > > > Our newest freeipa package on F-17 (2.2.0-1) is not affected, koji F-17 > > build root may have a different $PATH which translates "python" to > > "/usr/bin/python" and not "/bin/python". > > > > Martin > > > > > > _______________________________________________ > > Freeipa-devel mailing list > > Freeipa-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/freeipa-devel > works as proposed, ACK > Pushed to master. Martin From pvoborni at redhat.com Tue May 15 10:57:09 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 15 May 2012 12:57:09 +0200 Subject: [Freeipa-devel] [PATCH] 135 Host page fixed to work with disabled DNS support In-Reply-To: References: <4FA3C92F.7070206@redhat.com> Message-ID: <4FB23685.1010000@redhat.com> On 05/11/2012 06:46 PM, JR Aquino wrote: > On May 4, 2012, at 5:18 AM, Petr Vobornik wrote: > >> When DNS support was disabled there were following errors in Web UI: >> 1) Host details page was not filled with data >> 2) Host adder dialog was broken -> unusable >> 3) DNS tab was displayed in navigation >> 8><---------------------- > > Patch tested and works as advertised! > > ACK Pushed to master, ipa-2-2 -- Petr Vobornik From pvoborni at redhat.com Tue May 15 11:09:35 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 15 May 2012 13:09:35 +0200 Subject: [Freeipa-devel] [PATCH] 136 Correction of nested search facets tab labels In-Reply-To: <4FB13C34.6040007@redhat.com> References: <4FAB988C.5020905@redhat.com> <4FB13C34.6040007@redhat.com> Message-ID: <4FB2396F.8070006@redhat.com> On 05/14/2012 07:09 PM, Endi Sukma Dewata wrote: > On 5/10/2012 5:29 AM, Petr Vobornik wrote: >> Nested search facets were using 'search' tab label instead of their >> nested entity name. >> >> This patch is fixing that regression. >> >> https://fedorahosted.org/freeipa/ticket/2744 > > ACK. > Pushed to master. -- Petr Vobornik From pvoborni at redhat.com Tue May 15 11:09:46 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 15 May 2012 13:09:46 +0200 Subject: [Freeipa-devel] [PATCHES] 124-132 Inconsistent ways to show/change entry status In-Reply-To: <4FB13C05.7090005@redhat.com> References: <4FA137C4.7000101@redhat.com> <4FA285D0.90301@redhat.com> <4FA287A0.2070107@redhat.com> <4FA883BA.8080803@redhat.com> <4FABB239.2040500@redhat.com> <4FB13C05.7090005@redhat.com> Message-ID: <4FB2397A.8050800@redhat.com> On 05/14/2012 07:08 PM, Endi Sukma Dewata wrote: > On 5/10/2012 7:19 AM, Petr Vobornik wrote: >> Updated patch attached. See comments below. >> >> On 05/08/2012 04:23 AM, Endi Sukma Dewata wrote: >>> Try adding a very long DNS zone, then open the zone. Compare the >>> breadcrumbs in the DNS Resource Records page and in the Settings page, >>> in my case the second one is a bit longer. I think the length of the >>> breadcrumb should be calculated differently. >> >> I remade how breadcrumb is limited. Now it tries to use as much space >> available with keeping low complexity level of calculation. It still >> uses an average but now the average is calculated in two phases in order >> to neglect a presence of short keys. > > ACK. > Pushed to master. -- Petr Vobornik From pviktori at redhat.com Tue May 15 11:35:55 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 15 May 2012 13:35:55 +0200 Subject: [Freeipa-devel] [PATCH] 0050 Fail on unknown Command options In-Reply-To: <1337068555.10688.20.camel@balmora.brq.redhat.com> References: <4F8E97F6.2070703@redhat.com> <1336392614.29911.13.camel@balmora.brq.redhat.com> <4FABA16A.10802@redhat.com> <1336982456.4344.28.camel@balmora.brq.redhat.com> <1336984817.4344.43.camel@balmora.brq.redhat.com> <4FB0FEDB.4020806@redhat.com> <1337068555.10688.20.camel@balmora.brq.redhat.com> Message-ID: <4FB23F9B.10000@redhat.com> On 05/15/2012 09:55 AM, Martin Kosek wrote: > On Mon, 2012-05-14 at 14:47 +0200, Petr Viktorin wrote: >> The final part of rejecting unknown Command arguments: enable the >> validation, add tests. >> Also fix up things that were changed since the previous patches. >> >> https://fedorahosted.org/freeipa/ticket/2509 >> > > The patch looks OK so far. I just found an error in permission/aci > plugin - --subtree does not work when it matches a result: > > # ipa permission-find --subtree=foo > --------------------- > 0 permissions matched > --------------------- > ---------------------------- > Number of entries returned 0 > ---------------------------- > > ipa permission-find > --subtree='ldap:///ipauniqueid=*,cn=hbac,dc=idm,dc=lab,dc=bos,dc=redhat,dc=Com' > ipa: ERROR: Unknown option: subtree Attaching fixed patch. > We should not pass **options to aci_show, it is too risky. There may be > other places where we don't use an option-safe approach that we want to > have fixed. We shouldn't really pass **options to any command; listing everything explicitly would be much safer. Unfortunately, in a lot of cases where commands call other commands, it's currently done this way. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0050-02-Fail-on-unknown-Command-options.patch Type: text/x-patch Size: 16112 bytes Desc: not available URL: From pviktori at redhat.com Tue May 15 12:02:58 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 15 May 2012 14:02:58 +0200 Subject: [Freeipa-devel] [PATCH] 260 Replace DNS client based on acutil with python-dns In-Reply-To: <1336755130.19268.12.camel@priserak> References: <1336755130.19268.12.camel@priserak> Message-ID: <4FB245F2.6000500@redhat.com> On 05/11/2012 06:52 PM, Martin Kosek wrote: > python-dns is very feature-rich and it can help us a lot with our DNS > related code. This patch does the first step, i.e. replaces acutil use > with python-dns, which is more convenient to use as you will see in the > patch. More integration will follow in the future. > > I send this patch rather early, so that I can get responses to this > patch early and also so that we are able to catch issues in a safe > distance from the next release. > --- > IPA client and server tool set used authconfig acutil module to > for client DNS operations. This is not optimal DNS interface for > several reasons: > - does not provide native Python object oriented interface > but but rather C-like interface based on functions and > structures which is not easy to use and extend > - acutil is not meant to be used by third parties besides > authconfig and thus can break without notice > > Replace the acutil with python-dns package which has a feature rich > interface for dealing with all different aspects of DNS including > DNSSEC. The main target of this patch is to replace all uses of > acutil DNS library with a use python-dns. In most cases, even > though the larger parts of the code are changed, the actual > functionality is changed only in the following cases: > - redundant DNS checks were removed from verify_fqdn function > in installutils to make the whole DNS check simpler and > less error-prone. Logging was improves for the remaining > checks > - improved logging for ipa-client-install DNS discovery > > https://fedorahosted.org/freeipa/ticket/2730 Also relevant: https://fedorahosted.org/freeipa/ticket/1837 There is a forgotten acutil reference in a comment in install/tools/ipa-dns-install:226 I did find some style issues/suggestions. Stop me if the bikeshedding is too useless :) [...] > diff --git a/ipa-client/ipaclient/ipadiscovery.py b/ipa-client/ipaclient/ipadiscovery.py > index 86bef28b2d7fdfc8111b493bddec7ac6888f021a..800b4ef9a6396884a1fe2e93bdf94e948f35c57e 100644 > --- a/ipa-client/ipaclient/ipadiscovery.py > +++ b/ipa-client/ipaclient/ipadiscovery.py > @@ -20,10 +20,12 @@ > import socket > import os > from ipapython.ipa_log_manager import * > -import ipapython.dnsclient > import tempfile > import ldap > from ldap import LDAPError > +from dns import resolver, rdatatype > +from dns.exception import DNSException > + > from ipapython.ipautil import run, CalledProcessError, valid_ip, get_ipa_basedn, \ > realm_to_suffix, format_netloc, parse_items > > @@ -311,81 +313,89 @@ class IPADiscovery: > > > def ipadnssearchldap(self, tdomain): > - servers = "" > - rserver = "" > + server = "" > > - qname = "_ldap._tcp."+tdomain > - # terminate the name > - if not qname.endswith("."): > - qname += "." > - results = ipapython.dnsclient.query(qname, ipapython.dnsclient.DNS_C_IN, ipapython.dnsclient.DNS_T_SRV) > + qname = "_ldap._tcp." + tdomain > > - for result in results: > - if result.dns_type == ipapython.dnsclient.DNS_T_SRV: > - rserver = result.rdata.server.rstrip(".") > - if result.rdata.port and result.rdata.port != 389: > - rserver += ":" + str(result.rdata.port) > - if servers: > - servers += "," + rserver > - else: > - servers = rserver > - break > + root_logger.debug("Search DNS for SRV record of %s", qname) > > - return servers > + try: > + answers = resolver.query(qname, rdatatype.SRV) > + except DNSException, e: > + root_logger.debug("DNS record not found: %s", e.__class__.__name__) > + answers = [] > + > + for answer in answers: > + root_logger.debug("DNS record found: %s", answer) > + server = str(answer.target).rstrip(".") > + if answer.port != 389: > + server += ":" + str(answer.port) > + break > + return server > > def ipadnssearchntp(self, tdomain): > - servers = "" > - rserver = "" > + server = "" > > - qname = "_ntp._udp."+tdomain > - # terminate the name > - if not qname.endswith("."): > - qname += "." > - results = ipapython.dnsclient.query(qname, ipapython.dnsclient.DNS_C_IN, ipapython.dnsclient.DNS_T_SRV) > + qname = "_ntp._udp." + tdomain > > - for result in results: > - if result.dns_type == ipapython.dnsclient.DNS_T_SRV: > - rserver = result.rdata.server.rstrip(".") > - if servers: > - servers += "," + rserver > - else: > - servers = rserver > - break > + root_logger.debug("Search DNS for SRV record of %s", qname) > > - return servers > + try: > + answers = resolver.query(qname, rdatatype.SRV) > + except DNSException, e: > + root_logger.debug("DNS record not found: %s", e.__class__.__name__) > + answers = [] > + > + for answer in answers: > + root_logger.debug("DNS record found: %s", answer) > + server = str(answer.target).rstrip(".") > + break > + > + return server These function ars mostly the same, except ipadnssearchntp uses "_ntp" instead of "_ldap", and it doesn't take the port into account (which it probably should?). The last part of ipadnssearchkrb is also very similar. Please merge the common parts. > def ipadnssearchkrb(self, tdomain): [...] > diff --git a/ipapython/config.py b/ipapython/config.py > index d4c724dc9ac754cb221fe60d7c13bd0c716dd296..e06f51a318a2b20c53a8c6933c43d58c34075407 100644 > --- a/ipapython/config.py > +++ b/ipapython/config.py [...] > @@ -153,7 +155,7 @@ def __parse_config(discover_server = True): > try: > s = p.get("global", "xmlrpc_uri") > server = urlparse.urlsplit(s) > - config.default_server.extend(server.netloc) > + config.default_server.append(server.netloc) This is unrelated to the DNS library replacement, right? It should go in a separate patch, in case e.g. the big patch gets reverted. [...] > + try: > + servers = resolver.query(name, rdatatype.SRV) > + domain_ok = True > + except DNSException: > + pass > + > + if not domain_ok: > + # try cycling on domain components of FQDN This can be just: try: servers = resolver.query(name, rdatatype.SRV) domain_ok = True except DNSException: # try cycling on domain components of FQDN The extra if and flag variable just makes it less clear. > + try: > + domain = dns.name.from_text(socket.getfqdn()) > + except DNSException: > return False > - dom_name = dom_name[tok+1:] > - name = "_ldap._tcp." + dom_name + "." > - rs = ipapython.dnsclient.query(name, ipapython.dnsclient.DNS_C_IN, ipapython.dnsclient.DNS_T_SRV) > - rl = len(rs) > > - config.default_domain = dom_name > + while not domain_ok: > + domain = domain.parent() > + > + if str(domain) == '.': > + return False > + name = "_ldap._tcp.%s" % domain > + try: > + servers = resolver.query(name, rdatatype.SRV) > + domain_ok = True > + except DNSException: > + pass > + > + config.default_domain = str(domain).rstrip(".") Drop the `domain_ok` variable and just use a while True/break. The code flow would be more clear that way, IMO. [...] > diff --git a/ipapython/ipautil.py b/ipapython/ipautil.py > index 4a9db11e23c1b9ab76c9fce9150bc1546426452f..71be39132ea40172595e026a9815acc58d29f4ff 100644 > --- a/ipapython/ipautil.py > +++ b/ipapython/ipautil.py [...] > +def is_host_resolvable(fqdn): > + found = False > + for rdtype in (rdatatype.A, rdatatype.AAAA): > + try: > + resolver.query(fqdn, rdtype) > + found = True > + except DNSException: > + continue > + > + return found > + This seems too complicated; why not return directly? for rdtype in rdatatype.A, rdatatype.AAAA: try: resolver.query(fqdn, rdtype) except DNSException: pass else: return True return False > def get_ipa_basedn(conn): > """ > Get base DN of IPA suffix in given LDAP server. > diff --git a/ipaserver/install/installutils.py b/ipaserver/install/installutils.py > index 3e7ae41b5fdbc11353e43a63424f19fbc331435a..76b54c94c9d1dad169545dd30c0b9a926f19a3c6 100644 > --- a/ipaserver/install/installutils.py > +++ b/ipaserver/install/installutils.py [...] > + > + # list of verified addresses to prevent multiple searches for the same address > + verified = [] Use a set for this. > for a in hostaddr: > - if a[4][0] == '127.0.0.1' or a[4][0] == '::1': > - raise HostForwardLookupError("The IPA Server hostname must not resolve to localhost (%s). A routable IP address must be used. Check /etc/hosts to see if %s is an alias for %s" % (a[4][0], host_name, a[4][0])) > + address = a[4][0] > + if address in verified: > + continue [...] -- Petr? From atkac at redhat.com Tue May 15 12:32:01 2012 From: atkac at redhat.com (Adam Tkac) Date: Tue, 15 May 2012 14:32:01 +0200 Subject: [Freeipa-devel] [PATCH 0020] Separate LDAP result from LDAP connection, fix deadlock. In-Reply-To: <4FB11A5A.8000804@redhat.com> References: <4FA7C4C3.7020202@redhat.com> <20120511102654.GA2785@redhat.com> <4FB11A5A.8000804@redhat.com> Message-ID: <20120515123139.GA2688@redhat.com> On Mon, May 14, 2012 at 04:44:42PM +0200, Petr Spacek wrote: > On 05/11/2012 12:26 PM, Adam Tkac wrote: > >On Mon, May 07, 2012 at 02:49:07PM +0200, Petr Spacek wrote: > >>Hello, > >> > >>this patch fixes https://fedorahosted.org/bind-dyndb-ldap/ticket/66: > >>Plugin deadlocks during new zone load when connections == 1. > >> > >>It fixes structural problem, when LDAP query result was tied with > >>LDAP connection up. It wasn't possible to release connection and > >>work with query result after that. > >>Described deadlock is consequence of this problematic design. > >> > >>Now LDAP connection is separated from LDAP result. Next planed patch > >>will avoid "manual" connection management, so possibility of > >>deadlock should be next to zero. > >> > >>Petr^2 Spacek > > > >Hello Peter, > > > >good work, please check my comments below. > > > >Regards, Adam > > > >> From 8ee1fd607531ef71369e99c9228456baea45b65d Mon Sep 17 00:00:00 2001 > >>From: Petr Spacek > >>Date: Mon, 7 May 2012 12:51:09 +0200 > >>Subject: [PATCH] Separate LDAP result from LDAP connection, fix deadlock. > >> https://fedorahosted.org/bind-dyndb-ldap/ticket/66 > >> Signed-off-by: Petr Spacek > > Hello Adam, > > thanks for ideas/improvements! > > Reworked patch is attached. I did all proposed changes except this one: > > @ ldap_psearch_watcher: > >> restart: > (... snip ...) > >> soft_err: > >>- > >>- ldap_msgfree(conn->result); > >>- ldap_entrylist_destroy(conn->mctx, > >>- &conn->ldap_entries); > >>+ ; > > > >Empty label "soft_err:" is useless, please remove it and use "continue;" on > >appropriate places; > > I think "continue" in this place can lead to memory leak, so I > removed soft_err by other way. Thanks for the patch, now it looks fine to me, except that it doesn't apply on the current master: [atkac at drtic bind-dyndb-ldap]$ git am ../bind-dyndb-ldap-pspacek-0020-2-Separate-LDAP-result-from-LDAP-connection-fix-deadlo.patch Applying: Separate LDAP result from LDAP connection, fix deadlock. https://fedorahosted.org/bind-dyndb-ldap/ticket/66 Signed-off-by: Petr Spacek error: patch failed: src/ldap_helper.c:271 error: src/ldap_helper.c: patch does not apply Patch failed at 0001 Separate LDAP result from LDAP connection, fix deadlock. https://fedorahosted.org/bind-dyndb-ldap/ticket/66 Signed-off-by: Petr Spacek When you have resolved this problem run "git am --resolved". If you would prefer to skip this patch, instead run "git am --skip". To restore the original branch and stop patching run "git am --abort". Please rebase the patch and then push it, you don't have to resend it here. Regards, Adam > From 4c8c8c1ec7777217d3219a0380f6baec4b41248d Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Mon, 7 May 2012 12:51:09 +0200 > Subject: [PATCH] Separate LDAP result from LDAP connection, fix deadlock. > https://fedorahosted.org/bind-dyndb-ldap/ticket/66 > Signed-off-by: Petr Spacek > > --- > src/ldap_helper.c | 269 ++++++++++++++++++++++++++++++++--------------------- > 1 files changed, 162 insertions(+), 107 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index 304c37296f8f3a428c4c72b45fe4154b2c9e954c..dae3e28804b602ca91d813c4d68d258e7065608f 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -108,6 +108,7 @@ > * must acquire the semaphore and the lock. > */ > > +typedef struct ldap_qresult ldap_qresult_t; > typedef struct ldap_connection ldap_connection_t; > typedef struct ldap_pool ldap_pool_t; > typedef struct ldap_auth_pair ldap_auth_pair_t; > @@ -188,28 +189,28 @@ struct ldap_connection { > ld_string_t *query_string; > > LDAP *handle; > - LDAPMessage *result; > LDAPControl *serverctrls[2]; /* psearch/NULL or NULL/NULL */ > int msgid; > > /* Parsing. */ > isc_lex_t *lex; > isc_buffer_t rdata_target; > unsigned char *rdata_target_mem; > > - /* Cache. */ > - ldap_entrylist_t ldap_entries; > - > /* For reconnection logic. */ > isc_time_t next_reconnect; > unsigned int tries; > +}; > > - /* Temporary stuff. */ > - LDAPMessage *entry; > - BerElement *ber; > - char *attribute; > - char **values; > - char *dn; > +/** > + * Result from single LDAP query. > + */ > +struct ldap_qresult { > + isc_mem_t *mctx; > + LDAPMessage *result; > + > + /* Cache. */ > + ldap_entrylist_t ldap_entries; > }; > > /* > @@ -271,9 +272,10 @@ static int handle_connection_error(ldap_instance_t *ldap_inst, > ldap_connection_t *ldap_conn, isc_boolean_t force, > isc_result_t *result); > static isc_result_t ldap_query(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn, > - const char *base, > - int scope, char **attrs, int attrsonly, const char *filter, ...); > -static void ldap_query_free(ldap_connection_t *ldap_conn); > + ldap_qresult_t **ldap_qresultp, const char *base, int scope, char **attrs, > + int attrsonly, const char *filter, ...); > +static isc_result_t ldap_query_create(isc_mem_t *mctx, ldap_qresult_t **ldap_qresultp); > +static void ldap_query_free(isc_boolean_t prepare_reuse, ldap_qresult_t **ldap_qresultp); > > /* Functions for writing to LDAP. */ > static isc_result_t ldap_modify_do(ldap_connection_t *ldap_conn, const char *dn, > @@ -1095,6 +1097,8 @@ refresh_zones_from_ldap(ldap_instance_t *ldap_inst) > { > isc_result_t result = ISC_R_SUCCESS; > ldap_connection_t *ldap_conn = NULL; > + ldap_qresult_t *ldap_config_qresult = NULL; > + ldap_qresult_t *ldap_zones_qresult = NULL; > int zone_count = 0; > ldap_entry_t *entry; > dns_rbt_t *rbt = NULL; > @@ -1119,31 +1123,31 @@ refresh_zones_from_ldap(ldap_instance_t *ldap_inst) > > log_debug(2, "refreshing list of zones for %s", ldap_inst->db_name); > > + /* Query for configuration and zones from LDAP and release LDAP connection > + * before processing them. It prevents deadlock in situations where > + * ldap_parse_zoneentry() requests another connection. */ > CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); > - > - CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(ldap_inst->base), > + CHECK(ldap_query(ldap_inst, ldap_conn, &ldap_config_qresult, str_buf(ldap_inst->base), > LDAP_SCOPE_SUBTREE, config_attrs, 0, > "(objectClass=idnsConfigObject)")); > - > - for (entry = HEAD(ldap_conn->ldap_entries); > - entry != NULL; > - entry = NEXT(entry, link)) { > - CHECK(ldap_parse_configentry(entry, ldap_inst)); > - } > - > - ldap_query_free(ldap_conn); > - > - CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(ldap_inst->base), > + CHECK(ldap_query(ldap_inst, ldap_conn, &ldap_zones_qresult, str_buf(ldap_inst->base), > LDAP_SCOPE_SUBTREE, zone_attrs, 0, > "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); > + ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > + > + for (entry = HEAD(ldap_config_qresult->ldap_entries); > + entry != NULL; > + entry = NEXT(entry, link)) { > + CHECK(ldap_parse_configentry(entry, ldap_inst)); > + } > > /* > * Create RB-tree with all zones stored in LDAP for cross check > * with registered zones in plugin. > */ > CHECK(dns_rbt_create(ldap_inst->mctx, NULL, NULL, &rbt)); > > - for (entry = HEAD(ldap_conn->ldap_entries); > + for (entry = HEAD(ldap_zones_qresult->ldap_entries); > entry != NULL; > entry = NEXT(entry, link)) { > > @@ -1232,6 +1236,8 @@ cleanup: > if (invalidate_nodechain) > dns_rbtnodechain_invalidate(&chain); > > + ldap_query_free(ISC_FALSE, &ldap_config_qresult); > + ldap_query_free(ISC_FALSE, &ldap_zones_qresult); > ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > > log_debug(2, "finished refreshing list of zones"); > @@ -1387,6 +1393,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > { > isc_result_t result; > ldap_connection_t *ldap_conn = NULL; > + ldap_qresult_t *ldap_qresult = NULL; > ldap_entry_t *entry; > ld_string_t *string = NULL; > ldapdb_node_t *node; > @@ -1403,15 +1410,15 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > CHECK(str_new(mctx, &string)); > CHECK(dnsname_to_dn(ldap_inst->zone_register, name, string)); > > - CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(string), > + CHECK(ldap_query(ldap_inst, ldap_conn, &ldap_qresult, str_buf(string), > LDAP_SCOPE_SUBTREE, NULL, 0, "(objectClass=idnsRecord)")); > > - if (EMPTY(ldap_conn->ldap_entries)) { > + if (EMPTY(ldap_qresult->ldap_entries)) { > result = ISC_R_NOTFOUND; > goto cleanup; > } > > - for (entry = HEAD(ldap_conn->ldap_entries); > + for (entry = HEAD(ldap_qresult->ldap_entries); > entry != NULL; > entry = NEXT(entry, link)) { > node = NULL; > @@ -1444,6 +1451,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > result = ISC_R_SUCCESS; > > cleanup: > + ldap_query_free(ISC_FALSE, &ldap_qresult); > ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > str_destroy(&string); > > @@ -1456,6 +1464,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > { > isc_result_t result; > ldap_connection_t *ldap_conn = NULL; > + ldap_qresult_t *ldap_qresult = NULL; > ldap_entry_t *entry; > ld_string_t *string = NULL; > ldap_cache_t *cache; > @@ -1478,15 +1487,15 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > CHECK(dnsname_to_dn(ldap_inst->zone_register, name, string)); > > CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); > - CHECK(ldap_query(ldap_inst, ldap_conn, str_buf(string), > + CHECK(ldap_query(ldap_inst, ldap_conn, &ldap_qresult, str_buf(string), > LDAP_SCOPE_BASE, NULL, 0, "(objectClass=idnsRecord)")); > > - if (EMPTY(ldap_conn->ldap_entries)) { > + if (EMPTY(ldap_qresult->ldap_entries)) { > result = ISC_R_NOTFOUND; > goto cleanup; > } > > - for (entry = HEAD(ldap_conn->ldap_entries); > + for (entry = HEAD(ldap_qresult->ldap_entries); > entry != NULL; > entry = NEXT(entry, link)) { > if (ldap_parse_rrentry(mctx, entry, ldap_conn, > @@ -1504,6 +1513,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > result = ISC_R_NOTFOUND; > > cleanup: > + ldap_query_free(ISC_FALSE, &ldap_qresult); > ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > str_destroy(&string); > > @@ -1601,16 +1611,24 @@ cleanup: > return result; > } > > +/** > + * @param ldap_conn A LDAP connection structure obtained via ldap_get_connection(). > + * @param ldap_qresult New ldap_qresult structure will be allocated and pointer > + * to it will be returned through this parameter. The result > + * has to be freed by caller via ldap_query_free(). > + */ > static isc_result_t > ldap_query(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn, > - const char *base, int scope, char **attrs, > + ldap_qresult_t **ldap_qresultp, const char *base, int scope, char **attrs, > int attrsonly, const char *filter, ...) > { > va_list ap; > isc_result_t result; > + ldap_qresult_t *ldap_qresult = NULL; > int cnt; > > REQUIRE(ldap_conn != NULL); > + REQUIRE(ldap_qresultp != NULL && *ldap_qresultp == NULL); > > va_start(ap, filter); > str_vsprintf(ldap_conn->query_string, filter, ap); > @@ -1629,62 +1647,96 @@ ldap_query(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn, > return result; > } > > + CHECK(ldap_query_create(ldap_inst->mctx, &ldap_qresult)); > + > do { > int ret; > > ret = ldap_search_ext_s(ldap_conn->handle, base, scope, > str_buf(ldap_conn->query_string), > attrs, attrsonly, NULL, NULL, NULL, > - LDAP_NO_LIMIT, &ldap_conn->result); > + LDAP_NO_LIMIT, &(ldap_qresult->result)); > if (ret == 0) { > ldap_conn->tries = 0; > - cnt = ldap_count_entries(ldap_conn->handle, ldap_conn->result); > + cnt = ldap_count_entries(ldap_conn->handle, ldap_qresult->result); > log_debug(2, "entry count: %d", cnt); > > result = ldap_entrylist_create(ldap_conn->mctx, > ldap_conn->handle, > - ldap_conn->result, > - &ldap_conn->ldap_entries); > + ldap_qresult->result, > + &ldap_qresult->ldap_entries); > if (result != ISC_R_SUCCESS) { > log_error("failed to save LDAP query results"); > - return result; > } > > - return ISC_R_SUCCESS; > + break; > } > + /* Special case is LDAP_NO_SUCH_OBJECT: handle_connection_error() will > + * set result = ISC_R_SUCCESS and break the cycle. */ > } while (handle_connection_error(ldap_inst, ldap_conn, ISC_FALSE, &result)); > > +cleanup: > + if (result == ISC_R_SUCCESS) > + *ldap_qresultp = ldap_qresult; > + else > + ldap_query_free(ISC_FALSE, &ldap_qresult); > return result; > } > > +/** > + * Allocate and initialize new ldap_qresult structure. > + * @param[out] ldap_qresultp Newly allocated ldap_qresult structure. > + * @return ISC_R_SUCCESS or ISC_R_NOMEMORY (from CHECKED_MEM_GET_PTR) > + */ > +static isc_result_t > +ldap_query_create(isc_mem_t *mctx, ldap_qresult_t **ldap_qresultp) { > + ldap_qresult_t *ldap_qresult = NULL; > + isc_result_t result; > + > + CHECKED_MEM_GET_PTR(mctx, ldap_qresult); > + ldap_qresult->mctx = mctx; > + ldap_qresult->result = NULL; > + INIT_LIST(ldap_qresult->ldap_entries); > + > + *ldap_qresultp = ldap_qresult; > + return ISC_R_SUCCESS; > + > +cleanup: > + return result; > +} > + > +/** > + * Free LDAP query result. Can free whole structure or internal parts only. > + * Freeing internal parts is suitable before reusing the structure. > + * @param[in] prepare_reuse ISC_TRUE implies freeing internal parts, > + * but not the whole structure. > + * @param[in,out] ldap_qresultp Pointer to freed query. Will be set to NULL > + * if prepare_reuse == ISC_FALSE. > + */ > static void > -ldap_query_free(ldap_connection_t *ldap_conn) > +ldap_query_free(isc_boolean_t prepare_reuse, ldap_qresult_t **ldap_qresultp) > { > - if (ldap_conn == NULL) > + ldap_qresult_t *qresult; > + REQUIRE(ldap_qresultp != NULL); > + > + qresult = *ldap_qresultp; > + > + if (qresult == NULL) > return; > > - if (ldap_conn->dn) { > - ldap_memfree(ldap_conn->dn); > - ldap_conn->dn = NULL; > - } > - if (ldap_conn->values) { > - ldap_value_free(ldap_conn->values); > - ldap_conn->values = NULL; > - } > - if (ldap_conn->attribute) { > - ldap_memfree(ldap_conn->attribute); > - ldap_conn->attribute = NULL; > - } > - if (ldap_conn->ber) { > - ber_free(ldap_conn->ber, 0); > - ldap_conn->ber = NULL; > - } > - if (ldap_conn->result) { > - ldap_msgfree(ldap_conn->result); > - ldap_conn->result = NULL; > + if (qresult->result) { > + ldap_msgfree(qresult->result); > + qresult->result = NULL; > } > > - ldap_entrylist_destroy(ldap_conn->mctx, &ldap_conn->ldap_entries); > + ldap_entrylist_destroy(qresult->mctx, &qresult->ldap_entries); > + > + if (prepare_reuse) { > + INIT_LIST(qresult->ldap_entries); > + } else { /* free whole structure */ > + SAFE_MEM_PUT_PTR(qresult->mctx, qresult); > + *ldap_qresultp = NULL; > + } > } > > /* FIXME: Tested with SASL/GSSAPI/KRB5 only */ > @@ -2229,6 +2281,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > isc_result_t result; > isc_mem_t *mctx = ldap_inst->mctx; > ldap_connection_t *ldap_conn = NULL; > + ldap_qresult_t *ldap_qresult = NULL; > ld_string_t *owner_dn = NULL; > LDAPMod *change[3] = { NULL }; > LDAPMod *change_ptr = NULL; > @@ -2255,12 +2308,12 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > } > > CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); > - CHECK(ldap_query(ldap_inst, ldap_conn, zone_dn, > + CHECK(ldap_query(ldap_inst, ldap_conn, &ldap_qresult, zone_dn, > LDAP_SCOPE_BASE, attrs, 0, > "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); > > /* only 0 or 1 active zone can be returned from query */ > - entry = HEAD(ldap_conn->ldap_entries); > + entry = HEAD(ldap_qresult->ldap_entries); > if (entry == NULL) { > log_debug(3, "Active zone %s not found", zone_dn); > result = ISC_R_NOTFOUND; > @@ -2392,14 +2445,14 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > char *owner_zone_dn_ptr = strstr(str_buf(owner_dn_ptr),", ") + 1; > > /* Get attribute "idnsAllowDynUpdate" for reverse zone or use default. */ > - ldap_query_free(ldap_conn); > + ldap_query_free(ISC_FALSE, &ldap_qresult); > zone_dyn_update = ldap_inst->dyn_update; > - CHECK(ldap_query(ldap_inst, ldap_conn, owner_zone_dn_ptr, > + CHECK(ldap_query(ldap_inst, ldap_conn, &ldap_qresult, owner_zone_dn_ptr, > LDAP_SCOPE_BASE, attrs, 0, > "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); > > /* Only 0 or 1 active zone can be returned from query. */ > - entry = HEAD(ldap_conn->ldap_entries); > + entry = HEAD(ldap_qresult->ldap_entries); > if (entry == NULL) { > log_debug(3, "Active zone %s not found", zone_dn); > result = ISC_R_NOTFOUND; > @@ -2485,6 +2538,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > } > > cleanup: > + ldap_query_free(ISC_FALSE, &ldap_qresult); > ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > str_destroy(&owner_dn_ptr); > str_destroy(&owner_dn); > @@ -2587,7 +2641,6 @@ ldap_pool_getconnection(ldap_pool_t *pool, ldap_connection_t ** conn) > > RUNTIME_CHECK(ldap_conn != NULL); > > - INIT_LIST(ldap_conn->ldap_entries); > *conn = ldap_conn; > > cleanup: > @@ -2607,7 +2660,6 @@ ldap_pool_putconnection(ldap_pool_t *pool, ldap_connection_t **conn) > if (ldap_conn == NULL) > return; > > - ldap_query_free(ldap_conn); > UNLOCK(&ldap_conn->lock); > semaphore_signal(&pool->conn_semaphore); > > @@ -2739,6 +2791,7 @@ update_action(isc_task_t *task, isc_event_t *event) > isc_result_t result ; > ldap_instance_t *inst = NULL; > ldap_connection_t *conn = NULL; > + ldap_qresult_t *ldap_qresult = NULL; > ldap_entry_t *entry; > isc_boolean_t delete = ISC_TRUE; > isc_mem_t *mctx; > @@ -2757,11 +2810,11 @@ update_action(isc_task_t *task, isc_event_t *event) > > CHECK(ldap_pool_getconnection(inst->pool, &conn)); > > - CHECK(ldap_query(inst, conn, pevent->dn, > + CHECK(ldap_query(inst, conn, &ldap_qresult, pevent->dn, > LDAP_SCOPE_BASE, attrs, 0, > "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); > > - for (entry = HEAD(conn->ldap_entries); > + for (entry = HEAD(ldap_qresult->ldap_entries); > entry != NULL; > entry = NEXT(entry, link)) { > delete = ISC_FALSE; > @@ -2779,6 +2832,7 @@ cleanup: > "Zones can be outdated, run `rndc reload`", > pevent->dn); > > + ldap_query_free(ISC_FALSE, &ldap_qresult); > ldap_pool_putconnection(inst->pool, &conn); > isc_mem_free(mctx, pevent->dbname); > isc_mem_free(mctx, pevent->dn); > @@ -2793,6 +2847,7 @@ update_config(isc_task_t *task, isc_event_t *event) > isc_result_t result ; > ldap_instance_t *inst = NULL; > ldap_connection_t *conn = NULL; > + ldap_qresult_t *ldap_qresult = NULL; > ldap_entry_t *entry; > isc_mem_t *mctx; > char *attrs[] = { > @@ -2810,14 +2865,14 @@ update_config(isc_task_t *task, isc_event_t *event) > > CHECK(ldap_pool_getconnection(inst->pool, &conn)); > > - CHECK(ldap_query(inst, conn, pevent->dn, > + CHECK(ldap_query(inst, conn, &ldap_qresult, pevent->dn, > LDAP_SCOPE_BASE, attrs, 0, > "(objectClass=idnsConfigObject)")); > > - if (EMPTY(conn->ldap_entries)) > - log_error("Config object can not be empty"); > + if (EMPTY(ldap_qresult->ldap_entries)) > + log_error("Config object can not be empty"); // WHY? > > - for (entry = HEAD(conn->ldap_entries); > + for (entry = HEAD(ldap_qresult->ldap_entries); > entry != NULL; > entry = NEXT(entry, link)) { > result = ldap_parse_configentry(entry, inst); > @@ -2832,6 +2887,7 @@ cleanup: > "Configuration can be outdated, run `rndc reload`", > pevent->dn); > > + ldap_query_free(ISC_FALSE, &ldap_qresult); > ldap_pool_putconnection(inst->pool, &conn); > isc_mem_free(mctx, pevent->dbname); > isc_mem_free(mctx, pevent->dn); > @@ -3071,6 +3127,7 @@ ldap_psearch_watcher(isc_threadarg_t arg) > { > ldap_instance_t *inst = (ldap_instance_t *)arg; > ldap_connection_t *conn; > + ldap_qresult_t *ldap_qresult = NULL; > struct timeval tv; > int ret, cnt; > isc_result_t result; > @@ -3108,6 +3165,8 @@ ldap_psearch_watcher(isc_threadarg_t arg) > ldap_connect(inst, conn, ISC_TRUE); > } > > + CHECK(ldap_query_create(conn->mctx, &ldap_qresult)); > + > restart: > /* Perform initial lookup */ > if (inst->psearch) { > @@ -3134,7 +3193,7 @@ restart: > > while (!inst->exiting) { > ret = ldap_result(conn->handle, conn->msgid, 0, &tv, > - &conn->result); > + &ldap_qresult->result); > > if (ret <= 0) { > int ok; > @@ -3153,58 +3212,54 @@ restart: > case LDAP_RES_SEARCH_ENTRY: > break; > default: > - log_debug(3, "Ignoring psearch msg with retcode %x", > - ret); > + log_debug(3, "Ignoring psearch msg with retcode %x", ret); > + break; > } > > conn->tries = 0; > - cnt = ldap_count_entries(conn->handle, conn->result); > + cnt = ldap_count_entries(conn->handle, ldap_qresult->result); > > if (cnt > 0) { > log_debug(3, "Got psearch updates (%d)", cnt); > - result = ldap_entrylist_append(conn->mctx, > + result = ldap_entrylist_append(ldap_qresult->mctx, > conn->handle, > - conn->result, > - &conn->ldap_entries); > - if (result != ISC_R_SUCCESS) { > + ldap_qresult->result, > + &ldap_qresult->ldap_entries); > + > + if (result == ISC_R_SUCCESS) { > + ldap_entry_t *entry; > + for (entry = HEAD(ldap_qresult->ldap_entries); > + entry != NULL; > + entry = NEXT(entry, link)) { > + LDAPControl **ctrls = NULL; > + ret = ldap_get_entry_controls(conn->handle, > + entry->ldap_entry, > + &ctrls); > + if (ret != LDAP_SUCCESS) { > + log_error("failed to extract controls " > + "from psearch update. Zones " > + "might be outdated, run " > + "`rndc reload`"); > + break; > + } > + psearch_update(inst, entry, ctrls); > + } > + } else { /* result != ISC_R_SUCCESS */ > /* > * Error means inconsistency of our zones > * data. > */ > log_error("ldap_psearch_watcher failed, zones " > "might be outdated. Run `rndc reload`"); > - goto soft_err; > } > - > - ldap_entry_t *entry; > - for (entry = HEAD(conn->ldap_entries); > - entry != NULL; > - entry = NEXT(entry, link)) { > - LDAPControl **ctrls = NULL; > - ret = ldap_get_entry_controls(conn->handle, > - entry->ldap_entry, > - &ctrls); > - if (ret != LDAP_SUCCESS) { > - log_error("failed to extract controls " > - "from psearch update. Zones " > - "might be outdated, run " > - "`rndc reload"); > - goto soft_err; > - } > - > - psearch_update(inst, entry, ctrls); > - } > -soft_err: > - > - ldap_msgfree(conn->result); > - ldap_entrylist_destroy(conn->mctx, > - &conn->ldap_entries); > } > + ldap_query_free(ISC_TRUE, &ldap_qresult); > } > > log_debug(1, "Ending ldap_psearch_watcher"); > > cleanup: > + ldap_query_free(ISC_FALSE, &ldap_qresult); > ldap_pool_putconnection(inst->pool, &conn); > > return (isc_threadresult_t)0; > -- > 1.7.7.6 > -- Adam Tkac, Red Hat, Inc. From simo at redhat.com Tue May 15 12:39:47 2012 From: simo at redhat.com (Simo Sorce) Date: Tue, 15 May 2012 08:39:47 -0400 Subject: [Freeipa-devel] Adding indices and permissions to FreeIPA In-Reply-To: <4FB200BA.70803@firstyear.id.au> References: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> <1337064693.10688.15.camel@balmora.brq.redhat.com> <4FB200BA.70803@firstyear.id.au> Message-ID: <1337085587.16840.19.camel@willson.li.ssimo.org> On Tue, 2012-05-15 at 16:37 +0930, William Brown wrote: > On 15/05/12 16:21, Martin Kosek wrote: > I think the best approach would be to implement > > an install script "ipa-dhcp-install" (analogous to > > install/tools/ipa-dns-install) which could be used to optionally install > > this feature to running IPA server. This script would do all the needed > > set up and add the necessary DS indices via LDIFs as I described in case > > 1). > > I have already created this script, and was planning to do as you say. > I'll add the index creations into this, and just make note of this. > > Additionally, would you use the same approach for adding aci's and > groups into cn=pbac for this feature? Hi William, do you have a public repo you are pushing your work to ? It would be nice to have early access so we can check your implementation is in line with FreeIPA. It will allow your contribution to get in more easily if we can comment early on around schema, DIT and other behavior you need to implement. An architecture document posted on freeipa.org would also be very welcome, feel free to ask for an account on the wiki so you can post stuff in there. Simo. -- Simo Sorce * Red Hat, Inc * New York From william at firstyear.id.au Tue May 15 13:29:26 2012 From: william at firstyear.id.au (William Brown) Date: Tue, 15 May 2012 22:59:26 +0930 Subject: [Freeipa-devel] Adding indices and permissions to FreeIPA In-Reply-To: <1337085587.16840.19.camel@willson.li.ssimo.org> References: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> <1337064693.10688.15.camel@balmora.brq.redhat.com> <4FB200BA.70803@firstyear.id.au> <1337085587.16840.19.camel@willson.li.ssimo.org> Message-ID: <4FB25A36.4020205@firstyear.id.au> Hi simo > Hi William, > do you have a public repo you are pushing your work to ? > It would be nice to have early access so we can check your > implementation is in line with FreeIPA. It will allow your contribution > to get in more easily if we can comment early on around schema, DIT and > other behavior you need to implement. I haven't had much time to focus on this so far, So I only have a limited amount of work completed. It has mainly been learning the FreeIPA code, adding some skeleton files, and working out the schema. I have tried to push my work to my github from my branch, but I'm getting an error on push. git push github master Counting objects: 38943, done. Compressing objects: 100% (8100/8100), done. error: object 0b36ce6dcbfc8d7e6cda632e06a09c369428a2db:invalid author/committer line - bad date fatal: Error in object error: pack-objects died of signal 13 error: failed to push some refs to 'git at github.com:Firstyear/FreeIPA-dhcp.git' That object appears to be. commit 0b36ce6dcbfc8d7e6cda632e06a09c369428a2db Merge: 451a28c a9e4e5a Author: Karl MacMillan Date: Thu Jan 1 00:00:00 1970 +0000 Merge. Any advice on this issue would be welcome. In the mean time, I'm happy to just send in formatted patches. > An architecture document posted on freeipa.org would also be very > welcome, feel free to ask for an account on the wiki so you can post > stuff in there. Who should I ask to an account on the wiki? -- Sincerely, William Brown pgp.mit.edu http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 945 bytes Desc: OpenPGP digital signature URL: From simo at redhat.com Tue May 15 13:49:12 2012 From: simo at redhat.com (Simo Sorce) Date: Tue, 15 May 2012 09:49:12 -0400 Subject: [Freeipa-devel] Adding indices and permissions to FreeIPA In-Reply-To: <4FB25A36.4020205@firstyear.id.au> References: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> <1337064693.10688.15.camel@balmora.brq.redhat.com> <4FB200BA.70803@firstyear.id.au> <1337085587.16840.19.camel@willson.li.ssimo.org> <4FB25A36.4020205@firstyear.id.au> Message-ID: <1337089752.16840.24.camel@willson.li.ssimo.org> On Tue, 2012-05-15 at 22:59 +0930, William Brown wrote: > Hi simo > > > Hi William, > > do you have a public repo you are pushing your work to ? > > It would be nice to have early access so we can check your > > implementation is in line with FreeIPA. It will allow your contribution > > to get in more easily if we can comment early on around schema, DIT and > > other behavior you need to implement. > > I haven't had much time to focus on this so far, So I only have a > limited amount of work completed. It has mainly been learning the > FreeIPA code, adding some skeleton files, and working out the schema. > > I have tried to push my work to my github from my branch, but I'm > getting an error on push. > > git push github master > Counting objects: 38943, done. > Compressing objects: 100% (8100/8100), done. > error: object 0b36ce6dcbfc8d7e6cda632e06a09c369428a2db:invalid > author/committer line - bad date > fatal: Error in object > error: pack-objects died of signal 13 > error: failed to push some refs to > 'git at github.com:Firstyear/FreeIPA-dhcp.git' > > That object appears to be. > > commit 0b36ce6dcbfc8d7e6cda632e06a09c369428a2db > Merge: 451a28c a9e4e5a > Author: Karl MacMillan > Date: Thu Jan 1 00:00:00 1970 +0000 > > Merge. > > Any advice on this issue would be welcome. In the mean time, I'm happy > to just send in formatted patches. Odd that github doesn't like us. What you could do is to rebase that specific commit in your tree and fix whatever github doesn't like about it (I guess the date). This would cause your tree to not be pullable directly on top of other people trees aligned with the freeipa master tree but that is not a big deal as our process requires sending patches to the list in git format-patch format. So you can easily rebase your work on top of the master tree with git rebase --onto before sending. In the meanwhile being able to reference the tree would be nice. > > An architecture document posted on freeipa.org would also be very > > welcome, feel free to ask for an account on the wiki so you can post > > stuff in there. > > Who should I ask to an account on the wiki? Taken care off on IRC. Simo. -- Simo Sorce * Red Hat, Inc * New York From pviktori at redhat.com Tue May 15 13:53:20 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 15 May 2012 15:53:20 +0200 Subject: [Freeipa-devel] Adding indices and permissions to FreeIPA In-Reply-To: <4FB25A36.4020205@firstyear.id.au> References: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> <1337064693.10688.15.camel@balmora.brq.redhat.com> <4FB200BA.70803@firstyear.id.au> <1337085587.16840.19.camel@willson.li.ssimo.org> <4FB25A36.4020205@firstyear.id.au> Message-ID: <4FB25FD0.3070902@redhat.com> On 05/15/2012 03:29 PM, William Brown wrote: > Hi simo > >> Hi William, >> do you have a public repo you are pushing your work to ? >> It would be nice to have early access so we can check your >> implementation is in line with FreeIPA. It will allow your contribution >> to get in more easily if we can comment early on around schema, DIT and >> other behavior you need to implement. > > I haven't had much time to focus on this so far, So I only have a > limited amount of work completed. It has mainly been learning the > FreeIPA code, adding some skeleton files, and working out the schema. > > I have tried to push my work to my github from my branch, but I'm > getting an error on push. > > git push github master > Counting objects: 38943, done. > Compressing objects: 100% (8100/8100), done. > error: object 0b36ce6dcbfc8d7e6cda632e06a09c369428a2db:invalid > author/committer line - bad date > fatal: Error in object > error: pack-objects died of signal 13 > error: failed to push some refs to > 'git at github.com:Firstyear/FreeIPA-dhcp.git' > > That object appears to be. > > commit 0b36ce6dcbfc8d7e6cda632e06a09c369428a2db > Merge: 451a28c a9e4e5a > Author: Karl MacMillan > Date: Thu Jan 1 00:00:00 1970 +0000 > > Merge. > > Any advice on this issue would be welcome. In the mean time, I'm happy > to just send in formatted patches. Unfortunately, there are some patches in the FreeIPA history that Github chokes on (most likely the problem is with the zero commit date). Since Git history is immutable, there's not much we can do now. As a workaround, Bitbucket seems to work: https://bitbucket.org/encukou/freeipa I'll contact Github about this later today. -- Petr? From sgallagh at redhat.com Tue May 15 13:59:18 2012 From: sgallagh at redhat.com (Stephen Gallagher) Date: Tue, 15 May 2012 09:59:18 -0400 Subject: [Freeipa-devel] Adding indices and permissions to FreeIPA In-Reply-To: <4FB25FD0.3070902@redhat.com> References: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> <1337064693.10688.15.camel@balmora.brq.redhat.com> <4FB200BA.70803@firstyear.id.au> <1337085587.16840.19.camel@willson.li.ssimo.org> <4FB25A36.4020205@firstyear.id.au> <4FB25FD0.3070902@redhat.com> Message-ID: <1337090358.2227.7.camel@sgallagh520.sgallagh.bos.redhat.com> On Tue, 2012-05-15 at 15:53 +0200, Petr Viktorin wrote: > On 05/15/2012 03:29 PM, William Brown wrote: > > Hi simo > > > >> Hi William, > >> do you have a public repo you are pushing your work to ? > >> It would be nice to have early access so we can check your > >> implementation is in line with FreeIPA. It will allow your contribution > >> to get in more easily if we can comment early on around schema, DIT and > >> other behavior you need to implement. > > > > I haven't had much time to focus on this so far, So I only have a > > limited amount of work completed. It has mainly been learning the > > FreeIPA code, adding some skeleton files, and working out the schema. > > > > I have tried to push my work to my github from my branch, but I'm > > getting an error on push. > > > > git push github master > > Counting objects: 38943, done. > > Compressing objects: 100% (8100/8100), done. > > error: object 0b36ce6dcbfc8d7e6cda632e06a09c369428a2db:invalid > > author/committer line - bad date > > fatal: Error in object > > error: pack-objects died of signal 13 > > error: failed to push some refs to > > 'git at github.com:Firstyear/FreeIPA-dhcp.git' > > > > That object appears to be. > > > > commit 0b36ce6dcbfc8d7e6cda632e06a09c369428a2db > > Merge: 451a28c a9e4e5a > > Author: Karl MacMillan > > Date: Thu Jan 1 00:00:00 1970 +0000 > > > > Merge. > > > > Any advice on this issue would be welcome. In the mean time, I'm happy > > to just send in formatted patches. > > Unfortunately, there are some patches in the FreeIPA history that Github > chokes on (most likely the problem is with the zero commit date). Since > Git history is immutable, there's not much we can do now. > > As a workaround, Bitbucket seems to work: > https://bitbucket.org/encukou/freeipa > > > I'll contact Github about this later today. > Another option would be to use a Fedora People repository: http://fedoraproject.org/wiki/Infrastructure/fedorapeople.org#BETA_git_hosting_support The catch is that you need to be a member of at least one group in Fedora besides the FPCA-signed group. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From yzhang at redhat.com Tue May 15 14:59:30 2012 From: yzhang at redhat.com (yi zhang) Date: Tue, 15 May 2012 07:59:30 -0700 Subject: [Freeipa-devel] New beaker server in mountain view In-Reply-To: <4FB1B7A3.8030706@redhat.com> References: <4FB1B7A3.8030706@redhat.com> Message-ID: <4FB26F52.1060505@redhat.com> On 05/14/2012 06:55 PM, Michael Gregg wrote: > > At long last, a beaker server is now up and running here in Mountain View. wow, thanks! I will definitely give it a try. Yi > > The beaker server is able to control nearly all of the machines here at > the mountain view location. > > Login to it here: > http://hammer1.dsdev.sjc.redhat.com/bkr/ > > You should be able to login with your kerberos username and password. > > If you would like to use beaker through the cli, follow these steps: > 1. make sure you have beaker client installed and configured for use > with the wesford lab controller. > 2. Download and install this RPM: > https://engineering.redhat.com/trac/ipa-tests/browser/trunk/ipa-tests/beaker/ipa-server/shared/bkr-hammer-1.0-1.noarch.rpm > 3. you should be able to use "bkr-hammer" just as you would use the bkr > command. > > I may need to add you to some beaker groups to allow you to use all of > the machines here. > > There is only a small number of tests in this beaker server. I tried to > add all of the ipa tasks. Feel free to add any missing tasks that you > need. Or, as me to add them, and I'm move them into this lab controller. > > There is also a reduced set of rhel builds as there isn't enough > bandwidth here to sync the nightlies here. I'll keep this server syned > up with the latest snapshots. Look at the currently available distros > here: http://hammer1.dsdev.sjc.redhat.com/bkr/distros/ > > Give it a try, feedback is welcome. > > Michael- > -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | Yi Zhang | | QA @ Mountain View, California | | Cell: 408-509-6375 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -------------- next part -------------- A non-text attachment was scrubbed... Name: yzhang.vcf Type: text/x-vcard Size: 134 bytes Desc: not available URL: From mkosek at redhat.com Wed May 16 07:44:33 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 16 May 2012 09:44:33 +0200 Subject: [Freeipa-devel] [PATCH] 260 Replace DNS client based on acutil with python-dns In-Reply-To: <4FB245F2.6000500@redhat.com> References: <1336755130.19268.12.camel@priserak> <4FB245F2.6000500@redhat.com> Message-ID: <1337154273.2963.8.camel@balmora.brq.redhat.com> On Tue, 2012-05-15 at 14:02 +0200, Petr Viktorin wrote: > On 05/11/2012 06:52 PM, Martin Kosek wrote: > > python-dns is very feature-rich and it can help us a lot with our DNS > > related code. This patch does the first step, i.e. replaces acutil use > > with python-dns, which is more convenient to use as you will see in the > > patch. More integration will follow in the future. > > > > I send this patch rather early, so that I can get responses to this > > patch early and also so that we are able to catch issues in a safe > > distance from the next release. > > > --- > > IPA client and server tool set used authconfig acutil module to > > for client DNS operations. This is not optimal DNS interface for > > several reasons: > > - does not provide native Python object oriented interface > > but but rather C-like interface based on functions and > > structures which is not easy to use and extend > > - acutil is not meant to be used by third parties besides > > authconfig and thus can break without notice > > > > Replace the acutil with python-dns package which has a feature rich > > interface for dealing with all different aspects of DNS including > > DNSSEC. The main target of this patch is to replace all uses of > > acutil DNS library with a use python-dns. In most cases, even > > though the larger parts of the code are changed, the actual > > functionality is changed only in the following cases: > > - redundant DNS checks were removed from verify_fqdn function > > in installutils to make the whole DNS check simpler and > > less error-prone. Logging was improves for the remaining > > checks > > - improved logging for ipa-client-install DNS discovery > > > > https://fedorahosted.org/freeipa/ticket/2730 > > Also relevant: https://fedorahosted.org/freeipa/ticket/1837 Yup, added to commit message. > > There is a forgotten acutil reference in a comment in > install/tools/ipa-dns-install:226 Fixed. > > I did find some style issues/suggestions. Stop me if the bikeshedding is > too useless :) > > [...] > > diff --git a/ipa-client/ipaclient/ipadiscovery.py b/ipa-client/ipaclient/ipadiscovery.py > > index 86bef28b2d7fdfc8111b493bddec7ac6888f021a..800b4ef9a6396884a1fe2e93bdf94e948f35c57e 100644 > > --- a/ipa-client/ipaclient/ipadiscovery.py > > +++ b/ipa-client/ipaclient/ipadiscovery.py > > @@ -20,10 +20,12 @@ > > import socket > > import os > > from ipapython.ipa_log_manager import * > > -import ipapython.dnsclient > > import tempfile > > import ldap > > from ldap import LDAPError > > +from dns import resolver, rdatatype > > +from dns.exception import DNSException > > + > > from ipapython.ipautil import run, CalledProcessError, valid_ip, get_ipa_basedn, \ > > realm_to_suffix, format_netloc, parse_items > > > > @@ -311,81 +313,89 @@ class IPADiscovery: > > > > > > def ipadnssearchldap(self, tdomain): > > - servers = "" > > - rserver = "" > > + server = "" > > > > - qname = "_ldap._tcp."+tdomain > > - # terminate the name > > - if not qname.endswith("."): > > - qname += "." > > - results = ipapython.dnsclient.query(qname, ipapython.dnsclient.DNS_C_IN, ipapython.dnsclient.DNS_T_SRV) > > + qname = "_ldap._tcp." + tdomain > > > > - for result in results: > > - if result.dns_type == ipapython.dnsclient.DNS_T_SRV: > > - rserver = result.rdata.server.rstrip(".") > > - if result.rdata.port and result.rdata.port != 389: > > - rserver += ":" + str(result.rdata.port) > > - if servers: > > - servers += "," + rserver > > - else: > > - servers = rserver > > - break > > + root_logger.debug("Search DNS for SRV record of %s", qname) > > > > - return servers > > + try: > > + answers = resolver.query(qname, rdatatype.SRV) > > + except DNSException, e: > > + root_logger.debug("DNS record not found: %s", e.__class__.__name__) > > + answers = [] > > + > > + for answer in answers: > > + root_logger.debug("DNS record found: %s", answer) > > + server = str(answer.target).rstrip(".") > > + if answer.port != 389: > > + server += ":" + str(answer.port) > > + break > > + return server > > > > def ipadnssearchntp(self, tdomain): > > - servers = "" > > - rserver = "" > > + server = "" > > > > - qname = "_ntp._udp."+tdomain > > - # terminate the name > > - if not qname.endswith("."): > > - qname += "." > > - results = ipapython.dnsclient.query(qname, ipapython.dnsclient.DNS_C_IN, ipapython.dnsclient.DNS_T_SRV) > > + qname = "_ntp._udp." + tdomain > > > > - for result in results: > > - if result.dns_type == ipapython.dnsclient.DNS_T_SRV: > > - rserver = result.rdata.server.rstrip(".") > > - if servers: > > - servers += "," + rserver > > - else: > > - servers = rserver > > - break > > + root_logger.debug("Search DNS for SRV record of %s", qname) > > > > - return servers > > + try: > > + answers = resolver.query(qname, rdatatype.SRV) > > + except DNSException, e: > > + root_logger.debug("DNS record not found: %s", e.__class__.__name__) > > + answers = [] > > + > > + for answer in answers: > > + root_logger.debug("DNS record found: %s", answer) > > + server = str(answer.target).rstrip(".") > > + break > > + > > + return server > > These function ars mostly the same, except ipadnssearchntp uses "_ntp" > instead of "_ldap", and it doesn't take the port into account (which it > probably should?). > The last part of ipadnssearchkrb is also very similar. > Please merge the common parts. Yup, these parts could be merged - done. > > > def ipadnssearchkrb(self, tdomain): > > [...] > > > diff --git a/ipapython/config.py b/ipapython/config.py > > index d4c724dc9ac754cb221fe60d7c13bd0c716dd296..e06f51a318a2b20c53a8c6933c43d58c34075407 100644 > > --- a/ipapython/config.py > > +++ b/ipapython/config.py > [...] > > @@ -153,7 +155,7 @@ def __parse_config(discover_server = True): > > try: > > s = p.get("global", "xmlrpc_uri") > > server = urlparse.urlsplit(s) > > - config.default_server.extend(server.netloc) > > + config.default_server.append(server.netloc) > > This is unrelated to the DNS library replacement, right? It should go in > a separate patch, in case e.g. the big patch gets reverted. Ok. > > [...] > > + try: > > + servers = resolver.query(name, rdatatype.SRV) > > + domain_ok = True > > + except DNSException: > > + pass > > + > > + if not domain_ok: > > + # try cycling on domain components of FQDN > > This can be just: > > try: > servers = resolver.query(name, rdatatype.SRV) > domain_ok = True > except DNSException: > # try cycling on domain components of FQDN > > The extra if and flag variable just makes it less clear. > > > + try: > > + domain = dns.name.from_text(socket.getfqdn()) > > + except DNSException: > > return False > > - dom_name = dom_name[tok+1:] > > - name = "_ldap._tcp." + dom_name + "." > > - rs = ipapython.dnsclient.query(name, ipapython.dnsclient.DNS_C_IN, ipapython.dnsclient.DNS_T_SRV) > > - rl = len(rs) > > > > - config.default_domain = dom_name > > + while not domain_ok: > > + domain = domain.parent() > > + > > + if str(domain) == '.': > > + return False > > + name = "_ldap._tcp.%s" % domain > > + try: > > + servers = resolver.query(name, rdatatype.SRV) > > + domain_ok = True > > + except DNSException: > > + pass > > + > > + config.default_domain = str(domain).rstrip(".") > > Drop the `domain_ok` variable and just use a while True/break. > The code flow would be more clear that way, IMO. > > [...] > > > diff --git a/ipapython/ipautil.py b/ipapython/ipautil.py > > index 4a9db11e23c1b9ab76c9fce9150bc1546426452f..71be39132ea40172595e026a9815acc58d29f4ff 100644 > > --- a/ipapython/ipautil.py > > +++ b/ipapython/ipautil.py > [...] > > +def is_host_resolvable(fqdn): > > + found = False > > + for rdtype in (rdatatype.A, rdatatype.AAAA): > > + try: > > + resolver.query(fqdn, rdtype) > > + found = True > > + except DNSException: > > + continue > > + > > + return found > > + > > This seems too complicated; why not return directly? > > for rdtype in rdatatype.A, rdatatype.AAAA: > try: > resolver.query(fqdn, rdtype) > except DNSException: > pass > else: > return True > > > return False > > > def get_ipa_basedn(conn): > > """ > > Get base DN of IPA suffix in given LDAP server. > > diff --git a/ipaserver/install/installutils.py b/ipaserver/install/installutils.py > > index 3e7ae41b5fdbc11353e43a63424f19fbc331435a..76b54c94c9d1dad169545dd30c0b9a926f19a3c6 100644 > > --- a/ipaserver/install/installutils.py > > +++ b/ipaserver/install/installutils.py > [...] > > + > > + # list of verified addresses to prevent multiple searches for the same address > > + verified = [] > > Use a set for this. > > > for a in hostaddr: > > - if a[4][0] == '127.0.0.1' or a[4][0] == '::1': > > - raise HostForwardLookupError("The IPA Server hostname must not resolve to localhost (%s). A routable IP address must be used. Check /etc/hosts to see if %s is an alias for %s" % (a[4][0], host_name, a[4][0])) > > + address = a[4][0] > > + if address in verified: > > + continue > [...] > > Fixed. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-260-2-replace-dns-client-based-on-acutil-with-python-dns.patch Type: text/x-patch Size: 44483 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-261-fix-default_server-configuration-in-ipapython.config.patch Type: text/x-patch Size: 1007 bytes Desc: not available URL: From mkosek at redhat.com Wed May 16 07:58:09 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 16 May 2012 09:58:09 +0200 Subject: [Freeipa-devel] [PATCH] 0050 Fail on unknown Command options In-Reply-To: <4FB23F9B.10000@redhat.com> References: <4F8E97F6.2070703@redhat.com> <1336392614.29911.13.camel@balmora.brq.redhat.com> <4FABA16A.10802@redhat.com> <1336982456.4344.28.camel@balmora.brq.redhat.com> <1336984817.4344.43.camel@balmora.brq.redhat.com> <4FB0FEDB.4020806@redhat.com> <1337068555.10688.20.camel@balmora.brq.redhat.com> <4FB23F9B.10000@redhat.com> Message-ID: <1337155089.2963.10.camel@balmora.brq.redhat.com> On Tue, 2012-05-15 at 13:35 +0200, Petr Viktorin wrote: > On 05/15/2012 09:55 AM, Martin Kosek wrote: > > On Mon, 2012-05-14 at 14:47 +0200, Petr Viktorin wrote: > >> The final part of rejecting unknown Command arguments: enable the > >> validation, add tests. > >> Also fix up things that were changed since the previous patches. > >> > >> https://fedorahosted.org/freeipa/ticket/2509 > >> > > > > The patch looks OK so far. I just found an error in permission/aci > > plugin - --subtree does not work when it matches a result: > > > > # ipa permission-find --subtree=foo > > --------------------- > > 0 permissions matched > > --------------------- > > ---------------------------- > > Number of entries returned 0 > > ---------------------------- > > > > ipa permission-find > > --subtree='ldap:///ipauniqueid=*,cn=hbac,dc=idm,dc=lab,dc=bos,dc=redhat,dc=Com' > > ipa: ERROR: Unknown option: subtree > > Attaching fixed patch. > > > We should not pass **options to aci_show, it is too risky. There may be > > other places where we don't use an option-safe approach that we want to > > have fixed. > > We shouldn't really pass **options to any command; listing everything > explicitly would be much safer. Unfortunately, in a lot of cases where > commands call other commands, it's currently done this way. > There is something wrong with the patch you sent (seen on F16 and F17): $ git apply ~/freeipa-pviktori-0050-02-Fail-on-unknown-Command-options.patch fatal: corrupt patch at line 129 Martin From pviktori at redhat.com Wed May 16 08:37:30 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 16 May 2012 10:37:30 +0200 Subject: [Freeipa-devel] [PATCH] 0050 Fail on unknown Command options In-Reply-To: <1337155089.2963.10.camel@balmora.brq.redhat.com> References: <4F8E97F6.2070703@redhat.com> <1336392614.29911.13.camel@balmora.brq.redhat.com> <4FABA16A.10802@redhat.com> <1336982456.4344.28.camel@balmora.brq.redhat.com> <1336984817.4344.43.camel@balmora.brq.redhat.com> <4FB0FEDB.4020806@redhat.com> <1337068555.10688.20.camel@balmora.brq.redhat.com> <4FB23F9B.10000@redhat.com> <1337155089.2963.10.camel@balmora.brq.redhat.com> Message-ID: <4FB3674A.1010402@redhat.com> On 05/16/2012 09:58 AM, Martin Kosek wrote: > On Tue, 2012-05-15 at 13:35 +0200, Petr Viktorin wrote: >> On 05/15/2012 09:55 AM, Martin Kosek wrote: >>> On Mon, 2012-05-14 at 14:47 +0200, Petr Viktorin wrote: >>>> The final part of rejecting unknown Command arguments: enable the >>>> validation, add tests. >>>> Also fix up things that were changed since the previous patches. >>>> >>>> https://fedorahosted.org/freeipa/ticket/2509 >>>> >>> >>> The patch looks OK so far. I just found an error in permission/aci >>> plugin - --subtree does not work when it matches a result: >>> >>> # ipa permission-find --subtree=foo >>> --------------------- >>> 0 permissions matched >>> --------------------- >>> ---------------------------- >>> Number of entries returned 0 >>> ---------------------------- >>> >>> ipa permission-find >>> --subtree='ldap:///ipauniqueid=*,cn=hbac,dc=idm,dc=lab,dc=bos,dc=redhat,dc=Com' >>> ipa: ERROR: Unknown option: subtree >> >> Attaching fixed patch. >> >>> We should not pass **options to aci_show, it is too risky. There may be >>> other places where we don't use an option-safe approach that we want to >>> have fixed. >> >> We shouldn't really pass **options to any command; listing everything >> explicitly would be much safer. Unfortunately, in a lot of cases where >> commands call other commands, it's currently done this way. >> > > There is something wrong with the patch you sent (seen on F16 and F17): > > $ git apply ~/freeipa-pviktori-0050-02-Fail-on-unknown-Command-options.patch > fatal: corrupt patch at line 129 > > Martin > Attaching a rebased patch. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0050-03-Fail-on-unknown-Command-options.patch Type: text/x-patch Size: 16112 bytes Desc: not available URL: From william at firstyear.id.au Wed May 16 10:13:23 2012 From: william at firstyear.id.au (William Brown) Date: Wed, 16 May 2012 19:43:23 +0930 Subject: [Freeipa-devel] Adding indices and permissions to FreeIPA In-Reply-To: <4FB25FD0.3070902@redhat.com> References: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> <1337064693.10688.15.camel@balmora.brq.redhat.com> <4FB200BA.70803@firstyear.id.au> <1337085587.16840.19.camel@willson.li.ssimo.org> <4FB25A36.4020205@firstyear.id.au> <4FB25FD0.3070902@redhat.com> Message-ID: <4FB37DC3.5060901@firstyear.id.au> Hi, >>> do you have a public repo you are pushing your work to ? >>> It would be nice to have early access so we can check your >>> implementation is in line with FreeIPA. It will allow your contribution >>> to get in more easily if we can comment early on around schema, DIT and >>> other behavior you need to implement. >> >> I haven't had much time to focus on this so far, So I only have a >> limited amount of work completed. It has mainly been learning the >> FreeIPA code, adding some skeleton files, and working out the schema. >> //snip > > As a workaround, Bitbucket seems to work: > https://bitbucket.org/encukou/freeipa https://bitbucket.org/Firstyear/freeipa-dhcp I have pushed what I currently have into this repository. Happy to recieve comments. I probably won't get a lot of time to work on this in the next few days, but I plan to put some time into it on the weekend. -- Sincerely, William Brown pgp.mit.edu http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 945 bytes Desc: OpenPGP digital signature URL: From pviktori at redhat.com Wed May 16 10:20:16 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 16 May 2012 12:20:16 +0200 Subject: [Freeipa-devel] Adding indices and permissions to FreeIPA In-Reply-To: <4FB37DC3.5060901@firstyear.id.au> References: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> <1337064693.10688.15.camel@balmora.brq.redhat.com> <4FB200BA.70803@firstyear.id.au> <1337085587.16840.19.camel@willson.li.ssimo.org> <4FB25A36.4020205@firstyear.id.au> <4FB25FD0.3070902@redhat.com> <4FB37DC3.5060901@firstyear.id.au> Message-ID: <4FB37F60.6070806@redhat.com> On 05/16/2012 12:13 PM, William Brown wrote: > Hi, > >>>> do you have a public repo you are pushing your work to ? >>>> It would be nice to have early access so we can check your >>>> implementation is in line with FreeIPA. It will allow your contribution >>>> to get in more easily if we can comment early on around schema, DIT and >>>> other behavior you need to implement. >>> >>> I haven't had much time to focus on this so far, So I only have a >>> limited amount of work completed. It has mainly been learning the >>> FreeIPA code, adding some skeleton files, and working out the schema. >>> > > //snip > >> >> As a workaround, Bitbucket seems to work: >> https://bitbucket.org/encukou/freeipa > > https://bitbucket.org/Firstyear/freeipa-dhcp > > I have pushed what I currently have into this repository. Happy to > recieve comments. I probably won't get a lot of time to work on this in > the next few days, but I plan to put some time into it on the weekend. > Your repository is private, please go to its Admin section and make it public: https://bitbucket.org/Firstyear/freeipa-dhcp/admin -- Petr? From william at firstyear.id.au Wed May 16 10:51:02 2012 From: william at firstyear.id.au (William Brown) Date: Wed, 16 May 2012 20:21:02 +0930 Subject: [Freeipa-devel] Adding indices and permissions to FreeIPA In-Reply-To: <4FB37F60.6070806@redhat.com> References: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> <1337064693.10688.15.camel@balmora.brq.redhat.com> <4FB200BA.70803@firstyear.id.au> <1337085587.16840.19.camel@willson.li.ssimo.org> <4FB25A36.4020205@firstyear.id.au> <4FB25FD0.3070902@redhat.com> <4FB37DC3.5060901@firstyear.id.au> <4FB37F60.6070806@redhat.com> Message-ID: <4FB38696.7070800@firstyear.id.au> On 16/05/12 19:50, Petr Viktorin wrote: > On 05/16/2012 12:13 PM, William Brown wrote: >> Hi, >> >>>>> do you have a public repo you are pushing your work to ? >>>>> It would be nice to have early access so we can check your >>>>> implementation is in line with FreeIPA. It will allow your >>>>> contribution >>>>> to get in more easily if we can comment early on around schema, DIT >>>>> and >>>>> other behavior you need to implement. >>>> >>>> I haven't had much time to focus on this so far, So I only have a >>>> limited amount of work completed. It has mainly been learning the >>>> FreeIPA code, adding some skeleton files, and working out the schema. >>>> >> >> //snip >> >>> >>> As a workaround, Bitbucket seems to work: >>> https://bitbucket.org/encukou/freeipa >> >> https://bitbucket.org/Firstyear/freeipa-dhcp >> >> I have pushed what I currently have into this repository. Happy to >> recieve comments. I probably won't get a lot of time to work on this in >> the next few days, but I plan to put some time into it on the weekend. >> > > Your repository is private, please go to its Admin section and make it > public: https://bitbucket.org/Firstyear/freeipa-dhcp/admin > I could have sworn I marked it public earlier. Fixed now. -- Sincerely, William Brown pgp.mit.edu http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 945 bytes Desc: OpenPGP digital signature URL: From mkosek at redhat.com Wed May 16 12:11:38 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 16 May 2012 14:11:38 +0200 Subject: [Freeipa-devel] [PATCH] 0050 Fail on unknown Command options In-Reply-To: <4FB3674A.1010402@redhat.com> References: <4F8E97F6.2070703@redhat.com> <1336392614.29911.13.camel@balmora.brq.redhat.com> <4FABA16A.10802@redhat.com> <1336982456.4344.28.camel@balmora.brq.redhat.com> <1336984817.4344.43.camel@balmora.brq.redhat.com> <4FB0FEDB.4020806@redhat.com> <1337068555.10688.20.camel@balmora.brq.redhat.com> <4FB23F9B.10000@redhat.com> <1337155089.2963.10.camel@balmora.brq.redhat.com> <4FB3674A.1010402@redhat.com> Message-ID: <1337170298.2963.16.camel@balmora.brq.redhat.com> On Wed, 2012-05-16 at 10:37 +0200, Petr Viktorin wrote: > On 05/16/2012 09:58 AM, Martin Kosek wrote: > > On Tue, 2012-05-15 at 13:35 +0200, Petr Viktorin wrote: > >> On 05/15/2012 09:55 AM, Martin Kosek wrote: > >>> On Mon, 2012-05-14 at 14:47 +0200, Petr Viktorin wrote: > >>>> The final part of rejecting unknown Command arguments: enable the > >>>> validation, add tests. > >>>> Also fix up things that were changed since the previous patches. > >>>> > >>>> https://fedorahosted.org/freeipa/ticket/2509 > >>>> > >>> > >>> The patch looks OK so far. I just found an error in permission/aci > >>> plugin - --subtree does not work when it matches a result: > >>> > >>> # ipa permission-find --subtree=foo > >>> --------------------- > >>> 0 permissions matched > >>> --------------------- > >>> ---------------------------- > >>> Number of entries returned 0 > >>> ---------------------------- > >>> > >>> ipa permission-find > >>> --subtree='ldap:///ipauniqueid=*,cn=hbac,dc=idm,dc=lab,dc=bos,dc=redhat,dc=Com' > >>> ipa: ERROR: Unknown option: subtree > >> > >> Attaching fixed patch. > >> > >>> We should not pass **options to aci_show, it is too risky. There may be > >>> other places where we don't use an option-safe approach that we want to > >>> have fixed. > >> > >> We shouldn't really pass **options to any command; listing everything > >> explicitly would be much safer. Unfortunately, in a lot of cases where > >> commands call other commands, it's currently done this way. > >> > > > > There is something wrong with the patch you sent (seen on F16 and F17): > > > > $ git apply ~/freeipa-pviktori-0050-02-Fail-on-unknown-Command-options.patch > > fatal: corrupt patch at line 129 > > > > Martin > > > > Attaching a rebased patch. > Yup, this one is fine. Now, I did not find issues in the patch itself, tests are clean. However, thanks to this new check I found issues in Web UI (automember, selfservice, delegation screen) which use illegal options and which should be fixed before we push your patch: https://fedorahosted.org/freeipa/ticket/2760 Martin From simo at redhat.com Wed May 16 13:56:08 2012 From: simo at redhat.com (Simo Sorce) Date: Wed, 16 May 2012 09:56:08 -0400 Subject: [Freeipa-devel] Adding indices and permissions to FreeIPA In-Reply-To: <4FB38696.7070800@firstyear.id.au> References: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> <1337064693.10688.15.camel@balmora.brq.redhat.com> <4FB200BA.70803@firstyear.id.au> <1337085587.16840.19.camel@willson.li.ssimo.org> <4FB25A36.4020205@firstyear.id.au> <4FB25FD0.3070902@redhat.com> <4FB37DC3.5060901@firstyear.id.au> <4FB37F60.6070806@redhat.com> <4FB38696.7070800@firstyear.id.au> Message-ID: <1337176568.16840.81.camel@willson.li.ssimo.org> On Wed, 2012-05-16 at 20:21 +0930, William Brown wrote: > On 16/05/12 19:50, Petr Viktorin wrote: > > On 05/16/2012 12:13 PM, William Brown wrote: > >> Hi, > >> > >>>>> do you have a public repo you are pushing your work to ? > >>>>> It would be nice to have early access so we can check your > >>>>> implementation is in line with FreeIPA. It will allow your > >>>>> contribution > >>>>> to get in more easily if we can comment early on around schema, DIT > >>>>> and > >>>>> other behavior you need to implement. > >>>> > >>>> I haven't had much time to focus on this so far, So I only have a > >>>> limited amount of work completed. It has mainly been learning the > >>>> FreeIPA code, adding some skeleton files, and working out the schema. > >>>> > >> > >> //snip > >> > >>> > >>> As a workaround, Bitbucket seems to work: > >>> https://bitbucket.org/encukou/freeipa > >> > >> https://bitbucket.org/Firstyear/freeipa-dhcp > >> > >> I have pushed what I currently have into this repository. Happy to > >> recieve comments. I probably won't get a lot of time to work on this in > >> the next few days, but I plan to put some time into it on the weekend. > >> > > > > Your repository is private, please go to its Admin section and make it > > public: https://bitbucket.org/Firstyear/freeipa-dhcp/admin > > > > I could have sworn I marked it public earlier. Fixed now. I will take a look soon, however please do never do merges, please always rebase on top of master and force push. (see git rebase -i and git push -f) Also if you can split commits in small patches for each functionality you'll make our life much easier. Simo. -- Simo Sorce * Red Hat, Inc * New York From william at firstyear.id.au Wed May 16 14:00:02 2012 From: william at firstyear.id.au (William Brown) Date: Wed, 16 May 2012 23:30:02 +0930 Subject: [Freeipa-devel] Adding indices and permissions to FreeIPA In-Reply-To: <1337176568.16840.81.camel@willson.li.ssimo.org> References: <62777208-4524-43B4-B2CE-94EB47B63706@firstyear.id.au> <1337064693.10688.15.camel@balmora.brq.redhat.com> <4FB200BA.70803@firstyear.id.au> <1337085587.16840.19.camel@willson.li.ssimo.org> <4FB25A36.4020205@firstyear.id.au> <4FB25FD0.3070902@redhat.com> <4FB37DC3.5060901@firstyear.id.au> <4FB37F60.6070806@redhat.com> <4FB38696.7070800@firstyear.id.au> <1337176568.16840.81.camel@willson.li.ssimo.org> Message-ID: <4FB3B2E2.909@firstyear.id.au> On 16/05/12 23:26, Simo Sorce wrote: > On Wed, 2012-05-16 at 20:21 +0930, William Brown wrote: >> On 16/05/12 19:50, Petr Viktorin wrote: >>> On 05/16/2012 12:13 PM, William Brown wrote: >>>> Hi, >>>> >>>>>>> do you have a public repo you are pushing your work to ? >>>>>>> It would be nice to have early access so we can check your >>>>>>> implementation is in line with FreeIPA. It will allow your >>>>>>> contribution >>>>>>> to get in more easily if we can comment early on around schema, DIT >>>>>>> and >>>>>>> other behavior you need to implement. >>>>>> >>>>>> I haven't had much time to focus on this so far, So I only have a >>>>>> limited amount of work completed. It has mainly been learning the >>>>>> FreeIPA code, adding some skeleton files, and working out the schema. >>>>>> >>>> >>>> //snip >>>> >>>>> >>>>> As a workaround, Bitbucket seems to work: >>>>> https://bitbucket.org/encukou/freeipa >>>> >>>> https://bitbucket.org/Firstyear/freeipa-dhcp >>>> >>>> I have pushed what I currently have into this repository. Happy to >>>> recieve comments. I probably won't get a lot of time to work on this in >>>> the next few days, but I plan to put some time into it on the weekend. >>>> >>> >>> Your repository is private, please go to its Admin section and make it >>> public: https://bitbucket.org/Firstyear/freeipa-dhcp/admin >>> >> >> I could have sworn I marked it public earlier. Fixed now. > > I will take a look soon, however please do never do merges, please > always rebase on top of master and force push. > (see git rebase -i and git push -f) > > Also if you can split commits in small patches for each functionality > you'll make our life much easier. > Yep, Ill make sure to do this. I normally rebase and squash commits into groups, but I didn't with the last one. -- Sincerely, William Brown pgp.mit.edu http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 945 bytes Desc: OpenPGP digital signature URL: From ohamada at redhat.com Wed May 16 17:15:00 2012 From: ohamada at redhat.com (Ondrej Hamada) Date: Wed, 16 May 2012 19:15:00 +0200 Subject: [Freeipa-devel] [PATCH] 24 ipa permission-mod prompts for all parameters Message-ID: <4FB3E094.8000009@redhat.com> ipa permission-mod was prompting for all parameters because they had specified flag 'ask_update'. The flag was removed. Additionally the exec_callback for permission-mod was updated to unify the behaviour with other ipa commands (raise exception when no modification was specified). https://fedorahosted.org/freeipa/ticket/2280 -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ohamada-24-permission-mod-prompts-for-all-parameters.patch Type: text/x-patch Size: 6130 bytes Desc: not available URL: From rcritten at redhat.com Wed May 16 18:04:33 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 16 May 2012 14:04:33 -0400 Subject: [Freeipa-devel] [PATCH] 1012 validate domain in installer In-Reply-To: <1337063881.10688.4.camel@balmora.brq.redhat.com> References: <4FAD7084.1090905@redhat.com> <1336984596.4344.41.camel@balmora.brq.redhat.com> <4FB17942.2050605@redhat.com> <1337063881.10688.4.camel@balmora.brq.redhat.com> Message-ID: <4FB3EC31.8090806@redhat.com> Martin Kosek wrote: > On Mon, 2012-05-14 at 17:29 -0400, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On Fri, 2012-05-11 at 16:03 -0400, Rob Crittenden wrote: >>>> Use our domain validator to validate the domain name we get in the >>>> installer. >>>> >>>> rob >>> >>> I found few issues with the patch: >>> >>> 1) The "unexpected error" is not very user friendly error message: >>> >>> # ipa-server-install >>> ... >>> Server host name [vm-109.idm.lab.bos.redhat.com]: >>> >>> The domain name has been calculated based on the host name. >>> >>> Please confirm the domain name [idm.lab.bos.redhat.com]: bar-.com >>> >>> Unexpected error - see ipaserver-install.log for details: >>> only letters, numbers, and - are allowed. DNS label may not start or >>> end with - >>> >>> >>> I think we should make it consistent with hostname validation error >>> message format: >>> >>> # ipa-server-install >>> ... >>> Server host name [vm-109.idm.lab.bos.redhat.com]: foo- >>> >>> Invalid hostname 'foo-', must be fully-qualified. >>> >>> >>> 2) The error is also confusing when domain is passed as an option, user >>> won't know what actually failed: >>> >>> # ipa-server-install --domain .. >>> ... >>> Server host name [vm-109.idm.lab.bos.redhat.com]: >>> >>> Unexpected error - see ipaserver-install.log for details: >>> empty DNS label >>> >>> As for value passed via option, I think we should quit immediately and >>> not in second or third interactive step - like we do for --zonemgr >>> validation: >>> >>> # ipa-server-install --zonemgr=foo >>> Usage: ipa-server-install [options] >>> >>> ipa-server-install: error: invalid zonemgr: missing address domain >>> >>> This applies both for --hostname and --domain options. >>> >>> Martin >>> >> >> Done. >> >> There is a separate ticket to validate hostname in the parser. It is a >> bit more complicated since it depends on other options so I punted on that. >> >> rob > > Nack. Domain name passed via option is not used. I assume this is the > reason: > > +def domain_callback(option, opt_str, value, parser): > ... > + > + parser.values.zonemgr = value > + > > Btw. callback is not necessary for domain validation, it may be done > right in parse_options() in ipa-server-install with other checks. > > Martin > Ouch, that was embarrassing. Took your advise and dumped the validator, it isn't like this is going to be shared so yeah, it's overkill. I went back and forth whether to validate in read_domain_name() or not and decided against it. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1012-3-validator.patch Type: text/x-diff Size: 2715 bytes Desc: not available URL: From rcritten at redhat.com Wed May 16 18:31:03 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 16 May 2012 14:31:03 -0400 Subject: [Freeipa-devel] [PATCH] 1014 configurable service timeout Message-ID: <4FB3F267.2070104@redhat.com> A hardcoded timeout was used in ipactl for service restarts, set rather low. A separate timeout was hardcoded into the installer. I centralized them into a single timeout, configurable in the standard way in /etc/ipa/*.conf. On install it will always default to 120 seconds and remain there unless changed in default.conf (not replicated either). I tested this on systemd systems and sysV systems and it works ok for me. You'll also want to double-check that this works when other 389-ds instances are installed. Getting the naming of instances right was a bit tricky. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1014-timeout.patch Type: text/x-diff Size: 28009 bytes Desc: not available URL: From rcritten at redhat.com Wed May 16 20:40:31 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 16 May 2012 16:40:31 -0400 Subject: [Freeipa-devel] [PATCH] 1015 improve network address not local error Message-ID: <4FB410BF.8000307@redhat.com> The error raised when the IP address for a host is not a configured network interface wasn't very clear. Include the IP address that we resolved when displaying the error. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1015-notlocal.patch Type: text/x-diff Size: 1349 bytes Desc: not available URL: From rcritten at redhat.com Wed May 16 22:25:22 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 16 May 2012 18:25:22 -0400 Subject: [Freeipa-devel] [PATCH] 1016 check for replication agreements Message-ID: <4FB42952.5010504@redhat.com> In ipa-replica-install we infer that a replication agreement exists if the host exists. Add an explicit check instead. What I did to test this was to break out of the installation right after replication succeeded. Then I'd run ipa-server-install --uninstall and try the replica install again. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1016-replicainstall.patch Type: text/x-diff Size: 6266 bytes Desc: not available URL: From mkosek at redhat.com Thu May 17 05:59:00 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 17 May 2012 07:59:00 +0200 Subject: [Freeipa-devel] [PATCH] 1015 improve network address not local error In-Reply-To: <4FB410BF.8000307@redhat.com> References: <4FB410BF.8000307@redhat.com> Message-ID: <1337234340.23027.0.camel@balmora.brq.redhat.com> On Wed, 2012-05-16 at 16:40 -0400, Rob Crittenden wrote: > The error raised when the IP address for a host is not a configured > network interface wasn't very clear. Include the IP address that we > resolved when displaying the error. > > rob ACK. Pushed to master. Martin From mkosek at redhat.com Thu May 17 06:03:26 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 17 May 2012 08:03:26 +0200 Subject: [Freeipa-devel] [PATCH] 1012 validate domain in installer In-Reply-To: <4FB3EC31.8090806@redhat.com> References: <4FAD7084.1090905@redhat.com> <1336984596.4344.41.camel@balmora.brq.redhat.com> <4FB17942.2050605@redhat.com> <1337063881.10688.4.camel@balmora.brq.redhat.com> <4FB3EC31.8090806@redhat.com> Message-ID: <1337234606.23027.1.camel@balmora.brq.redhat.com> On Wed, 2012-05-16 at 14:04 -0400, Rob Crittenden wrote: > Martin Kosek wrote: > > On Mon, 2012-05-14 at 17:29 -0400, Rob Crittenden wrote: > >> Martin Kosek wrote: > >>> On Fri, 2012-05-11 at 16:03 -0400, Rob Crittenden wrote: > >>>> Use our domain validator to validate the domain name we get in the > >>>> installer. > >>>> > >>>> rob > >>> > >>> I found few issues with the patch: > >>> > >>> 1) The "unexpected error" is not very user friendly error message: > >>> > >>> # ipa-server-install > >>> ... > >>> Server host name [vm-109.idm.lab.bos.redhat.com]: > >>> > >>> The domain name has been calculated based on the host name. > >>> > >>> Please confirm the domain name [idm.lab.bos.redhat.com]: bar-.com > >>> > >>> Unexpected error - see ipaserver-install.log for details: > >>> only letters, numbers, and - are allowed. DNS label may not start or > >>> end with - > >>> > >>> > >>> I think we should make it consistent with hostname validation error > >>> message format: > >>> > >>> # ipa-server-install > >>> ... > >>> Server host name [vm-109.idm.lab.bos.redhat.com]: foo- > >>> > >>> Invalid hostname 'foo-', must be fully-qualified. > >>> > >>> > >>> 2) The error is also confusing when domain is passed as an option, user > >>> won't know what actually failed: > >>> > >>> # ipa-server-install --domain .. > >>> ... > >>> Server host name [vm-109.idm.lab.bos.redhat.com]: > >>> > >>> Unexpected error - see ipaserver-install.log for details: > >>> empty DNS label > >>> > >>> As for value passed via option, I think we should quit immediately and > >>> not in second or third interactive step - like we do for --zonemgr > >>> validation: > >>> > >>> # ipa-server-install --zonemgr=foo > >>> Usage: ipa-server-install [options] > >>> > >>> ipa-server-install: error: invalid zonemgr: missing address domain > >>> > >>> This applies both for --hostname and --domain options. > >>> > >>> Martin > >>> > >> > >> Done. > >> > >> There is a separate ticket to validate hostname in the parser. It is a > >> bit more complicated since it depends on other options so I punted on that. > >> > >> rob > > > > Nack. Domain name passed via option is not used. I assume this is the > > reason: > > > > +def domain_callback(option, opt_str, value, parser): > > ... > > + > > + parser.values.zonemgr = value > > + > > > > Btw. callback is not necessary for domain validation, it may be done > > right in parse_options() in ipa-server-install with other checks. > > > > Martin > > > > Ouch, that was embarrassing. Took your advise and dumped the validator, > it isn't like this is going to be shared so yeah, it's overkill. > > I went back and forth whether to validate in read_domain_name() or not > and decided against it. > > rob Yup, this one is ok :-) ACK, pushed to master. Martin From mkosek at redhat.com Thu May 17 07:51:34 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 17 May 2012 09:51:34 +0200 Subject: [Freeipa-devel] [PATCH] 1016 check for replication agreements In-Reply-To: <4FB42952.5010504@redhat.com> References: <4FB42952.5010504@redhat.com> Message-ID: <1337241094.23027.8.camel@balmora.brq.redhat.com> On Wed, 2012-05-16 at 18:25 -0400, Rob Crittenden wrote: > In ipa-replica-install we infer that a replication agreement exists if > the host exists. Add an explicit check instead. > > What I did to test this was to break out of the installation right after > replication succeeded. Then I'd run ipa-server-install --uninstall and > try the replica install again. > > rob I was also able to reproduce the issue by installing master and replica, then deleting the replica host entry and running the ipa-replica-install again. The patch works fine, I would just improve the following notice: $ ipa-replica-install INFO_FILE ... A replication agreement for this host already exists. It needs to be removed. % ipa-replica-manage del vm-098.idm.lab.bos.redhat.com ... and add "--force" flag to the recommended command. Otherwise, user will receive an error as obviously the replica is not alive anymore. Maybe we could also suggest that this should be run on a master machine so that users don't try to run it on a replica. Martin From mkosek at redhat.com Thu May 17 08:14:23 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 17 May 2012 10:14:23 +0200 Subject: [Freeipa-devel] [PATCH] 24 ipa permission-mod prompts for all parameters In-Reply-To: <4FB3E094.8000009@redhat.com> References: <4FB3E094.8000009@redhat.com> Message-ID: <1337242463.23027.9.camel@balmora.brq.redhat.com> On Wed, 2012-05-16 at 19:15 +0200, Ondrej Hamada wrote: > ipa permission-mod was prompting for all parameters because they had > specified flag 'ask_update'. The flag was removed. Additionally the > exec_callback for permission-mod was updated to unify the behaviour with > other ipa commands (raise exception when no modification was specified). > > https://fedorahosted.org/freeipa/ticket/2280 ACK. Pushed to master. Martin From simo at redhat.com Thu May 17 14:51:34 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 17 May 2012 10:51:34 -0400 Subject: [Freeipa-devel] [PATCH] 491 Fix password migration issues Message-ID: <1337266294.16840.122.camel@willson.li.ssimo.org> Migration code was failing to set krbExtraData, this irks kadmind. Fixes #2764 -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-491-1-Fix-migration-code-password-setting.patch Type: text/x-patch Size: 1728 bytes Desc: not available URL: From rcritten at redhat.com Thu May 17 14:52:27 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 17 May 2012 10:52:27 -0400 Subject: [Freeipa-devel] [PATCH] 1016 check for replication agreements In-Reply-To: <1337241094.23027.8.camel@balmora.brq.redhat.com> References: <4FB42952.5010504@redhat.com> <1337241094.23027.8.camel@balmora.brq.redhat.com> Message-ID: <4FB510AB.70404@redhat.com> Martin Kosek wrote: > On Wed, 2012-05-16 at 18:25 -0400, Rob Crittenden wrote: >> In ipa-replica-install we infer that a replication agreement exists if >> the host exists. Add an explicit check instead. >> >> What I did to test this was to break out of the installation right after >> replication succeeded. Then I'd run ipa-server-install --uninstall and >> try the replica install again. >> >> rob > > I was also able to reproduce the issue by installing master and replica, > then deleting the replica host entry and running the ipa-replica-install > again. > > The patch works fine, I would just improve the following notice: > > $ ipa-replica-install INFO_FILE > ... > A replication agreement for this host already exists. It needs to be > removed. > % ipa-replica-manage del vm-098.idm.lab.bos.redhat.com > > ... and add "--force" flag to the recommended command. Otherwise, user > will receive an error as obviously the replica is not alive anymore. > Maybe we could also suggest that this should be run on a master machine > so that users don't try to run it on a replica. > > Martin > Good idea, updated. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1016-2-replicainstall.patch Type: text/x-diff Size: 6327 bytes Desc: not available URL: From mkosek at redhat.com Thu May 17 15:20:03 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 17 May 2012 17:20:03 +0200 Subject: [Freeipa-devel] [PATCH] 491 Fix password migration issues In-Reply-To: <1337266294.16840.122.camel@willson.li.ssimo.org> References: <1337266294.16840.122.camel@willson.li.ssimo.org> Message-ID: <1337268003.6836.0.camel@balmora.brq.redhat.com> On Thu, 2012-05-17 at 10:51 -0400, Simo Sorce wrote: > Migration code was failing to set krbExtraData, this irks kadmind. > > Fixes #2764 ACK, this fixed the issue. Pushed to ipa-2-2, master. Martin From mkosek at redhat.com Thu May 17 15:24:26 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 17 May 2012 17:24:26 +0200 Subject: [Freeipa-devel] [PATCH] 1016 check for replication agreements In-Reply-To: <4FB510AB.70404@redhat.com> References: <4FB42952.5010504@redhat.com> <1337241094.23027.8.camel@balmora.brq.redhat.com> <4FB510AB.70404@redhat.com> Message-ID: <1337268266.6836.1.camel@balmora.brq.redhat.com> On Thu, 2012-05-17 at 10:52 -0400, Rob Crittenden wrote: > Martin Kosek wrote: > > On Wed, 2012-05-16 at 18:25 -0400, Rob Crittenden wrote: > >> In ipa-replica-install we infer that a replication agreement exists if > >> the host exists. Add an explicit check instead. > >> > >> What I did to test this was to break out of the installation right after > >> replication succeeded. Then I'd run ipa-server-install --uninstall and > >> try the replica install again. > >> > >> rob > > > > I was also able to reproduce the issue by installing master and replica, > > then deleting the replica host entry and running the ipa-replica-install > > again. > > > > The patch works fine, I would just improve the following notice: > > > > $ ipa-replica-install INFO_FILE > > ... > > A replication agreement for this host already exists. It needs to be > > removed. > > % ipa-replica-manage del vm-098.idm.lab.bos.redhat.com > > > > ... and add "--force" flag to the recommended command. Otherwise, user > > will receive an error as obviously the replica is not alive anymore. > > Maybe we could also suggest that this should be run on a master machine > > so that users don't try to run it on a replica. > > > > Martin > > > > Good idea, updated. > > rob Thanks ok, thanks. ACK, pushed to master. Martin From rcritten at redhat.com Thu May 17 17:25:58 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 17 May 2012 13:25:58 -0400 Subject: [Freeipa-devel] [PATCH] 1017 fix handling of unlocked users Message-ID: <4FB534A6.3070900@redhat.com> There was a problem in admin-unlocked accounts in that while still in the lock duration they would never be re-locked. More info in the patch. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1017-failcount.patch Type: text/x-diff Size: 2233 bytes Desc: not available URL: From rcritten at redhat.com Thu May 17 20:11:21 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 17 May 2012 16:11:21 -0400 Subject: [Freeipa-devel] [PATCH] 1018 enforce sizelimit when searching for permissions Message-ID: <4FB55B69.5080506@redhat.com> We do two searches when looking for permissions. One within the permission object itself and a second in the ACIs. We weren't enforcing a sizelimit on either search. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1018-sizelimit.patch Type: text/x-diff Size: 4703 bytes Desc: not available URL: From mareynol at redhat.com Thu May 17 20:34:32 2012 From: mareynol at redhat.com (Mark Reynolds) Date: Thu, 17 May 2012 16:34:32 -0400 Subject: [Freeipa-devel] please review ticket #321 - krbExtraData is being null modified and replicated on each ssh login Message-ID: <4FB560D8.9090509@redhat.com> https://fedorahosted.org/389/ticket/321 https://fedorahosted.org/389/attachment/ticket/321/0001-Ticket-321-krbExtraData-is-being-null-modified-and-r.patch Thanks, Mark From mkosek at redhat.com Fri May 18 07:06:25 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 18 May 2012 09:06:25 +0200 Subject: [Freeipa-devel] [PATCH] 1017 fix handling of unlocked users In-Reply-To: <4FB534A6.3070900@redhat.com> References: <4FB534A6.3070900@redhat.com> Message-ID: <1337324785.25787.3.camel@balmora.brq.redhat.com> On Thu, 2012-05-17 at 13:25 -0400, Rob Crittenden wrote: > There was a problem in admin-unlocked accounts in that while still in > the lock duration they would never be re-locked. More info in the patch. > > rob Yup, this fixed this nasty issue. Thanks, Rob. ACK. Pushed to master, ipa-2-2. Martin From mkosek at redhat.com Fri May 18 07:53:19 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 18 May 2012 09:53:19 +0200 Subject: [Freeipa-devel] [PATCH] 1018 enforce sizelimit when searching for permissions In-Reply-To: <4FB55B69.5080506@redhat.com> References: <4FB55B69.5080506@redhat.com> Message-ID: <1337327599.25787.10.camel@balmora.brq.redhat.com> On Thu, 2012-05-17 at 16:11 -0400, Rob Crittenden wrote: > We do two searches when looking for permissions. One within the > permission object itself and a second in the ACIs. We weren't enforcing > a sizelimit on either search. > > rob This returns the right result, but I don't think it is right with respect to "truncated" flag because of several reasons: 1) You manipulate and set "truncated" flag in post_callback but this won't affect the flag in the returned result because the new value is not propagated outside of the post_callback function. I.e. truncated flag will be set correctly only when it was raised during original permission_find. 2) The part with "ind" is strange: + # enforce --sizelimit + if len(entries) == max_entries: + if ind + 1 < len(results): + truncated = True + break I think it would be much easier to just do ... if (dn, permission) not in entries: if len(entries) < max_entries: entries.append((dn, permission)) else: truncated = True break Otherwise you would rise "truncated" even when the rest of "results" does not contain relevant entries that would have not been added anyway. Martin From pviktori at redhat.com Fri May 18 11:18:17 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 18 May 2012 13:18:17 +0200 Subject: [Freeipa-devel] [RANT] --setattr validation is a minefield. In-Reply-To: <4FB10FFF.4050900@redhat.com> References: <4F843D0F.5060906@redhat.com> <4F844BC4.2020109@redhat.com> <1334077635.16034.20.camel@balmora.brq.redhat.com> <4F846CF8.3020708@redhat.com> <1334080383.2282.9.camel@priserak> <4FABC076.7020308@redhat.com> <1336980986.4344.24.camel@balmora.brq.redhat.com> <4FB10FFF.4050900@redhat.com> Message-ID: <4FB62FF9.3000707@redhat.com> On 05/14/2012 04:00 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On Thu, 2012-05-10 at 15:19 +0200, Petr Viktorin wrote: >>> On 04/10/2012 07:53 PM, Martin Kosek wrote: >>>> On Tue, 2012-04-10 at 19:25 +0200, Petr Viktorin wrote: >>>>> On 04/10/2012 07:07 PM, Martin Kosek wrote: >>>>>> On Tue, 2012-04-10 at 17:03 +0200, Jan Cholasta wrote: >>>>>>> On 10.4.2012 16:00, Petr Viktorin wrote: >>>> [snip] >>>>>>> Like you said above, we should either not validate >>>>>>> --{set,add,del}attr >>>>>>> or don't allow them on known attributes. >>>>>> >>>>>> IMHO, validating attributes we manage in the same way for both >>>>>> --setattr >>>>>> and standard attrs is not that wrong. It is a good precaution, >>>>>> because >>>>>> if we let an unvalidated value in, it can make even a bigger mess >>>>>> later >>>>>> in our pre_callbacks or post_callbacks where we assume that at this >>>>>> point everything is valid. >>>>> >>>>> Then we should validate *exactly* the same way, including not allowing >>>>> no_update attributes to be updated. >>>> >>>> That makes some sense, I could agree with that. >>>> >>> >>> Now that I have a ticket on this >>> (https://fedorahosted.org/freeipa/ticket/2580), I would like to get some >>> wider agreement here. >>> >>> The no_update/no_create attributes are mainly "enabled" flags >>> (ipaenabledflag, nsaccountlock, idnszoneactive), administrative >>> (krbprincipalname, ipauniqueid, ipacertificatesubjectbase), DNS record >>> type and data, and various virtual attributes. >>> >>> If setattr etc. is disabled for all of these, it will mainly matter for >>> the "enabled" flags. To be honest I don't know why we only allow >>> modifying those through special commands. >>> If there's some security reason for that, then setattr etc. should be >>> disabled for them; otherwise I think they should be changeable through >>> xyz-mod. >> >> I am not aware of any security reasons why we use special commands for >> enabling/disabling objects. I assume this is to make it different from >> standard object attribute changes and make sure that user does not >> disable the object "by accident" when doing a mod operation. Rob, maybe >> you remember the reason for this interface.... >> >> But since we already have this approach, we should keep it and implement >> missing "xyz-enable" and "xyz-disable" command so that user's using >> *attr interface to play with enabled/disabled attributes can switch to >> the specialized commands. >> >> So far, I noticed that only DNS zone object misses the enable/disable >> commands, I created a ticket to fix that: >> >> https://fedorahosted.org/freeipa/ticket/2754 >> >>> Either way, setattr etc. should honor the no_update flags. Any >>> objections? >>> >> >> Nope - as long as ticket 2754 is fixed. >> >> Martin >> > > I think those are there so they don't show up for the -mod command since > we have a separate interface for doing it. > > rob For that purpose, would a no_cli flag be better than no_create+no_update? I found one more no_update attribute that might be useful with --setattr: externalhost. -- Petr? From pviktori at redhat.com Fri May 18 13:18:47 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 18 May 2012 15:18:47 +0200 Subject: [Freeipa-devel] [PATCH] 0052 Fix the pwpolicy_find post_callback Message-ID: <4FB64C37.1080304@redhat.com> The pwpolicy-find command had a bad condition that made IPA convert lifetime values for output only if those values were *not* being retrieved. With this patch, the conversion routine is called unconditionally (it has its own check for presence of the attributes). https://fedorahosted.org/freeipa/ticket/2726 Also, have the sorting code use a key function instead of a cmp; this is a Python best practice. Improve a comment -- policies with higher priority are and always were at the *end* of the list, not the beginning. (I'm not sure if we want it this way?) Regression test included. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0052-Fix-the-pwpolicy_find-post_callback.patch Type: text/x-patch Size: 4947 bytes Desc: not available URL: From rcritten at redhat.com Fri May 18 13:36:05 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 18 May 2012 09:36:05 -0400 Subject: [Freeipa-devel] [PATCH] 1018 enforce sizelimit when searching for permissions In-Reply-To: <1337327599.25787.10.camel@balmora.brq.redhat.com> References: <4FB55B69.5080506@redhat.com> <1337327599.25787.10.camel@balmora.brq.redhat.com> Message-ID: <4FB65045.5060706@redhat.com> Martin Kosek wrote: > On Thu, 2012-05-17 at 16:11 -0400, Rob Crittenden wrote: >> We do two searches when looking for permissions. One within the >> permission object itself and a second in the ACIs. We weren't enforcing >> a sizelimit on either search. >> >> rob > > This returns the right result, but I don't think it is right with > respect to "truncated" flag because of several reasons: > > 1) You manipulate and set "truncated" flag in post_callback but this > won't affect the flag in the returned result because the new value is > not propagated outside of the post_callback function. I.e. truncated > flag will be set correctly only when it was raised during original > permission_find. Truncated is still honored as expected. I even added a test case for this. > 2) The part with "ind" is strange: > > + # enforce --sizelimit > + if len(entries) == max_entries: > + if ind + 1< len(results): > + truncated = True > + break > > I think it would be much easier to just do > > ... > if (dn, permission) not in entries: > if len(entries)< max_entries: > entries.append((dn, permission)) > else: > truncated = True > break > > Otherwise you would rise "truncated" even when the rest of "results" > does not contain relevant entries that would have not been added anyway. Yes, that makes sense. rob From rcritten at redhat.com Fri May 18 13:49:05 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 18 May 2012 09:49:05 -0400 Subject: [Freeipa-devel] [RANT] --setattr validation is a minefield. In-Reply-To: <4FB62FF9.3000707@redhat.com> References: <4F843D0F.5060906@redhat.com> <4F844BC4.2020109@redhat.com> <1334077635.16034.20.camel@balmora.brq.redhat.com> <4F846CF8.3020708@redhat.com> <1334080383.2282.9.camel@priserak> <4FABC076.7020308@redhat.com> <1336980986.4344.24.camel@balmora.brq.redhat.com> <4FB10FFF.4050900@redhat.com> <4FB62FF9.3000707@redhat.com> Message-ID: <4FB65351.9090504@redhat.com> Petr Viktorin wrote: > On 05/14/2012 04:00 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On Thu, 2012-05-10 at 15:19 +0200, Petr Viktorin wrote: >>>> On 04/10/2012 07:53 PM, Martin Kosek wrote: >>>>> On Tue, 2012-04-10 at 19:25 +0200, Petr Viktorin wrote: >>>>>> On 04/10/2012 07:07 PM, Martin Kosek wrote: >>>>>>> On Tue, 2012-04-10 at 17:03 +0200, Jan Cholasta wrote: >>>>>>>> On 10.4.2012 16:00, Petr Viktorin wrote: >>>>> [snip] >>>>>>>> Like you said above, we should either not validate >>>>>>>> --{set,add,del}attr >>>>>>>> or don't allow them on known attributes. >>>>>>> >>>>>>> IMHO, validating attributes we manage in the same way for both >>>>>>> --setattr >>>>>>> and standard attrs is not that wrong. It is a good precaution, >>>>>>> because >>>>>>> if we let an unvalidated value in, it can make even a bigger mess >>>>>>> later >>>>>>> in our pre_callbacks or post_callbacks where we assume that at this >>>>>>> point everything is valid. >>>>>> >>>>>> Then we should validate *exactly* the same way, including not >>>>>> allowing >>>>>> no_update attributes to be updated. >>>>> >>>>> That makes some sense, I could agree with that. >>>>> >>>> >>>> Now that I have a ticket on this >>>> (https://fedorahosted.org/freeipa/ticket/2580), I would like to get >>>> some >>>> wider agreement here. >>>> >>>> The no_update/no_create attributes are mainly "enabled" flags >>>> (ipaenabledflag, nsaccountlock, idnszoneactive), administrative >>>> (krbprincipalname, ipauniqueid, ipacertificatesubjectbase), DNS record >>>> type and data, and various virtual attributes. >>>> >>>> If setattr etc. is disabled for all of these, it will mainly matter for >>>> the "enabled" flags. To be honest I don't know why we only allow >>>> modifying those through special commands. >>>> If there's some security reason for that, then setattr etc. should be >>>> disabled for them; otherwise I think they should be changeable through >>>> xyz-mod. >>> >>> I am not aware of any security reasons why we use special commands for >>> enabling/disabling objects. I assume this is to make it different from >>> standard object attribute changes and make sure that user does not >>> disable the object "by accident" when doing a mod operation. Rob, maybe >>> you remember the reason for this interface.... >>> >>> But since we already have this approach, we should keep it and implement >>> missing "xyz-enable" and "xyz-disable" command so that user's using >>> *attr interface to play with enabled/disabled attributes can switch to >>> the specialized commands. >>> >>> So far, I noticed that only DNS zone object misses the enable/disable >>> commands, I created a ticket to fix that: >>> >>> https://fedorahosted.org/freeipa/ticket/2754 >>> >>>> Either way, setattr etc. should honor the no_update flags. Any >>>> objections? >>>> >>> >>> Nope - as long as ticket 2754 is fixed. >>> >>> Martin >>> >> >> I think those are there so they don't show up for the -mod command since >> we have a separate interface for doing it. >> >> rob > > For that purpose, would a no_cli flag be better than no_create+no_update? > > > I found one more no_update attribute that might be useful with > --setattr: externalhost. > I think the flags are really only considered on the cli now. If a different flag can make the intention clearer I'm ok with that. Having two flags seems more flexible though. rob From rcritten at redhat.com Fri May 18 14:03:00 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 18 May 2012 10:03:00 -0400 Subject: [Freeipa-devel] [PATCH] 0052 Fix the pwpolicy_find post_callback In-Reply-To: <4FB64C37.1080304@redhat.com> References: <4FB64C37.1080304@redhat.com> Message-ID: <4FB65694.8000505@redhat.com> Petr Viktorin wrote: > The pwpolicy-find command had a bad condition that made IPA convert > lifetime values for output only if those values were *not* being retrieved. > With this patch, the conversion routine is called unconditionally (it > has its own check for presence of the attributes). > > https://fedorahosted.org/freeipa/ticket/2726 > > Also, have the sorting code use a key function instead of a cmp; this is > a Python best practice. > Improve a comment -- policies with higher priority are and always were > at the *end* of the list, not the beginning. (I'm not sure if we want it > this way?) > > Regression test included. I haven't checked this in detail yet but the comment about priority may be wrong. The priority is such that the lower the number the higher the priority. I think that is what it meant. rob From rcritten at redhat.com Fri May 18 14:17:48 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 18 May 2012 10:17:48 -0400 Subject: [Freeipa-devel] [PATCH] 1018 enforce sizelimit when searching for permissions In-Reply-To: <4FB65045.5060706@redhat.com> References: <4FB55B69.5080506@redhat.com> <1337327599.25787.10.camel@balmora.brq.redhat.com> <4FB65045.5060706@redhat.com> Message-ID: <4FB65A0C.5030306@redhat.com> Rob Crittenden wrote: > Martin Kosek wrote: >> On Thu, 2012-05-17 at 16:11 -0400, Rob Crittenden wrote: >>> We do two searches when looking for permissions. One within the >>> permission object itself and a second in the ACIs. We weren't enforcing >>> a sizelimit on either search. >>> >>> rob >> >> This returns the right result, but I don't think it is right with >> respect to "truncated" flag because of several reasons: >> >> 1) You manipulate and set "truncated" flag in post_callback but this >> won't affect the flag in the returned result because the new value is >> not propagated outside of the post_callback function. I.e. truncated >> flag will be set correctly only when it was raised during original >> permission_find. > > Truncated is still honored as expected. I even added a test case for this. > >> 2) The part with "ind" is strange: >> >> + # enforce --sizelimit >> + if len(entries) == max_entries: >> + if ind + 1< len(results): >> + truncated = True >> + break >> >> I think it would be much easier to just do >> >> ... >> if (dn, permission) not in entries: >> if len(entries)< max_entries: >> entries.append((dn, permission)) >> else: >> truncated = True >> break >> >> Otherwise you would rise "truncated" even when the rest of "results" >> does not contain relevant entries that would have not been added anyway. > > Yes, that makes sense. And now updated patch. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1018-2-sizelimit.patch Type: text/x-diff Size: 4727 bytes Desc: not available URL: From mkosek at redhat.com Fri May 18 14:24:26 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 18 May 2012 16:24:26 +0200 Subject: [Freeipa-devel] [PATCH] 0052 Fix the pwpolicy_find post_callback In-Reply-To: <4FB65694.8000505@redhat.com> References: <4FB64C37.1080304@redhat.com> <4FB65694.8000505@redhat.com> Message-ID: <1337351066.25787.51.camel@balmora.brq.redhat.com> On Fri, 2012-05-18 at 10:03 -0400, Rob Crittenden wrote: > Petr Viktorin wrote: > > The pwpolicy-find command had a bad condition that made IPA convert > > lifetime values for output only if those values were *not* being retrieved. > > With this patch, the conversion routine is called unconditionally (it > > has its own check for presence of the attributes). > > > > https://fedorahosted.org/freeipa/ticket/2726 > > > > Also, have the sorting code use a key function instead of a cmp; this is > > a Python best practice. > > Improve a comment -- policies with higher priority are and always were > > at the *end* of the list, not the beginning. (I'm not sure if we want it > > this way?) > > > > Regression test included. > > I haven't checked this in detail yet but the comment about priority may > be wrong. The priority is such that the lower the number the higher the > priority. I think that is what it meant. > > rob > Yup, the comment was right. Lower numbers, i.e. password policies with higher priority, were sorted in the top. Martin From pviktori at redhat.com Fri May 18 14:41:02 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 18 May 2012 16:41:02 +0200 Subject: [Freeipa-devel] [PATCH] 0052 Fix the pwpolicy_find post_callback In-Reply-To: <1337351066.25787.51.camel@balmora.brq.redhat.com> References: <4FB64C37.1080304@redhat.com> <4FB65694.8000505@redhat.com> <1337351066.25787.51.camel@balmora.brq.redhat.com> Message-ID: <4FB65F7E.8090509@redhat.com> On 05/18/2012 04:24 PM, Martin Kosek wrote: > On Fri, 2012-05-18 at 10:03 -0400, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> The pwpolicy-find command had a bad condition that made IPA convert >>> lifetime values for output only if those values were *not* being retrieved. >>> With this patch, the conversion routine is called unconditionally (it >>> has its own check for presence of the attributes). >>> >>> https://fedorahosted.org/freeipa/ticket/2726 >>> >>> Also, have the sorting code use a key function instead of a cmp; this is >>> a Python best practice. >>> Improve a comment -- policies with higher priority are and always were >>> at the *end* of the list, not the beginning. (I'm not sure if we want it >>> this way?) >>> >>> Regression test included. >> >> I haven't checked this in detail yet but the comment about priority may >> be wrong. The priority is such that the lower the number the higher the >> priority. I think that is what it meant. >> >> rob >> > > Yup, the comment was right. Lower numbers, i.e. password policies with > higher priority, were sorted in the top. > > Martin > Ah, I see. Made it clear so the next person is not confused. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0052-02-Fix-the-pwpolicy_find-post_callback.patch Type: text/x-patch Size: 4981 bytes Desc: not available URL: From pviktori at redhat.com Fri May 18 14:54:50 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 18 May 2012 16:54:50 +0200 Subject: [Freeipa-devel] [RANT] --setattr validation is a minefield. In-Reply-To: <4FB65351.9090504@redhat.com> References: <4F843D0F.5060906@redhat.com> <4F844BC4.2020109@redhat.com> <1334077635.16034.20.camel@balmora.brq.redhat.com> <4F846CF8.3020708@redhat.com> <1334080383.2282.9.camel@priserak> <4FABC076.7020308@redhat.com> <1336980986.4344.24.camel@balmora.brq.redhat.com> <4FB10FFF.4050900@redhat.com> <4FB62FF9.3000707@redhat.com> <4FB65351.9090504@redhat.com> Message-ID: <4FB662BA.1040205@redhat.com> On 05/18/2012 03:49 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> On 05/14/2012 04:00 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> On Thu, 2012-05-10 at 15:19 +0200, Petr Viktorin wrote: >>>>> On 04/10/2012 07:53 PM, Martin Kosek wrote: >>>>>> On Tue, 2012-04-10 at 19:25 +0200, Petr Viktorin wrote: >>>>>>> On 04/10/2012 07:07 PM, Martin Kosek wrote: >>>>>>>> On Tue, 2012-04-10 at 17:03 +0200, Jan Cholasta wrote: >>>>>>>>> On 10.4.2012 16:00, Petr Viktorin wrote: >>>>>> [snip] >>>>>>>>> Like you said above, we should either not validate >>>>>>>>> --{set,add,del}attr >>>>>>>>> or don't allow them on known attributes. >>>>>>>> >>>>>>>> IMHO, validating attributes we manage in the same way for both >>>>>>>> --setattr >>>>>>>> and standard attrs is not that wrong. It is a good precaution, >>>>>>>> because >>>>>>>> if we let an unvalidated value in, it can make even a bigger mess >>>>>>>> later >>>>>>>> in our pre_callbacks or post_callbacks where we assume that at this >>>>>>>> point everything is valid. >>>>>>> >>>>>>> Then we should validate *exactly* the same way, including not >>>>>>> allowing >>>>>>> no_update attributes to be updated. >>>>>> >>>>>> That makes some sense, I could agree with that. >>>>>> >>>>> >>>>> Now that I have a ticket on this >>>>> (https://fedorahosted.org/freeipa/ticket/2580), I would like to get >>>>> some >>>>> wider agreement here. >>>>> >>>>> The no_update/no_create attributes are mainly "enabled" flags >>>>> (ipaenabledflag, nsaccountlock, idnszoneactive), administrative >>>>> (krbprincipalname, ipauniqueid, ipacertificatesubjectbase), DNS record >>>>> type and data, and various virtual attributes. >>>>> >>>>> If setattr etc. is disabled for all of these, it will mainly matter >>>>> for >>>>> the "enabled" flags. To be honest I don't know why we only allow >>>>> modifying those through special commands. >>>>> If there's some security reason for that, then setattr etc. should be >>>>> disabled for them; otherwise I think they should be changeable through >>>>> xyz-mod. >>>> >>>> I am not aware of any security reasons why we use special commands for >>>> enabling/disabling objects. I assume this is to make it different from >>>> standard object attribute changes and make sure that user does not >>>> disable the object "by accident" when doing a mod operation. Rob, maybe >>>> you remember the reason for this interface.... >>>> >>>> But since we already have this approach, we should keep it and >>>> implement >>>> missing "xyz-enable" and "xyz-disable" command so that user's using >>>> *attr interface to play with enabled/disabled attributes can switch to >>>> the specialized commands. >>>> >>>> So far, I noticed that only DNS zone object misses the enable/disable >>>> commands, I created a ticket to fix that: >>>> >>>> https://fedorahosted.org/freeipa/ticket/2754 >>>> >>>>> Either way, setattr etc. should honor the no_update flags. Any >>>>> objections? >>>>> >>>> >>>> Nope - as long as ticket 2754 is fixed. >>>> >>>> Martin >>>> >>> >>> I think those are there so they don't show up for the -mod command since >>> we have a separate interface for doing it. >>> >>> rob >> >> For that purpose, would a no_cli flag be better than no_create+no_update? >> >> >> I found one more no_update attribute that might be useful with >> --setattr: externalhost. >> > > I think the flags are really only considered on the cli now. If a > different flag can make the intention clearer I'm ok with that. Having > two flags seems more flexible though. > > rob No. When the no_update flag is specified, the param doesn't get cloned from the object to the -mod command at all. (A lot of the getattr problems occured because this cloning also sets important data, the "attribute" bit, so the uncloned params are incomplete/unusable but the cloned ones aren't available.) So, I want to have no_create/no_update for things we really don't want the user to change (even through setattr), and "hidden" (a better name for it than no_cli IMO) for things we just don't advertise because there's a different way to do it. It would also make sense to make the "hidden" attributes available in -find ??? AFAIK, currently you can't search for disabled users for example. As for two flags being more flexible, I can't find a case where we'd want to hide an attribute in -mod but not in -add (or vice versa). I think that'd just be an unnecessary inconsistency. (Remember this is just hiding the param from the frontend; "krbprincipalname" is rightly no_update only.) Also, in the future, I think the enable/disable commands should call the -mod command (or the other way around), so there's just one code path for bugs to hide in. -- Petr? From rcritten at redhat.com Fri May 18 15:53:13 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 18 May 2012 11:53:13 -0400 Subject: [Freeipa-devel] [PATCH] 1019 require policycoreutils if SELinux is enabled Message-ID: <4FB67069.1040901@redhat.com> We don't have an explicit requires on the policycoreutils package in the client because SELinux is not required (just recommended). SELinux can be enabled without this package so check for that condition and don't allow installation if it is the case. The resulting install will be rather broken. Also check on the server when installing. This should never happen but in theory it could do the server install then fail in the client because of this. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1019-selinux.patch Type: text/x-diff Size: 8698 bytes Desc: not available URL: From rcritten at redhat.com Fri May 18 18:30:27 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 18 May 2012 14:30:27 -0400 Subject: [Freeipa-devel] [PATCH] 259 Remove ipa-server-install LDAP update errors In-Reply-To: <1336749023.3067.13.camel@priserak> References: <1336640545.27129.12.camel@balmora.brq.redhat.com> <1336749023.3067.13.camel@priserak> Message-ID: <4FB69543.8080700@redhat.com> Martin Kosek wrote: > On Thu, 2012-05-10 at 11:02 +0200, Martin Kosek wrote: >> LDAP addEntry method raises an exception when a parent entry of >> the entry being added does not exist. This may not be an error, >> for example NIS entries are only added when NIS is enabled and >> thus the NIS entry container exists. >> >> This patch adds an appropriate check so that we rather add >> a debug message to ipaupgrade.log instead of raising a user >> visible error. >> >> https://fedorahosted.org/freeipa/ticket/2743 >> > > I got inspired in ticket #2520 and prepared a better solution which > fixes both the incorrect exception processing in ipaldap + handles > gracefully the missing parent entry situation without emitting extra > LDAP query. Patch is attached. > > Martin ACK, pushed to master rob From rcritten at redhat.com Fri May 18 20:03:02 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 18 May 2012 16:03:02 -0400 Subject: [Freeipa-devel] [PATCH] 1014 configurable service timeout In-Reply-To: <4FB3F267.2070104@redhat.com> References: <4FB3F267.2070104@redhat.com> Message-ID: <4FB6AAF6.9080800@redhat.com> Rob Crittenden wrote: > A hardcoded timeout was used in ipactl for service restarts, set rather > low. A separate timeout was hardcoded into the installer. > > I centralized them into a single timeout, configurable in the standard > way in /etc/ipa/*.conf. > > On install it will always default to 120 seconds and remain there unless > changed in default.conf (not replicated either). > > I tested this on systemd systems and sysV systems and it works ok for > me. You'll also want to double-check that this works when other 389-ds > instances are installed. > > Getting the naming of instances right was a bit tricky. Noticed a problem on upgrades and fixed that. Updated patch attached. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1014-2-timeout.patch Type: text/x-diff Size: 28890 bytes Desc: not available URL: From encukou at gmail.com Fri May 18 23:06:09 2012 From: encukou at gmail.com (Petr Viktorin) Date: Sat, 19 May 2012 01:06:09 +0200 Subject: [Freeipa-devel] FreeIPA mirror on Github Message-ID: Hello, A friendly Githubber recently turned off consistency checks for my FreeIPA repository on Github, so I made a mirror there. If people fork it, they should be able to push new patches without problems. This should slightly lower the entry barrier for people like William Brown who want to maintain longer-lived feature branches. I'll try to keep the mirror updated, and refer people to the proper patch/bug submission channels if they use the Github features. https://github.com/encukou/freeipa -- Petr? Viktorin (from my personal mail account) From hahaha_30k at yahoo.com Sun May 20 09:22:17 2012 From: hahaha_30k at yahoo.com (Gelen James) Date: Sun, 20 May 2012 02:22:17 -0700 (PDT) Subject: [Freeipa-devel] Feature request: Web UI for IPA users to reset their own expired passwords Message-ID: <1337505737.89279.YahooMailNeo@web160703.mail.bf1.yahoo.com> The currently assumption is that all IPA users can login into Unix/Linux machines to change their IPA password, or reset their expired password.? ?But this is not available all the time, so a more general alternative -- web UI -- will be more appreciated. The basic requirements are: ?1, The web UI accept user's passwords, expired is also accepted. ? ?2, the authentication is based on IPA Kerberos. ?3, authenticated regular IPA user can only reset his/her password only. ?4, (bonus) authenticated admin users can alter other users' password as well. Thanks. --Gelen -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkosek at redhat.com Mon May 21 07:33:49 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 21 May 2012 09:33:49 +0200 Subject: [Freeipa-devel] Feature request: Web UI for IPA users to reset their own expired passwords In-Reply-To: <1337505737.89279.YahooMailNeo@web160703.mail.bf1.yahoo.com> References: <1337505737.89279.YahooMailNeo@web160703.mail.bf1.yahoo.com> Message-ID: <1337585629.20955.18.camel@balmora.brq.redhat.com> On Sun, 2012-05-20 at 02:22 -0700, Gelen James wrote: > The currently assumption is that all IPA users can login into > Unix/Linux machines to change their IPA password, or reset their > expired password. > > > But this is not available all the time, so a more general alternative > -- web UI -- will be more appreciated. The basic requirements are: > > > 1, The web UI accept user's passwords, expired is also accepted. Hello Gelen, Current Web UI allows only users with valid and non-expired password to log in. There is a ticket logged to improve this: https://fedorahosted.org/freeipa/ticket/2276 With this change in, users with expired passwords will be able to log in and change the expired password right after successful authentication. This feature is planned to be released as a part of FreeIPA 3.0. > > 2, the authentication is based on IPA Kerberos. > > > 3, authenticated regular IPA user can only reset his/her password > only. > > > 4, (bonus) authenticated admin users can alter other users' password > as well. All these features are already available in current upstream version of FreeIPA. For 4), this can be done also by non-admin user that has an appropriate privilege granted. Martin From pviktori at redhat.com Mon May 21 11:58:20 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 21 May 2012 13:58:20 +0200 Subject: [Freeipa-devel] [PATCH] 0053 Disallow setattr on no_update/no_create params Message-ID: <4FBA2DDC.7010607@redhat.com> Only use no_create/no_update for things we really don't want the user to change (even through setattr). This is stuff like ipacertificatesubjectbase. Make --{set,add,del}attr refuse to modify these params. For things we just don't advertise in the because there's a different way to do change them, there is the "no_option" flag (undocumented before this patch). This only causes the option to be hidden from the CLI; XML-RPC will still happily take it (and it will appear in API.txt). Use this for ipaenabledflag, nsacconuntlock, and externalhost. https://fedorahosted.org/freeipa/ticket/2580 -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0053-Disallow-setattr-on-no_update-no_create-params.patch Type: text/x-patch Size: 36059 bytes Desc: not available URL: From rcritten at redhat.com Mon May 21 20:32:30 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 21 May 2012 16:32:30 -0400 Subject: [Freeipa-devel] [PATCH] 1020 replication conversion retry Message-ID: <4FBAA65E.1090707@redhat.com> When converting to GSSAPI replication we need to fetch the ldap principal from the other side. We've seen this fail from time to time despite having a call to force_sync. Add a retry loop to try harder, and fix the error reporting. I was never able to force reproduction of the underlying problem. I used the netem module to force a very slow link and wasn't able to duplicate the problem (I added a 500ms delay and a server with a couple of hundred entries). tc qdisc add dev p3p1 root netem delay 500ms rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1020-replication.patch Type: text/x-diff Size: 6297 bytes Desc: not available URL: From rcritten at redhat.com Mon May 21 20:40:14 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 21 May 2012 16:40:14 -0400 Subject: [Freeipa-devel] [PATCH] 1021 index fqdn for 2.2. branch Message-ID: <4FBAA82E.7090404@redhat.com> We already have an index on fqdn in the master branch. Add this to the 2.2 branch as well. We do a search on host when installing a replica and an unindexed search might fail. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1021-fqdnindex.patch Type: text/x-diff Size: 1662 bytes Desc: not available URL: From simo at redhat.com Tue May 22 00:51:15 2012 From: simo at redhat.com (Simo Sorce) Date: Mon, 21 May 2012 20:51:15 -0400 Subject: [Freeipa-devel] [PATCH] 1020 replication conversion retry In-Reply-To: <4FBAA65E.1090707@redhat.com> References: <4FBAA65E.1090707@redhat.com> Message-ID: <1337647875.16840.174.camel@willson.li.ssimo.org> On Mon, 2012-05-21 at 16:32 -0400, Rob Crittenden wrote: > When converting to GSSAPI replication we need to fetch the ldap > principal from the other side. We've seen this fail from time to time > despite having a call to force_sync. Add a retry loop to try harder, > and > fix the error reporting. > > I was never able to force reproduction of the underlying problem. I > used > the netem module to force a very slow link and wasn't able to > duplicate > the problem (I added a 500ms delay and a server with a couple of > hundred > entries). > > tc qdisc add dev p3p1 root netem delay 500ms > ACK -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Tue May 22 00:51:42 2012 From: simo at redhat.com (Simo Sorce) Date: Mon, 21 May 2012 20:51:42 -0400 Subject: [Freeipa-devel] [PATCH] 1021 index fqdn for 2.2. branch In-Reply-To: <4FBAA82E.7090404@redhat.com> References: <4FBAA82E.7090404@redhat.com> Message-ID: <1337647902.16840.175.camel@willson.li.ssimo.org> On Mon, 2012-05-21 at 16:40 -0400, Rob Crittenden wrote: > We already have an index on fqdn in the master branch. Add this to > the > 2.2 branch as well. We do a search on host when installing a replica > and > an unindexed search might fail. ACK Simo. -- Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Tue May 22 06:28:16 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 22 May 2012 08:28:16 +0200 Subject: [Freeipa-devel] [PATCH] 1021 index fqdn for 2.2. branch In-Reply-To: <1337647902.16840.175.camel@willson.li.ssimo.org> References: <4FBAA82E.7090404@redhat.com> <1337647902.16840.175.camel@willson.li.ssimo.org> Message-ID: <1337668096.13976.0.camel@balmora.brq.redhat.com> On Mon, 2012-05-21 at 20:51 -0400, Simo Sorce wrote: > On Mon, 2012-05-21 at 16:40 -0400, Rob Crittenden wrote: > > We already have an index on fqdn in the master branch. Add this to > > the > > 2.2 branch as well. We do a search on host when installing a replica > > and > > an unindexed search might fail. > > ACK > > Simo. > Pushed to ipa-2-2. Martin From pviktori at redhat.com Tue May 22 10:15:32 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 22 May 2012 12:15:32 +0200 Subject: [Freeipa-devel] [PATCH] 258 Remove LDAP limits from DNS service In-Reply-To: <1336635104.27129.7.camel@balmora.brq.redhat.com> References: <1336635104.27129.7.camel@balmora.brq.redhat.com> Message-ID: <4FBB6744.3020101@redhat.com> On 05/10/2012 09:31 AM, Martin Kosek wrote: > bind-dyndb-ldap persistent search queries LDAP for all DNS records. > The LDAP connection must have no size or time limits to work > properly. > > This patch updates limits both for existing service principal > on updated machine and for new service principals added > as a part of DNS installation. > > https://fedorahosted.org/freeipa/ticket/2531 > Yes, the limits are set correctly on both install and upgrade. ACK Minor style nitpicks: > + root_logger.critical("Could not modify principal's %s entry: %s" \ > + % (dns_principal, str(e))) The backshashes are unnecessary inside parentheses. > + dnsupdates[dns_service_dn] = {'dn' : dns_service_dn, > + 'updates' : limit_updates} Don't put a space before a semicolon. -- Petr? From mkosek at redhat.com Tue May 22 10:35:15 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 22 May 2012 12:35:15 +0200 Subject: [Freeipa-devel] [PATCH] 258 Remove LDAP limits from DNS service In-Reply-To: <4FBB6744.3020101@redhat.com> References: <1336635104.27129.7.camel@balmora.brq.redhat.com> <4FBB6744.3020101@redhat.com> Message-ID: <1337682915.17468.1.camel@balmora.brq.redhat.com> On Tue, 2012-05-22 at 12:15 +0200, Petr Viktorin wrote: > On 05/10/2012 09:31 AM, Martin Kosek wrote: > > bind-dyndb-ldap persistent search queries LDAP for all DNS records. > > The LDAP connection must have no size or time limits to work > > properly. > > > > This patch updates limits both for existing service principal > > on updated machine and for new service principals added > > as a part of DNS installation. > > > > https://fedorahosted.org/freeipa/ticket/2531 > > > > Yes, the limits are set correctly on both install and upgrade. > ACK > > > > Minor style nitpicks: > > > + root_logger.critical("Could not modify principal's %s entry: %s" \ > > + % (dns_principal, str(e))) > > The backshashes are unnecessary inside parentheses. > > > + dnsupdates[dns_service_dn] = {'dn' : dns_service_dn, > > + 'updates' : limit_updates} > > Don't put a space before a semicolon. I removed the space before the colon. Pushed to master. Martin From ohamada at redhat.com Tue May 22 10:38:22 2012 From: ohamada at redhat.com (Ondrej Hamada) Date: Tue, 22 May 2012 12:38:22 +0200 Subject: [Freeipa-devel] [PATCH] 25 ipa-server-install: s/calculated/determined/ Message-ID: <4FBB6C9E.6070305@redhat.com> https://fedorahosted.org/freeipa/ticket/2704 Output message of the 'read_domain_name' function in ipa-server-install was reworded. -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-ohamada-25-ipa-server-install-reword-message.patch Type: text/x-patch Size: 1110 bytes Desc: not available URL: From pviktori at redhat.com Tue May 22 12:41:48 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 22 May 2012 14:41:48 +0200 Subject: [Freeipa-devel] [PATCH] 260 Replace DNS client based on acutil with python-dns In-Reply-To: <1337154273.2963.8.camel@balmora.brq.redhat.com> References: <1336755130.19268.12.camel@priserak> <4FB245F2.6000500@redhat.com> <1337154273.2963.8.camel@balmora.brq.redhat.com> Message-ID: <4FBB898C.2030009@redhat.com> On 05/16/2012 09:44 AM, Martin Kosek wrote: > On Tue, 2012-05-15 at 14:02 +0200, Petr Viktorin wrote: >> > On 05/11/2012 06:52 PM, Martin Kosek wrote: >>> > > python-dns is very feature-rich and it can help us a lot with our DNS >>> > > related code. This patch does the first step, i.e. replaces acutil use >>> > > with python-dns, which is more convenient to use as you will see in the >>> > > patch. More integration will follow in the future. >>> > > >>> > > I send this patch rather early, so that I can get responses to this >>> > > patch early and also so that we are able to catch issues in a safe >>> > > distance from the next release. >> > >>> > > --- >>> > > IPA client and server tool set used authconfig acutil module to >>> > > for client DNS operations. This is not optimal DNS interface for >>> > > several reasons: >>> > > - does not provide native Python object oriented interface >>> > > but but rather C-like interface based on functions and >>> > > structures which is not easy to use and extend >>> > > - acutil is not meant to be used by third parties besides >>> > > authconfig and thus can break without notice >>> > > >>> > > Replace the acutil with python-dns package which has a feature rich >>> > > interface for dealing with all different aspects of DNS including >>> > > DNSSEC. The main target of this patch is to replace all uses of >>> > > acutil DNS library with a use python-dns. In most cases, even >>> > > though the larger parts of the code are changed, the actual >>> > > functionality is changed only in the following cases: >>> > > - redundant DNS checks were removed from verify_fqdn function >>> > > in installutils to make the whole DNS check simpler and >>> > > less error-prone. Logging was improves for the remaining >>> > > checks >>> > > - improved logging for ipa-client-install DNS discovery >>> > > >>> > > https://fedorahosted.org/freeipa/ticket/2730 >> > [...] > Fixed. > > Martin I've been testing the patches in various setups and haven't found a regression so far. ACK on 261, for 260 I have a comment below. > diff --git a/ipa-client/ipaclient/ipadiscovery.py b/ipa-client/ipaclient/ipadiscovery.py > index 86bef28b2d7fdfc8111b493bddec7ac6888f021a..1889d1918c01fe0c80a3d56b9a7ef304c5a7d97c 100644 > --- a/ipa-client/ipaclient/ipadiscovery.py > +++ b/ipa-client/ipaclient/ipadiscovery.py > @@ -310,84 +313,74 @@ class IPADiscovery: > os.rmdir(temp_ca_dir) > > > - def ipadnssearchldap(self, tdomain): > - servers = "" > - rserver = "" > + def ipadns_search_srv(self, domain, srv_record_name, default_port, > + get_first=True): > + """ > + Search for SRV records in given domain. When no record is found, > + en empty string is returned > > - qname = "_ldap._tcp."+tdomain > - # terminate the name > - if not qname.endswith("."): > - qname += "." > - results = ipapython.dnsclient.query(qname, ipapython.dnsclient.DNS_C_IN, ipapython.dnsclient.DNS_T_SRV) > + :param domain: Search domain name > + :param srv_record_name: SRV record name, e.g. "_ldap._tcp" > + :param default_port: When default_port is not None, it is being > + checked with the port in SRV record and if they don't > + match, the port from SRV record is appended to > + found hostname in this format: "hostname:port" > + :param get_first: break on first find, otherwise multiple finds > + separated by ":" may be returned They are separated by ",". In the calling code, for splitting a comma-separated string it is better to use servers.split(',') than ipautil.parse_items(servers). Or, return a list directly from here. -- Petr? From pviktori at redhat.com Tue May 22 12:50:23 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 22 May 2012 14:50:23 +0200 Subject: [Freeipa-devel] [PATCH] 25 ipa-server-install: s/calculated/determined/ In-Reply-To: <4FBB6C9E.6070305@redhat.com> References: <4FBB6C9E.6070305@redhat.com> Message-ID: <4FBB8B8F.60007@redhat.com> On 05/22/2012 12:38 PM, Ondrej Hamada wrote: > https://fedorahosted.org/freeipa/ticket/2704 > > Output message of the 'read_domain_name' function in ipa-server-install > was reworded. > ACK -- Petr? From mkosek at redhat.com Tue May 22 13:11:57 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 22 May 2012 15:11:57 +0200 Subject: [Freeipa-devel] [PATCH] 1018 enforce sizelimit when searching for permissions In-Reply-To: <4FB65A0C.5030306@redhat.com> References: <4FB55B69.5080506@redhat.com> <1337327599.25787.10.camel@balmora.brq.redhat.com> <4FB65045.5060706@redhat.com> <4FB65A0C.5030306@redhat.com> Message-ID: <1337692317.17468.8.camel@balmora.brq.redhat.com> On Fri, 2012-05-18 at 10:17 -0400, Rob Crittenden wrote: > Rob Crittenden wrote: > > Martin Kosek wrote: > >> On Thu, 2012-05-17 at 16:11 -0400, Rob Crittenden wrote: > >>> We do two searches when looking for permissions. One within the > >>> permission object itself and a second in the ACIs. We weren't enforcing > >>> a sizelimit on either search. > >>> > >>> rob > >> > >> This returns the right result, but I don't think it is right with > >> respect to "truncated" flag because of several reasons: > >> > >> 1) You manipulate and set "truncated" flag in post_callback but this > >> won't affect the flag in the returned result because the new value is > >> not propagated outside of the post_callback function. I.e. truncated > >> flag will be set correctly only when it was raised during original > >> permission_find. > > > > Truncated is still honored as expected. I even added a test case for this. Yes, but it only works when the truncated flag is raised by the base LDAP search, i.e. the search for permission objects (which is a case of your unit test). If the search does not raise the flag and it is set later in post callback, it is never propagated to the user. Please check the attached (crappy) test that shows this issue: ====================================================================== FAIL: test_permission[20]: permission_find: Search for permissions by attr with a limit of 1 (truncated) ---------------------------------------------------------------------- ... AssertionError: assert_deepequal: expected != got. test_permission[20]: permission_find: Search for permissions by attr with a limit of 1 (truncated) expected = True got = False path = ('truncated',) I am not sure how to solve this right, we may need to add a new return attribute (truncated) to all LDAPSearch post callbacks so that the post callback can really modify it - something similar we already do with pre callbacks which are able to change LDAP search filter, scope etc. > > > >> 2) The part with "ind" is strange: > >> > >> + # enforce --sizelimit > >> + if len(entries) == max_entries: > >> + if ind + 1< len(results): > >> + truncated = True > >> + break > >> > >> I think it would be much easier to just do > >> > >> ... > >> if (dn, permission) not in entries: > >> if len(entries)< max_entries: > >> entries.append((dn, permission)) > >> else: > >> truncated = True > >> break > >> > >> Otherwise you would rise "truncated" even when the rest of "results" > >> does not contain relevant entries that would have not been added anyway. > > > > Yes, that makes sense. > > And now updated patch. We can now also remove the "enumerate" part, "ind" is no longer needed. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: test-aci_find-truncated.patch Type: text/x-patch Size: 1833 bytes Desc: not available URL: From mkosek at redhat.com Tue May 22 13:21:27 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 22 May 2012 15:21:27 +0200 Subject: [Freeipa-devel] [PATCH] 25 ipa-server-install: s/calculated/determined/ In-Reply-To: <4FBB8B8F.60007@redhat.com> References: <4FBB6C9E.6070305@redhat.com> <4FBB8B8F.60007@redhat.com> Message-ID: <1337692887.17468.9.camel@balmora.brq.redhat.com> On Tue, 2012-05-22 at 14:50 +0200, Petr Viktorin wrote: > On 05/22/2012 12:38 PM, Ondrej Hamada wrote: > > https://fedorahosted.org/freeipa/ticket/2704 > > > > Output message of the 'read_domain_name' function in ipa-server-install > > was reworded. > > > > ACK > > Pushed to master. Martin From pviktori at redhat.com Tue May 22 13:39:32 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 22 May 2012 15:39:32 +0200 Subject: [Freeipa-devel] [PATCH] 1019 require policycoreutils if SELinux is enabled In-Reply-To: <4FB67069.1040901@redhat.com> References: <4FB67069.1040901@redhat.com> Message-ID: <4FBB9714.6020100@redhat.com> On 2012-05-18 17:53, Rob Crittenden wrote: > We don't have an explicit requires on the policycoreutils package in the > client because SELinux is not required (just recommended). > > SELinux can be enabled without this package so check for that condition > and don't allow installation if it is the case. The resulting install > will be rather broken. > > Also check on the server when installing. This should never happen but > in theory it could do the server install then fail in the client because > of this. > > rob > All other platform-dependent services have a default defined in ipapython/services.py.in. Shouldn't check_selinux_status have one, too? -- Petr? From pviktori at redhat.com Tue May 22 13:45:58 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 22 May 2012 15:45:58 +0200 Subject: [Freeipa-devel] [PATCH] 0040 Move install script error handling to a common function In-Reply-To: <4F956FB7.3040000@redhat.com> References: <4F951EB7.9050908@redhat.com> <4F956FB7.3040000@redhat.com> Message-ID: <4FBB9896.7020908@redhat.com> On 2012-04-23 17:05, John Dennis wrote: > On 04/23/2012 05:19 AM, Petr Viktorin wrote: >> This fixes https://fedorahosted.org/freeipa/ticket/2071 (Add final debug >> message in installers). >> >> I submitted an earlier version of this patch before (0014), but it was >> too much to include in 2.2. Hopefully now there's more space for >> restructuring. I think it's better to start a new thread with this >> approach. >> >> The try/except blocks at the end of installers/management scripts are >> replaced by a call to a common function, which includes the final >> message. >> For each specific error, the error handlers in all scripts was almost >> the same, but each script handled a different selection of errors. >> Instead of having this copy/pasted code (with subtle differences >> creeping in over time), this patch consolidates it all in one place. > > I like this approach much better than the earlier patch, great, thanks. > I'm a big fan of calling into common code instead of copying code to my > mind the refactoring to utilize common code is great approach. I also > like the fact the logging configuration is not modified after it's > established. > > At some point we may want to revist how the log messages are generated. > For example should all communication to the console pass through the > console handler? Is there a logger established for the script? Should > the format of messages emitted to the console be altered? Should all > command line utilities accept the both the verbose and debug flag? Etc. > But for now this is fantastic start in the right direction. > > I have not installed and exercised the patch so I can't comment on any > runtime time issues that might be present, but from code inspection only > it has my ACK. > > Thanks John! Yes, this is just a start. Patch rebased to curent master -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0040-02-Move-install-script-error-handling-to-a-common-funct.patch Type: text/x-patch Size: 35284 bytes Desc: not available URL: From rcritten at redhat.com Tue May 22 20:45:39 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 22 May 2012 16:45:39 -0400 Subject: [Freeipa-devel] [PATCH] 1018 enforce sizelimit when searching for permissions In-Reply-To: <1337692317.17468.8.camel@balmora.brq.redhat.com> References: <4FB55B69.5080506@redhat.com> <1337327599.25787.10.camel@balmora.brq.redhat.com> <4FB65045.5060706@redhat.com> <4FB65A0C.5030306@redhat.com> <1337692317.17468.8.camel@balmora.brq.redhat.com> Message-ID: <4FBBFAF3.2070106@redhat.com> Martin Kosek wrote: > On Fri, 2012-05-18 at 10:17 -0400, Rob Crittenden wrote: >> Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> On Thu, 2012-05-17 at 16:11 -0400, Rob Crittenden wrote: >>>>> We do two searches when looking for permissions. One within the >>>>> permission object itself and a second in the ACIs. We weren't enforcing >>>>> a sizelimit on either search. >>>>> >>>>> rob >>>> >>>> This returns the right result, but I don't think it is right with >>>> respect to "truncated" flag because of several reasons: >>>> >>>> 1) You manipulate and set "truncated" flag in post_callback but this >>>> won't affect the flag in the returned result because the new value is >>>> not propagated outside of the post_callback function. I.e. truncated >>>> flag will be set correctly only when it was raised during original >>>> permission_find. >>> >>> Truncated is still honored as expected. I even added a test case for this. > > Yes, but it only works when the truncated flag is raised by the base > LDAP search, i.e. the search for permission objects (which is a case of > your unit test). If the search does not raise the flag and it is set > later in post callback, it is never propagated to the user. Please check > the attached (crappy) test that shows this issue: > > ====================================================================== > FAIL: test_permission[20]: permission_find: Search for permissions by > attr with a limit of 1 (truncated) > ---------------------------------------------------------------------- > ... > AssertionError: assert_deepequal: expected != got. > test_permission[20]: permission_find: Search for permissions by attr > with a limit of 1 (truncated) > expected = True > got = False > path = ('truncated',) > > I am not sure how to solve this right, we may need to add a new return > attribute (truncated) to all LDAPSearch post callbacks so that the post > callback can really modify it - something similar we already do with pre > callbacks which are able to change LDAP search filter, scope etc. > >>> >>>> 2) The part with "ind" is strange: >>>> >>>> + # enforce --sizelimit >>>> + if len(entries) == max_entries: >>>> + if ind + 1< len(results): >>>> + truncated = True >>>> + break >>>> >>>> I think it would be much easier to just do >>>> >>>> ... >>>> if (dn, permission) not in entries: >>>> if len(entries)< max_entries: >>>> entries.append((dn, permission)) >>>> else: >>>> truncated = True >>>> break >>>> >>>> Otherwise you would rise "truncated" even when the rest of "results" >>>> does not contain relevant entries that would have not been added anyway. >>> >>> Yes, that makes sense. >> >> And now updated patch. > > We can now also remove the "enumerate" part, "ind" is no longer needed. > > Martin You're right, I'd have sworn I tested that... The only solution is going to be to have the post_callback return truncated. This is going to be a rather intrusive change. rob From jcholast at redhat.com Wed May 23 09:16:27 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 23 May 2012 11:16:27 +0200 Subject: [Freeipa-devel] [PATCH] 79 SSH configuration fixes Message-ID: <4FBCAAEB.9000100@redhat.com> Hi, this fixes https://fedorahosted.org/freeipa/ticket/2769 as well as some other issues with SSH configuration in ipa-client-install. Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-79-ssh-configuration-fixes.patch Type: text/x-patch Size: 2013 bytes Desc: not available URL: From mkosek at redhat.com Wed May 23 12:24:27 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 23 May 2012 14:24:27 +0200 Subject: [Freeipa-devel] [PATCH] 260 Replace DNS client based on acutil with python-dns In-Reply-To: <4FBB898C.2030009@redhat.com> References: <1336755130.19268.12.camel@priserak> <4FB245F2.6000500@redhat.com> <1337154273.2963.8.camel@balmora.brq.redhat.com> <4FBB898C.2030009@redhat.com> Message-ID: <1337775867.25131.5.camel@balmora.brq.redhat.com> On Tue, 2012-05-22 at 14:41 +0200, Petr Viktorin wrote: > On 05/16/2012 09:44 AM, Martin Kosek wrote: > > On Tue, 2012-05-15 at 14:02 +0200, Petr Viktorin wrote: > >> > On 05/11/2012 06:52 PM, Martin Kosek wrote: > >>> > > python-dns is very feature-rich and it can help us a lot with our DNS > >>> > > related code. This patch does the first step, i.e. replaces acutil use > >>> > > with python-dns, which is more convenient to use as you will see in the > >>> > > patch. More integration will follow in the future. > >>> > > > >>> > > I send this patch rather early, so that I can get responses to this > >>> > > patch early and also so that we are able to catch issues in a safe > >>> > > distance from the next release. > >> > > >>> > > --- > >>> > > IPA client and server tool set used authconfig acutil module to > >>> > > for client DNS operations. This is not optimal DNS interface for > >>> > > several reasons: > >>> > > - does not provide native Python object oriented interface > >>> > > but but rather C-like interface based on functions and > >>> > > structures which is not easy to use and extend > >>> > > - acutil is not meant to be used by third parties besides > >>> > > authconfig and thus can break without notice > >>> > > > >>> > > Replace the acutil with python-dns package which has a feature rich > >>> > > interface for dealing with all different aspects of DNS including > >>> > > DNSSEC. The main target of this patch is to replace all uses of > >>> > > acutil DNS library with a use python-dns. In most cases, even > >>> > > though the larger parts of the code are changed, the actual > >>> > > functionality is changed only in the following cases: > >>> > > - redundant DNS checks were removed from verify_fqdn function > >>> > > in installutils to make the whole DNS check simpler and > >>> > > less error-prone. Logging was improves for the remaining > >>> > > checks > >>> > > - improved logging for ipa-client-install DNS discovery > >>> > > > >>> > > https://fedorahosted.org/freeipa/ticket/2730 > >> > > > [...] > > > Fixed. > > > > Martin > > I've been testing the patches in various setups and haven't found a > regression so far. ACK on 261, for 260 I have a comment below. > > > > diff --git a/ipa-client/ipaclient/ipadiscovery.py b/ipa-client/ipaclient/ipadiscovery.py > > index 86bef28b2d7fdfc8111b493bddec7ac6888f021a..1889d1918c01fe0c80a3d56b9a7ef304c5a7d97c 100644 > > --- a/ipa-client/ipaclient/ipadiscovery.py > > +++ b/ipa-client/ipaclient/ipadiscovery.py > > @@ -310,84 +313,74 @@ class IPADiscovery: > > os.rmdir(temp_ca_dir) > > > > > > - def ipadnssearchldap(self, tdomain): > > - servers = "" > > - rserver = "" > > + def ipadns_search_srv(self, domain, srv_record_name, default_port, > > + get_first=True): > > + """ > > + Search for SRV records in given domain. When no record is found, > > + en empty string is returned > > > > - qname = "_ldap._tcp."+tdomain > > - # terminate the name > > - if not qname.endswith("."): > > - qname += "." > > - results = ipapython.dnsclient.query(qname, ipapython.dnsclient.DNS_C_IN, ipapython.dnsclient.DNS_T_SRV) > > + :param domain: Search domain name > > + :param srv_record_name: SRV record name, e.g. "_ldap._tcp" > > + :param default_port: When default_port is not None, it is being > > + checked with the port in SRV record and if they don't > > + match, the port from SRV record is appended to > > + found hostname in this format: "hostname:port" > > + :param get_first: break on first find, otherwise multiple finds > > + separated by ":" may be returned > > They are separated by ",". > In the calling code, for splitting a comma-separated string it is better > to use servers.split(',') than ipautil.parse_items(servers). Or, return > a list directly from here. > I did not want to get too intrusive with the patch, but I took your advice and rather return now a list of servers - its more effective than to returning a comma-joined list and then splitting it back to standard list :-) That made parse_items function redundant. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-260-3-replace-dns-client-based-on-acutil-with-python-dns.patch Type: text/x-patch Size: 45632 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-261-3-fix-default_server-configuration-in-ipapython.config.patch Type: text/x-patch Size: 1007 bytes Desc: not available URL: From mkosek at redhat.com Wed May 23 12:30:57 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 23 May 2012 14:30:57 +0200 Subject: [Freeipa-devel] [PATCH] 260 Replace DNS client based on acutil with python-dns In-Reply-To: <1337775867.25131.5.camel@balmora.brq.redhat.com> References: <1336755130.19268.12.camel@priserak> <4FB245F2.6000500@redhat.com> <1337154273.2963.8.camel@balmora.brq.redhat.com> <4FBB898C.2030009@redhat.com> <1337775867.25131.5.camel@balmora.brq.redhat.com> Message-ID: <1337776257.25131.7.camel@balmora.brq.redhat.com> On Wed, 2012-05-23 at 14:24 +0200, Martin Kosek wrote: > On Tue, 2012-05-22 at 14:41 +0200, Petr Viktorin wrote: > > On 05/16/2012 09:44 AM, Martin Kosek wrote: > > > On Tue, 2012-05-15 at 14:02 +0200, Petr Viktorin wrote: > > >> > On 05/11/2012 06:52 PM, Martin Kosek wrote: > > >>> > > python-dns is very feature-rich and it can help us a lot with our DNS > > >>> > > related code. This patch does the first step, i.e. replaces acutil use > > >>> > > with python-dns, which is more convenient to use as you will see in the > > >>> > > patch. More integration will follow in the future. > > >>> > > > > >>> > > I send this patch rather early, so that I can get responses to this > > >>> > > patch early and also so that we are able to catch issues in a safe > > >>> > > distance from the next release. > > >> > > > >>> > > --- > > >>> > > IPA client and server tool set used authconfig acutil module to > > >>> > > for client DNS operations. This is not optimal DNS interface for > > >>> > > several reasons: > > >>> > > - does not provide native Python object oriented interface > > >>> > > but but rather C-like interface based on functions and > > >>> > > structures which is not easy to use and extend > > >>> > > - acutil is not meant to be used by third parties besides > > >>> > > authconfig and thus can break without notice > > >>> > > > > >>> > > Replace the acutil with python-dns package which has a feature rich > > >>> > > interface for dealing with all different aspects of DNS including > > >>> > > DNSSEC. The main target of this patch is to replace all uses of > > >>> > > acutil DNS library with a use python-dns. In most cases, even > > >>> > > though the larger parts of the code are changed, the actual > > >>> > > functionality is changed only in the following cases: > > >>> > > - redundant DNS checks were removed from verify_fqdn function > > >>> > > in installutils to make the whole DNS check simpler and > > >>> > > less error-prone. Logging was improves for the remaining > > >>> > > checks > > >>> > > - improved logging for ipa-client-install DNS discovery > > >>> > > > > >>> > > https://fedorahosted.org/freeipa/ticket/2730 > > >> > > > > > [...] > > > > > Fixed. > > > > > > Martin > > > > I've been testing the patches in various setups and haven't found a > > regression so far. ACK on 261, for 260 I have a comment below. > > > > > > > diff --git a/ipa-client/ipaclient/ipadiscovery.py b/ipa-client/ipaclient/ipadiscovery.py > > > index 86bef28b2d7fdfc8111b493bddec7ac6888f021a..1889d1918c01fe0c80a3d56b9a7ef304c5a7d97c 100644 > > > --- a/ipa-client/ipaclient/ipadiscovery.py > > > +++ b/ipa-client/ipaclient/ipadiscovery.py > > > @@ -310,84 +313,74 @@ class IPADiscovery: > > > os.rmdir(temp_ca_dir) > > > > > > > > > - def ipadnssearchldap(self, tdomain): > > > - servers = "" > > > - rserver = "" > > > + def ipadns_search_srv(self, domain, srv_record_name, default_port, > > > + get_first=True): > > > + """ > > > + Search for SRV records in given domain. When no record is found, > > > + en empty string is returned > > > > > > - qname = "_ldap._tcp."+tdomain > > > - # terminate the name > > > - if not qname.endswith("."): > > > - qname += "." > > > - results = ipapython.dnsclient.query(qname, ipapython.dnsclient.DNS_C_IN, ipapython.dnsclient.DNS_T_SRV) > > > + :param domain: Search domain name > > > + :param srv_record_name: SRV record name, e.g. "_ldap._tcp" > > > + :param default_port: When default_port is not None, it is being > > > + checked with the port in SRV record and if they don't > > > + match, the port from SRV record is appended to > > > + found hostname in this format: "hostname:port" > > > + :param get_first: break on first find, otherwise multiple finds > > > + separated by ":" may be returned > > > > They are separated by ",". > > In the calling code, for splitting a comma-separated string it is better > > to use servers.split(',') than ipautil.parse_items(servers). Or, return > > a list directly from here. > > > > I did not want to get too intrusive with the patch, but I took your > advice and rather return now a list of servers - its more effective than > to returning a comma-joined list and then splitting it back to standard > list :-) That made parse_items function redundant. > > Martin I forgot to include a change in the spec file - authconfig should be no longer needed for build. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-260-4-replace-dns-client-based-on-acutil-with-python-dns.patch Type: text/x-patch Size: 46786 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-261-4-fix-default_server-configuration-in-ipapython.config.patch Type: text/x-patch Size: 1007 bytes Desc: not available URL: From rcritten at redhat.com Wed May 23 17:55:15 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 23 May 2012 13:55:15 -0400 Subject: [Freeipa-devel] [PATCH] 1018 enforce sizelimit when searching for permissions In-Reply-To: <4FBBFAF3.2070106@redhat.com> References: <4FB55B69.5080506@redhat.com> <1337327599.25787.10.camel@balmora.brq.redhat.com> <4FB65045.5060706@redhat.com> <4FB65A0C.5030306@redhat.com> <1337692317.17468.8.camel@balmora.brq.redhat.com> <4FBBFAF3.2070106@redhat.com> Message-ID: <4FBD2483.7090101@redhat.com> Rob Crittenden wrote: > Martin Kosek wrote: >> On Fri, 2012-05-18 at 10:17 -0400, Rob Crittenden wrote: >>> Rob Crittenden wrote: >>>> Martin Kosek wrote: >>>>> On Thu, 2012-05-17 at 16:11 -0400, Rob Crittenden wrote: >>>>>> We do two searches when looking for permissions. One within the >>>>>> permission object itself and a second in the ACIs. We weren't >>>>>> enforcing >>>>>> a sizelimit on either search. >>>>>> >>>>>> rob >>>>> >>>>> This returns the right result, but I don't think it is right with >>>>> respect to "truncated" flag because of several reasons: >>>>> >>>>> 1) You manipulate and set "truncated" flag in post_callback but this >>>>> won't affect the flag in the returned result because the new value is >>>>> not propagated outside of the post_callback function. I.e. truncated >>>>> flag will be set correctly only when it was raised during original >>>>> permission_find. >>>> >>>> Truncated is still honored as expected. I even added a test case for >>>> this. >> >> Yes, but it only works when the truncated flag is raised by the base >> LDAP search, i.e. the search for permission objects (which is a case of >> your unit test). If the search does not raise the flag and it is set >> later in post callback, it is never propagated to the user. Please check >> the attached (crappy) test that shows this issue: >> >> ====================================================================== >> FAIL: test_permission[20]: permission_find: Search for permissions by >> attr with a limit of 1 (truncated) >> ---------------------------------------------------------------------- >> ... >> AssertionError: assert_deepequal: expected != got. >> test_permission[20]: permission_find: Search for permissions by attr >> with a limit of 1 (truncated) >> expected = True >> got = False >> path = ('truncated',) >> >> I am not sure how to solve this right, we may need to add a new return >> attribute (truncated) to all LDAPSearch post callbacks so that the post >> callback can really modify it - something similar we already do with pre >> callbacks which are able to change LDAP search filter, scope etc. >> >>>> >>>>> 2) The part with "ind" is strange: >>>>> >>>>> + # enforce --sizelimit >>>>> + if len(entries) == max_entries: >>>>> + if ind + 1< len(results): >>>>> + truncated = True >>>>> + break >>>>> >>>>> I think it would be much easier to just do >>>>> >>>>> ... >>>>> if (dn, permission) not in entries: >>>>> if len(entries)< max_entries: >>>>> entries.append((dn, permission)) >>>>> else: >>>>> truncated = True >>>>> break >>>>> >>>>> Otherwise you would rise "truncated" even when the rest of "results" >>>>> does not contain relevant entries that would have not been added >>>>> anyway. >>>> >>>> Yes, that makes sense. >>> >>> And now updated patch. >> >> We can now also remove the "enumerate" part, "ind" is no longer needed. >> >> Martin > > You're right, I'd have sworn I tested that... > > The only solution is going to be to have the post_callback return > truncated. This is going to be a rather intrusive change. > > rob Updated patch against master that adds a return value to post_callback. I also included Martin's test. It relies on the ordering of data in LDAP but it is better than nothing right now. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1018-3-sizelimit.patch Type: text/x-diff Size: 12516 bytes Desc: not available URL: From rcritten at redhat.com Wed May 23 18:03:25 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 23 May 2012 14:03:25 -0400 Subject: [Freeipa-devel] [PATCH] 1019 require policycoreutils if SELinux is enabled In-Reply-To: <4FBB9714.6020100@redhat.com> References: <4FB67069.1040901@redhat.com> <4FBB9714.6020100@redhat.com> Message-ID: <4FBD266D.6000607@redhat.com> Petr Viktorin wrote: > On 2012-05-18 17:53, Rob Crittenden wrote: >> We don't have an explicit requires on the policycoreutils package in the >> client because SELinux is not required (just recommended). >> >> SELinux can be enabled without this package so check for that condition >> and don't allow installation if it is the case. The resulting install >> will be rather broken. >> >> Also check on the server when installing. This should never happen but >> in theory it could do the server install then fail in the client because >> of this. >> >> rob >> > > All other platform-dependent services have a default defined in > ipapython/services.py.in. Shouldn't check_selinux_status have one, too? Yes, you're right, I forgot to add that. Patch updated. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1019-2-selinux.patch Type: text/x-diff Size: 10174 bytes Desc: not available URL: From rcritten at redhat.com Wed May 23 18:04:53 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 23 May 2012 14:04:53 -0400 Subject: [Freeipa-devel] [PATCH] 1020 replication conversion retry In-Reply-To: <1337647875.16840.174.camel@willson.li.ssimo.org> References: <4FBAA65E.1090707@redhat.com> <1337647875.16840.174.camel@willson.li.ssimo.org> Message-ID: <4FBD26C5.9020004@redhat.com> Simo Sorce wrote: > On Mon, 2012-05-21 at 16:32 -0400, Rob Crittenden wrote: >> When converting to GSSAPI replication we need to fetch the ldap >> principal from the other side. We've seen this fail from time to time >> despite having a call to force_sync. Add a retry loop to try harder, >> and >> fix the error reporting. >> >> I was never able to force reproduction of the underlying problem. I >> used >> the netem module to force a very slow link and wasn't able to >> duplicate >> the problem (I added a 500ms delay and a server with a couple of >> hundred >> entries). >> >> tc qdisc add dev p3p1 root netem delay 500ms >> > > ACK > pushed to master From hahaha_30k at yahoo.com Wed May 23 19:08:42 2012 From: hahaha_30k at yahoo.com (Gelen James) Date: Wed, 23 May 2012 12:08:42 -0700 (PDT) Subject: [Freeipa-devel] I've done it by myself and it works -- Re: Feature request: Web UI for IPA users to reset their own expired passwords In-Reply-To: <1337505737.89279.YahooMailNeo@web160703.mail.bf1.yahoo.com> References: <1337505737.89279.YahooMailNeo@web160703.mail.bf1.yahoo.com> Message-ID: <1337800122.16282.YahooMailNeo@web160706.mail.bf1.yahoo.com> I've coded it with python-kerberos and it works. Pretty rough though. --Gelen. ________________________________ From: Gelen James To: "freeipa-devel at redhat.com" Sent: Sunday, May 20, 2012 2:22 AM Subject: Feature request: Web UI for IPA users to reset their own expired passwords The currently assumption is that all IPA users can login into Unix/Linux machines to change their IPA password, or reset their expired password.? ?But this is not available all the time, so a more general alternative -- web UI -- will be more appreciated. The basic requirements are: ?1, The web UI accept user's passwords, expired is also accepted. ? ?2, the authentication is based on IPA Kerberos. ?3, authenticated regular IPA user can only reset his/her password only. ?4, (bonus) authenticated admin users can alter other users' password as well. Thanks. --Gelen -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcritten at redhat.com Wed May 23 19:14:10 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 23 May 2012 15:14:10 -0400 Subject: [Freeipa-devel] I've done it by myself and it works -- Re: Feature request: Web UI for IPA users to reset their own expired passwords In-Reply-To: <1337800122.16282.YahooMailNeo@web160706.mail.bf1.yahoo.com> References: <1337505737.89279.YahooMailNeo@web160703.mail.bf1.yahoo.com> <1337800122.16282.YahooMailNeo@web160706.mail.bf1.yahoo.com> Message-ID: <4FBD3702.9000702@redhat.com> Gelen James wrote: > I've coded it with python-kerberos and it works. Pretty rough though. Is this something you'd be interested in contributing? rob > > --Gelen. > > ------------------------------------------------------------------------ > *From:* Gelen James > *To:* "freeipa-devel at redhat.com" > *Sent:* Sunday, May 20, 2012 2:22 AM > *Subject:* Feature request: Web UI for IPA users to reset their own > expired passwords > > The currently assumption is that all IPA users can login into Unix/Linux > machines to change their IPA password, or reset their expired password. > > But this is not available all the time, so a more general alternative -- > web UI -- will be more appreciated. The basic requirements are: > > 1, The web UI accept user's passwords, expired is also accepted. > 2, the authentication is based on IPA Kerberos. > > 3, authenticated regular IPA user can only reset his/her password only. > > 4, (bonus) authenticated admin users can alter other users' password as > well. > > > Thanks. > > --Gelen > > > > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From hahaha_30k at yahoo.com Wed May 23 19:43:34 2012 From: hahaha_30k at yahoo.com (Gelen James) Date: Wed, 23 May 2012 12:43:34 -0700 (PDT) Subject: [Freeipa-devel] I've done it by myself and it works -- Re: Feature request: Web UI for IPA users to reset their own expired passwords In-Reply-To: <4FBD3702.9000702@redhat.com> References: <1337505737.89279.YahooMailNeo@web160703.mail.bf1.yahoo.com> <1337800122.16282.YahooMailNeo@web160706.mail.bf1.yahoo.com> <4FBD3702.9000702@redhat.com> Message-ID: <1337802214.62693.YahooMailNeo@web160703.mail.bf1.yahoo.com> No problem. The code is attached. It is just one python script, with configuration items on the top. ?Please be reminded that this code is pretty rough and not well-tested as I can not find appropriate documents on how to use python kerberos module. ?Disclaim: This piece of code just works as a prototype, it is not well-tested, nor DOS attack prove at all, so it could potentially harm or totally destroy someone's authentication system. :( Thanks. --Gelen ________________________________ From: Rob Crittenden To: Gelen James Cc: "freeipa-devel at redhat.com" ; "freeipa-users at redhat.com" Sent: Wednesday, May 23, 2012 12:14 PM Subject: Re: [Freeipa-devel] I've done it by myself and it works -- Re: Feature request: Web UI for IPA users to reset their own expired passwords Gelen James wrote: > I've coded it with python-kerberos and it works. Pretty rough though. Is this something you'd be interested in contributing? rob > > --Gelen. > > ------------------------------------------------------------------------ > *From:* Gelen James > *To:* "freeipa-devel at redhat.com" > *Sent:* Sunday, May 20, 2012 2:22 AM > *Subject:* Feature request: Web UI for IPA users to reset their own > expired passwords > > The currently assumption is that all IPA users can login into Unix/Linux > machines to change their IPA password, or reset their expired password. > > But this is not available all the time, so a more general alternative -- > web UI -- will be more appreciated. The basic requirements are: > > 1, The web UI accept user's passwords, expired is also accepted. > 2, the authentication is based on IPA Kerberos. > > 3, authenticated regular IPA user can only reset his/her password only. > > 4, (bonus) authenticated admin users can alter other users' password as > well. > > > Thanks. > > --Gelen > > > > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: kchange.py Type: application/octet-stream Size: 8598 bytes Desc: not available URL: From pviktori at redhat.com Thu May 24 08:33:40 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 24 May 2012 10:33:40 +0200 Subject: [Freeipa-devel] [PATCH] 1019 require policycoreutils if SELinux is enabled In-Reply-To: <4FBD266D.6000607@redhat.com> References: <4FB67069.1040901@redhat.com> <4FBB9714.6020100@redhat.com> <4FBD266D.6000607@redhat.com> Message-ID: <4FBDF264.1030202@redhat.com> On 05/23/2012 08:03 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> On 2012-05-18 17:53, Rob Crittenden wrote: >>> We don't have an explicit requires on the policycoreutils package in the >>> client because SELinux is not required (just recommended). >>> >>> SELinux can be enabled without this package so check for that condition >>> and don't allow installation if it is the case. The resulting install >>> will be rather broken. >>> >>> Also check on the server when installing. This should never happen but >>> in theory it could do the server install then fail in the client because >>> of this. >>> >>> rob >>> >> >> All other platform-dependent services have a default defined in >> ipapython/services.py.in. Shouldn't check_selinux_status have one, too? > > Yes, you're right, I forgot to add that. > > Patch updated. > > rob Unfortunately, the patch doesn't apply. I also found a comment typo in ipapython/platform/redhat.py & ipapython/platform/fedora.py: > +# Everything else is made available through these symbols when they directly > +# imported into ipapython.services: s/they directly/they are directly/ -- Petr? From pvoborni at redhat.com Thu May 24 08:54:09 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 24 May 2012 10:54:09 +0200 Subject: [Freeipa-devel] [PATCHES] 138-145 Action panel for user password reset Message-ID: <4FBDF731.5090805@redhat.com> This bunch of patches implements new concept: action panel and it's implementation in user page. First two patches refactorizes current action-list/control-buttons code to prepare ground for following patches. Sorry for added review work (could be done this way earlier). Patch descriptions: [PATCH] 138 Refactored action list and control buttons to use shared list of actions This is a first step for implementing action panels which will also use the shared list of actions. This effort changes the way how action list and control buttons are defined. First all actions are defined on facet level - attribute 'actions' in spec file. Implementation of action list widget is not specified on facet level. It is left in facet header. A list of action names used in action list can be now specified in facet spec in 'header_actions' attribute. Control buttons use similar concept. Facet by default is using control_buttons_widget. Details and search facet are defining their own default actions (refresh/add/remove/update/reset). Additional buttons can be defined as array of action names on facet level in control_buttons attribute. state_evaluators and state_listeners were united. They are called state_evaluators but they uses state_listener concept, they are attached to an event. For former state_evaluator the event is post_load. They are defined in spec in state attribute. State object purpose is to aggregate states from all state evaluators. It offers changed event to which can other objects subscribe. It also has summary evaluator which evaluation conditions. Summary evaluator creates summary status with human readable description. It can be used by facet header. https://fedorahosted.org/freeipa/ticket/2248 [PATCH] 139 Refactored entities to use changed actions concept It's continuation of previous refactoring effort. This part is changing specs in entities to used changed concept. https://fedorahosted.org/freeipa/ticket/2248 [PATCH] 140 Action panel This patch implements action panel. Action panel is a box located in facet details section which contains actions related to that object/section. In spec file can be configured actions and title used in action panel. Default title is 'Actions'. Actions are specified by their name. They have to be defined in action collection in facet. https://fedorahosted.org/freeipa/ticket/2248 [PATCH] 141 User password widget modified. Currently the user password is shown as follows in the details page: Password: Reset Password This is inconsistent with the rest of the page because the 'Reset Password' is an action, not the value of the password. Now password is shown as follows: Password: ******* (if set) Password: (if not set) Reset password link was removed as well the dialog for reset password was removed from password widget. The dialog was moved to its own object and can be now showed independently. An action for showing this dialog should be created. https://fedorahosted.org/freeipa/ticket/2248 [PATCH] 142 Action panel for user This patch adds action panel to user account section. The panel contain an action for reseting user password. https://fedorahosted.org/freeipa/ticket/2248 [PATCH] 143 Added missing i18n in action list and action panel This patch adds strings to internal.py which were not translated in action list/panel patches. https://fedorahosted.org/freeipa/ticket/2248 [PATCH] 144 Add shadow to dialog This patch adds shadow to dialog used in Web UI. It looks cooler. https://fedorahosted.org/freeipa/ticket/2248 note: I didn't want to create new ticket just for this minor visual enhancement. [PATCH] 145 Enable reset password action according to attribute perrmission This patch creates state_evaluator which creates permission states for defined attribute. The state format is: attributeName_permissionChar. This evaluator is used for user_password attribute and it control enabling/disabling of related action in user account action panel. https://fedorahosted.org/freeipa/ticket/2318 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0138-Refactored-action-list-and-control-buttons-to-use-sh.patch Type: text/x-patch Size: 39528 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0139-Refactored-entities-to-use-changed-actions-concept.patch Type: text/x-patch Size: 19214 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0140-Action-panel.patch Type: text/x-patch Size: 7539 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0141-User-password-widget-modified.patch Type: text/x-patch Size: 10346 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0142-Action-panel-for-user.patch Type: text/x-patch Size: 2213 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0143-Added-missing-i18n-in-action-list-and-action-panel.patch Type: text/x-patch Size: 3872 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0144-Add-shadow-to-dialog.patch Type: text/x-patch Size: 946 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0145-Enable-reset-password-action-according-to-attribute-.patch Type: text/x-patch Size: 3310 bytes Desc: not available URL: From pvoborni at redhat.com Thu May 24 09:05:01 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 24 May 2012 11:05:01 +0200 Subject: [Freeipa-devel] [PATCH] 133 Action list for user password In-Reply-To: <4FB13D09.7010505@redhat.com> References: <4FA248D9.50406@redhat.com> <4FB13D09.7010505@redhat.com> Message-ID: <4FBDF9BD.4060106@redhat.com> Just for record: this patch is replaced by my patches 141, 142 (sent to list on May 24). On 05/14/2012 07:12 PM, Endi Sukma Dewata wrote: > On 5/3/2012 3:59 AM, Petr Vobornik wrote: >> Currently the user password is shown as follows in the details page: >> Password: Reset Password >> >> This is inconsistent with the rest of the page because the 'Reset >> Password' is an action, not the value of the password. >> >> Now password is shown as follows: >> Password: ******* (if set) >> Password: (if not set) >> >> Reset password action was moved to Action List. >> >> https://fedorahosted.org/freeipa/ticket/2248 >> >> Note: I will address enabling/disabling 'reset password' option in >> action list in ticket #2318. > > Just for the record: > > On 5/11/2012 11:36 AM, Petr Vobornik wrote: > > On 05/08/2012 01:47 AM, Endi Sukma Dewata wrote: > >> 10. The 'Action List' in ticket #2248 for the password reset is > actually > >> different than the action list for Enable/Disable. I was proposing to > >> create a small panel under the 'Account Settings' section, probably > >> something like this: > >> > >> --------------------------------------------------------------- > >> Account Settings > >> +-----------------+ > >> User login: admin | Actions: | > >> Password: | Reset password | > >> Password expiration: +-----------------+ > >> UID: > >> GID: > >> > >> This panel might better be called 'Action Panel' to distinguish from > the > >> dropdown 'Action List' on the top. The reason for this panel is that a > >> Password Reset only affects an aspect of the user, not the entire > object > >> like Enable/Disable (although that can also be argued differently), so > >> the action should be placed where it's relevant, in this case near the > >> Password field. The same concept will be used for ticket #2250, #2251, > >> #2252. > >> > > > > I'll consider my patch #133 NACKed and first create the action panels. > -- Petr Vobornik From pvoborni at redhat.com Thu May 24 09:11:31 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 24 May 2012 11:11:31 +0200 Subject: [Freeipa-devel] [PATCH] 146 Added cancel button to service unprovision dialog Message-ID: <4FBDFB43.7050308@redhat.com> Service unprovision dialog was missing a cancel button. The button was added. https://fedorahosted.org/freeipa/ticket/1811 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0146-Added-cancel-button-to-service-unprovision-dialog.patch Type: text/x-patch Size: 1040 bytes Desc: not available URL: From pviktori at redhat.com Thu May 24 11:53:16 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 24 May 2012 13:53:16 +0200 Subject: [Freeipa-devel] [PATCH] 260 Replace DNS client based on acutil with python-dns In-Reply-To: <1337776257.25131.7.camel@balmora.brq.redhat.com> References: <1336755130.19268.12.camel@priserak> <4FB245F2.6000500@redhat.com> <1337154273.2963.8.camel@balmora.brq.redhat.com> <4FBB898C.2030009@redhat.com> <1337775867.25131.5.camel@balmora.brq.redhat.com> <1337776257.25131.7.camel@balmora.brq.redhat.com> Message-ID: <4FBE212C.2050707@redhat.com> On 05/23/2012 02:30 PM, Martin Kosek wrote: > On Wed, 2012-05-23 at 14:24 +0200, Martin Kosek wrote: >> On Tue, 2012-05-22 at 14:41 +0200, Petr Viktorin wrote: >>> On 05/16/2012 09:44 AM, Martin Kosek wrote: >>>> On Tue, 2012-05-15 at 14:02 +0200, Petr Viktorin wrote: >>>>>> On 05/11/2012 06:52 PM, Martin Kosek wrote: >>>>>>> > python-dns is very feature-rich and it can help us a lot with our DNS >>>>>>> > related code. This patch does the first step, i.e. replaces acutil use >>>>>>> > with python-dns, which is more convenient to use as you will see in the >>>>>>> > patch. More integration will follow in the future. >>>>>>> > >>>>>>> > I send this patch rather early, so that I can get responses to this >>>>>>> > patch early and also so that we are able to catch issues in a safe >>>>>>> > distance from the next release. >>>>>> >>>>>>> > --- >>>>>>> > IPA client and server tool set used authconfig acutil module to >>>>>>> > for client DNS operations. This is not optimal DNS interface for >>>>>>> > several reasons: >>>>>>> > - does not provide native Python object oriented interface >>>>>>> > but but rather C-like interface based on functions and >>>>>>> > structures which is not easy to use and extend >>>>>>> > - acutil is not meant to be used by third parties besides >>>>>>> > authconfig and thus can break without notice >>>>>>> > >>>>>>> > Replace the acutil with python-dns package which has a feature rich >>>>>>> > interface for dealing with all different aspects of DNS including >>>>>>> > DNSSEC. The main target of this patch is to replace all uses of >>>>>>> > acutil DNS library with a use python-dns. In most cases, even >>>>>>> > though the larger parts of the code are changed, the actual >>>>>>> > functionality is changed only in the following cases: >>>>>>> > - redundant DNS checks were removed from verify_fqdn function >>>>>>> > in installutils to make the whole DNS check simpler and >>>>>>> > less error-prone. Logging was improves for the remaining >>>>>>> > checks >>>>>>> > - improved logging for ipa-client-install DNS discovery >>>>>>> > >>>>>>> > https://fedorahosted.org/freeipa/ticket/2730 >>>>>> >>> >>> [...] >>> >>>> Fixed. >>>> >>>> Martin >>> >>> I've been testing the patches in various setups and haven't found a >>> regression so far. ACK on 261, for 260 I have a comment below. >>> >>> >>>> diff --git a/ipa-client/ipaclient/ipadiscovery.py b/ipa-client/ipaclient/ipadiscovery.py >>>> index 86bef28b2d7fdfc8111b493bddec7ac6888f021a..1889d1918c01fe0c80a3d56b9a7ef304c5a7d97c 100644 >>>> --- a/ipa-client/ipaclient/ipadiscovery.py >>>> +++ b/ipa-client/ipaclient/ipadiscovery.py >>>> @@ -310,84 +313,74 @@ class IPADiscovery: >>>> os.rmdir(temp_ca_dir) >>>> >>>> >>>> - def ipadnssearchldap(self, tdomain): >>>> - servers = "" >>>> - rserver = "" >>>> + def ipadns_search_srv(self, domain, srv_record_name, default_port, >>>> + get_first=True): >>>> + """ >>>> + Search for SRV records in given domain. When no record is found, >>>> + en empty string is returned >>>> >>>> - qname = "_ldap._tcp."+tdomain >>>> - # terminate the name >>>> - if not qname.endswith("."): >>>> - qname += "." >>>> - results = ipapython.dnsclient.query(qname, ipapython.dnsclient.DNS_C_IN, ipapython.dnsclient.DNS_T_SRV) >>>> + :param domain: Search domain name >>>> + :param srv_record_name: SRV record name, e.g. "_ldap._tcp" >>>> + :param default_port: When default_port is not None, it is being >>>> + checked with the port in SRV record and if they don't >>>> + match, the port from SRV record is appended to >>>> + found hostname in this format: "hostname:port" >>>> + :param get_first: break on first find, otherwise multiple finds >>>> + separated by ":" may be returned >>> >>> They are separated by ",". >>> In the calling code, for splitting a comma-separated string it is better >>> to use servers.split(',') than ipautil.parse_items(servers). Or, return >>> a list directly from here. >>> >> >> I did not want to get too intrusive with the patch, but I took your >> advice and rather return now a list of servers - its more effective than >> to returning a comma-joined list and then splitting it back to standard >> list :-) That made parse_items function redundant. >> Good riddance. >> Martin > > I forgot to include a change in the spec file - authconfig should be no > longer needed for build. > > Martin I tested several installs and couldn't find a regression. ACK. -- Petr? From mkosek at redhat.com Thu May 24 11:59:43 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 24 May 2012 13:59:43 +0200 Subject: [Freeipa-devel] [PATCH] 260 Replace DNS client based on acutil with python-dns In-Reply-To: <4FBE212C.2050707@redhat.com> References: <1336755130.19268.12.camel@priserak> <4FB245F2.6000500@redhat.com> <1337154273.2963.8.camel@balmora.brq.redhat.com> <4FBB898C.2030009@redhat.com> <1337775867.25131.5.camel@balmora.brq.redhat.com> <1337776257.25131.7.camel@balmora.brq.redhat.com> <4FBE212C.2050707@redhat.com> Message-ID: <1337860783.23463.0.camel@balmora.brq.redhat.com> On Thu, 2012-05-24 at 13:53 +0200, Petr Viktorin wrote: > On 05/23/2012 02:30 PM, Martin Kosek wrote: > > On Wed, 2012-05-23 at 14:24 +0200, Martin Kosek wrote: > >> On Tue, 2012-05-22 at 14:41 +0200, Petr Viktorin wrote: > >>> On 05/16/2012 09:44 AM, Martin Kosek wrote: > >>>> On Tue, 2012-05-15 at 14:02 +0200, Petr Viktorin wrote: > >>>>>> On 05/11/2012 06:52 PM, Martin Kosek wrote: > >>>>>>> > python-dns is very feature-rich and it can help us a lot with our DNS > >>>>>>> > related code. This patch does the first step, i.e. replaces acutil use > >>>>>>> > with python-dns, which is more convenient to use as you will see in the > >>>>>>> > patch. More integration will follow in the future. > >>>>>>> > > >>>>>>> > I send this patch rather early, so that I can get responses to this > >>>>>>> > patch early and also so that we are able to catch issues in a safe > >>>>>>> > distance from the next release. > >>>>>> > >>>>>>> > --- > >>>>>>> > IPA client and server tool set used authconfig acutil module to > >>>>>>> > for client DNS operations. This is not optimal DNS interface for > >>>>>>> > several reasons: > >>>>>>> > - does not provide native Python object oriented interface > >>>>>>> > but but rather C-like interface based on functions and > >>>>>>> > structures which is not easy to use and extend > >>>>>>> > - acutil is not meant to be used by third parties besides > >>>>>>> > authconfig and thus can break without notice > >>>>>>> > > >>>>>>> > Replace the acutil with python-dns package which has a feature rich > >>>>>>> > interface for dealing with all different aspects of DNS including > >>>>>>> > DNSSEC. The main target of this patch is to replace all uses of > >>>>>>> > acutil DNS library with a use python-dns. In most cases, even > >>>>>>> > though the larger parts of the code are changed, the actual > >>>>>>> > functionality is changed only in the following cases: > >>>>>>> > - redundant DNS checks were removed from verify_fqdn function > >>>>>>> > in installutils to make the whole DNS check simpler and > >>>>>>> > less error-prone. Logging was improves for the remaining > >>>>>>> > checks > >>>>>>> > - improved logging for ipa-client-install DNS discovery > >>>>>>> > > >>>>>>> > https://fedorahosted.org/freeipa/ticket/2730 > >>>>>> > >>> > >>> [...] > >>> > >>>> Fixed. > >>>> > >>>> Martin > >>> > >>> I've been testing the patches in various setups and haven't found a > >>> regression so far. ACK on 261, for 260 I have a comment below. > >>> > >>> > >>>> diff --git a/ipa-client/ipaclient/ipadiscovery.py b/ipa-client/ipaclient/ipadiscovery.py > >>>> index 86bef28b2d7fdfc8111b493bddec7ac6888f021a..1889d1918c01fe0c80a3d56b9a7ef304c5a7d97c 100644 > >>>> --- a/ipa-client/ipaclient/ipadiscovery.py > >>>> +++ b/ipa-client/ipaclient/ipadiscovery.py > >>>> @@ -310,84 +313,74 @@ class IPADiscovery: > >>>> os.rmdir(temp_ca_dir) > >>>> > >>>> > >>>> - def ipadnssearchldap(self, tdomain): > >>>> - servers = "" > >>>> - rserver = "" > >>>> + def ipadns_search_srv(self, domain, srv_record_name, default_port, > >>>> + get_first=True): > >>>> + """ > >>>> + Search for SRV records in given domain. When no record is found, > >>>> + en empty string is returned > >>>> > >>>> - qname = "_ldap._tcp."+tdomain > >>>> - # terminate the name > >>>> - if not qname.endswith("."): > >>>> - qname += "." > >>>> - results = ipapython.dnsclient.query(qname, ipapython.dnsclient.DNS_C_IN, ipapython.dnsclient.DNS_T_SRV) > >>>> + :param domain: Search domain name > >>>> + :param srv_record_name: SRV record name, e.g. "_ldap._tcp" > >>>> + :param default_port: When default_port is not None, it is being > >>>> + checked with the port in SRV record and if they don't > >>>> + match, the port from SRV record is appended to > >>>> + found hostname in this format: "hostname:port" > >>>> + :param get_first: break on first find, otherwise multiple finds > >>>> + separated by ":" may be returned > >>> > >>> They are separated by ",". > >>> In the calling code, for splitting a comma-separated string it is better > >>> to use servers.split(',') than ipautil.parse_items(servers). Or, return > >>> a list directly from here. > >>> > >> > >> I did not want to get too intrusive with the patch, but I took your > >> advice and rather return now a list of servers - its more effective than > >> to returning a comma-joined list and then splitting it back to standard > >> list :-) That made parse_items function redundant. > >> > Good riddance. > >> Martin > > > > I forgot to include a change in the spec file - authconfig should be no > > longer needed for build. > > > > Martin > > I tested several installs and couldn't find a regression. ACK. > > Thanks, pushed both to master. Martin From pviktori at redhat.com Thu May 24 12:15:16 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 24 May 2012 14:15:16 +0200 Subject: [Freeipa-devel] [PATCH] 1014 configurable service timeout In-Reply-To: <4FB6AAF6.9080800@redhat.com> References: <4FB3F267.2070104@redhat.com> <4FB6AAF6.9080800@redhat.com> Message-ID: <4FBE2654.4030207@redhat.com> On 05/18/2012 10:03 PM, Rob Crittenden wrote: > Rob Crittenden wrote: >> A hardcoded timeout was used in ipactl for service restarts, set rather >> low. A separate timeout was hardcoded into the installer. >> >> I centralized them into a single timeout, configurable in the standard >> way in /etc/ipa/*.conf. >> >> On install it will always default to 120 seconds and remain there unless >> changed in default.conf (not replicated either). >> >> I tested this on systemd systems and sysV systems and it works ok for >> me. You'll also want to double-check that this works when other 389-ds >> instances are installed. >> >> Getting the naming of instances right was a bit tricky. > > Noticed a problem on upgrades and fixed that. Updated patch attached. > > rob > > Please rebase the patch onto current master. -- Petr? From rcritten at redhat.com Thu May 24 15:38:54 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 24 May 2012 11:38:54 -0400 Subject: [Freeipa-devel] [PATCH] 1014 configurable service timeout In-Reply-To: <4FBE2654.4030207@redhat.com> References: <4FB3F267.2070104@redhat.com> <4FB6AAF6.9080800@redhat.com> <4FBE2654.4030207@redhat.com> Message-ID: <4FBE560E.1080108@redhat.com> Petr Viktorin wrote: > On 05/18/2012 10:03 PM, Rob Crittenden wrote: >> Rob Crittenden wrote: >>> A hardcoded timeout was used in ipactl for service restarts, set rather >>> low. A separate timeout was hardcoded into the installer. >>> >>> I centralized them into a single timeout, configurable in the standard >>> way in /etc/ipa/*.conf. >>> >>> On install it will always default to 120 seconds and remain there unless >>> changed in default.conf (not replicated either). >>> >>> I tested this on systemd systems and sysV systems and it works ok for >>> me. You'll also want to double-check that this works when other 389-ds >>> instances are installed. >>> >>> Getting the naming of instances right was a bit tricky. >> >> Noticed a problem on upgrades and fixed that. Updated patch attached. >> >> rob >> >> > > Please rebase the patch onto current master. > > Done -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1014-3-timeout.patch Type: text/x-diff Size: 29005 bytes Desc: not available URL: From rcritten at redhat.com Thu May 24 15:39:15 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 24 May 2012 11:39:15 -0400 Subject: [Freeipa-devel] [PATCH] 1019 require policycoreutils if SELinux is enabled In-Reply-To: <4FBDF264.1030202@redhat.com> References: <4FB67069.1040901@redhat.com> <4FBB9714.6020100@redhat.com> <4FBD266D.6000607@redhat.com> <4FBDF264.1030202@redhat.com> Message-ID: <4FBE5623.6090109@redhat.com> Petr Viktorin wrote: > On 05/23/2012 08:03 PM, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> On 2012-05-18 17:53, Rob Crittenden wrote: >>>> We don't have an explicit requires on the policycoreutils package in >>>> the >>>> client because SELinux is not required (just recommended). >>>> >>>> SELinux can be enabled without this package so check for that condition >>>> and don't allow installation if it is the case. The resulting install >>>> will be rather broken. >>>> >>>> Also check on the server when installing. This should never happen but >>>> in theory it could do the server install then fail in the client >>>> because >>>> of this. >>>> >>>> rob >>>> >>> >>> All other platform-dependent services have a default defined in >>> ipapython/services.py.in. Shouldn't check_selinux_status have one, too? >> >> Yes, you're right, I forgot to add that. >> >> Patch updated. >> >> rob > > Unfortunately, the patch doesn't apply. > > > I also found a comment typo in ipapython/platform/redhat.py & > ipapython/platform/fedora.py: > >> +# Everything else is made available through these symbols when they >> directly >> +# imported into ipapython.services: > > s/they directly/they are directly/ > Fixed and rebased. This requires patch 1014 to apply. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1019-3-selinux.patch Type: text/x-diff Size: 10182 bytes Desc: not available URL: From jdennis at redhat.com Thu May 24 16:07:45 2012 From: jdennis at redhat.com (John Dennis) Date: Thu, 24 May 2012 12:07:45 -0400 Subject: [Freeipa-devel] dns.py question Message-ID: <4FBE5CD1.4020500@redhat.com> In dns_is_enabled it computes the base_dn but never uses it. Instead it uses self.base_dn, but as far as I can figure out there is no self.base_dn in the class hierarchy (if so could you point me at it). The access to self.base_dn is wrapped in a try/except block that doesn't handle any errors so if in fact there was no self.base_dn that error would be silently ignored. So is self.base_dn a typo and we're supposed to be using the local base_dn that's computed but never used? Code snipped attached so it won't be mangled by line wrapping. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- class dns_is_enabled(Command): """ Checks if any of the servers has the DNS service enabled. """ NO_CLI = True has_output = output.standard_value base_dn = 'cn=masters,cn=ipa,cn=etc,%s' % api.env.basedn filter = '(&(objectClass=ipaConfigObject)(cn=DNS))' def execute(self, *args, **options): ldap = self.api.Backend.ldap2 dns_enabled = False try: ent = ldap.find_entries(filter=self.filter, base_dn=self.base_dn) if len(ent): dns_enabled = True except Exception, e: pass return dict(result=dns_enabled, value=u'') From pspacek at redhat.com Thu May 24 17:07:21 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 24 May 2012 19:07:21 +0200 Subject: [Freeipa-devel] routing requests to local servers - DNS SRV [discussion needed] Message-ID: <4FBE6AC9.4020703@redhat.com> Hello, some time ago there was a request for DNS to support "routing requests to local servers". Any opinions if/when do it and proposals how to do it are more than welcome. My best knowledge about this problem follows: This request actually means "differentiate answer to DNS query on client's IP address basics". Relevant thread on ipa-users-list: https://www.redhat.com/archives/freeipa-users/2012-April/msg00070.html First question is: Do we want to implement it? (IMHO it is very important for large-scale deployments.) It is related to IPA ticket #122 at architecture level: IPA DNS integration should support GeoIP https://fedorahosted.org/freeipa/ticket/122 IMHO it is better to support generic "locations" defined by user than real Geo-IP: - at least private IP addresses will be problematic - real Geo-IP database is huge and can slow all things down Bind-dyndb-ldap plugin supports BIND "views" already. It should be possible to define multiple plugin instances with different ldap search base and let it to serve records. The main problem problem is managing DNS subtree in LDAP. With current implementation you have to copy whole subtree to another place and then change necessary records. Problem is with maintaining "base" (non-overridden) records - you have to do same change n-times. Adam and I discussed possible approaches. We agreed on "stackable" approach: - Store whole original DNS tree in one subtree, let say "base". - Create "override" subtrees for each "locality" = set of customized records. Shared records are only in "base". During DNS query processing "override" subtree is searched first. If record does not exist in "override" subtree, search will continue in "base" subtree. Second question is how to realize these "overrides": - Let Directory Server to do the work. If DS supports this, it is mostly done. It do not require any change in bind-dyndb-ldap code. All merges/overrides will be done on Directory server. Only /etc/named.conf has to be adjusted with right search base and "view" clause. I asked on 389 mailing list and I'm waiting for reply: http://lists.fedoraproject.org/pipermail/389-users/2012-May/014653.html - Another approach is to add support directly to bind-dyndb-ldap, but it is not so nice solution. In that case both LDAP search bases have to be defined in /etc/named.conf. For each DNS query is necessary to search "override" base first. If required record is not present then is necessary to search through "base" subtree. With persistent search it should be better, because all records are in memory, but basic principle remains same. Thanks for all opinions. Petr^2 Spacek From rcritten at redhat.com Thu May 24 17:54:56 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 24 May 2012 13:54:56 -0400 Subject: [Freeipa-devel] [PATCH] 1022 normalize uid when in winsync Message-ID: <4FBE75F0.3050400@redhat.com> In case the uid that comes from AD is mixed-case we need to normalize it to all lower. It should be safe using tolower() because we only allow ASCII characters in uid. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1022-winsync.patch Type: text/x-diff Size: 2584 bytes Desc: not available URL: From dpal at redhat.com Thu May 24 18:00:03 2012 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 24 May 2012 14:00:03 -0400 Subject: [Freeipa-devel] routing requests to local servers - DNS SRV [discussion needed] In-Reply-To: <4FBE6AC9.4020703@redhat.com> References: <4FBE6AC9.4020703@redhat.com> Message-ID: <4FBE7723.2060206@redhat.com> On 05/24/2012 01:07 PM, Petr Spacek wrote: > Hello, > > some time ago there was a request for DNS to support "routing requests > to local servers". Any opinions if/when do it and proposals how to do > it are more than welcome. My best knowledge about this problem follows: > > > This request actually means "differentiate answer to DNS query on > client's IP address basics". > Relevant thread on ipa-users-list: > https://www.redhat.com/archives/freeipa-users/2012-April/msg00070.html > > First question is: Do we want to implement it? > (IMHO it is very important for large-scale deployments.) > > > It is related to IPA ticket #122 at architecture level: > IPA DNS integration should support GeoIP > https://fedorahosted.org/freeipa/ticket/122 > IMHO it is better to support generic "locations" defined by user than > real Geo-IP: > - at least private IP addresses will be problematic > - real Geo-IP database is huge and can slow all things down > > > Bind-dyndb-ldap plugin supports BIND "views" already. It should be > possible to define multiple plugin instances with different ldap > search base and let it to serve records. > > The main problem problem is managing DNS subtree in LDAP. With current > implementation you have to copy whole subtree to another place and > then change necessary records. Problem is with maintaining "base" > (non-overridden) records - you have to do same change n-times. > > > Adam and I discussed possible approaches. We agreed on "stackable" > approach: > - Store whole original DNS tree in one subtree, let say "base". > - Create "override" subtrees for each "locality" = set of customized > records. Shared records are only in "base". During DNS query > processing "override" subtree is searched first. If record does not > exist in "override" subtree, search will continue in "base" subtree. > > > Second question is how to realize these "overrides": > - Let Directory Server to do the work. If DS supports this, it is > mostly done. > It do not require any change in bind-dyndb-ldap code. All > merges/overrides will be done on Directory server. Only > /etc/named.conf has to be adjusted with right search base and "view" > clause. > > I asked on 389 mailing list and I'm waiting for reply: > http://lists.fedoraproject.org/pipermail/389-users/2012-May/014653.html > > > - Another approach is to add support directly to bind-dyndb-ldap, but > it is not so nice solution. In that case both LDAP search bases have > to be defined in /etc/named.conf. For each DNS query is necessary to > search "override" base first. If required record is not present then > is necessary to search through "base" subtree. > With persistent search it should be better, because all records are in > memory, but basic principle remains same. > > > Thanks for all opinions. May be I am oversimplifying things but it seems that it would make sense to just have an extra multi-value attribute added to the DNS schema. This attribute would contain a value that would allow the entry to be included into the view. This way the base is the same and what the view exposes is just a filter. The default view would have a filter that matches all entries like now. I assume that DNS server makes it decision based on the IP so it has a call to get a "view" data. The ldap driver can return a filter. The DNS server them makes a call using this filter to get all the records that apply. I know it is too ldap centric. There is some abstraction in DNS server and we do not want to change it. But the point is there are probably two calls in the DNS server (have not actually confirmed): lookup data X by IP to understand what the view is. Pass data X to get the actual DNS entries. I am saying that X can be filter. Thoughts? > > Petr^2 Spacek > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From mkosek at redhat.com Fri May 25 06:41:14 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 25 May 2012 08:41:14 +0200 Subject: [Freeipa-devel] dns.py question In-Reply-To: <4FBE5CD1.4020500@redhat.com> References: <4FBE5CD1.4020500@redhat.com> Message-ID: <1337928074.5899.12.camel@balmora.brq.redhat.com> On Thu, 2012-05-24 at 12:07 -0400, John Dennis wrote: > In dns_is_enabled it computes the base_dn but never uses it. > Instead it uses self.base_dn, but as far as I can figure out there is no > self.base_dn in the class hierarchy (if so could you point me at it). > The access to self.base_dn is wrapped in a try/except block that doesn't > handle any errors so if in fact there was no self.base_dn that error > would be silently ignored. > > So is self.base_dn a typo and we're supposed to be using the local > base_dn that's computed but never used? > > Code snipped attached so it won't be mangled by line wrapping. > Hello John, base_dn is defined in dns_is_enabled command class along with filter: class dns_is_enabled(Command): ... base_dn = 'cn=masters,cn=ipa,cn=etc,%s' % api.env.basedn filter = '(&(objectClass=ipaConfigObject)(cn=DNS))' ... def execute(self, *args, **options): ... try: ent = ldap.find_entries(filter=self.filter, base_dn=self.base_dn) if len(ent): dns_enabled = True except Exception, e: pass So self.base_dn and self.filter return correct value. Otherwise the function would never return True as in my case: >>> from ipalib import api >>> api.bootstrap(context='cli_installer') >>> api.finalize() >>> api.Backend.xmlclient.connect() ipa: INFO: trying https://vm-143.idm.lab.bos.redhat.com/ipa/xml >>> api.Command['dns_is_enabled']() ipa: INFO: Forwarding 'dns_is_enabled' to server 'http://vm-143.idm.lab.bos.redhat.com/ipa/xml' {'result': True, 'value': u'', 'summary': None} HTH, Martin From pvoborni at redhat.com Fri May 25 07:20:51 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 25 May 2012 09:20:51 +0200 Subject: [Freeipa-devel] [PATCH] 0050 Fail on unknown Command options In-Reply-To: <1337170298.2963.16.camel@balmora.brq.redhat.com> References: <4F8E97F6.2070703@redhat.com> <1336392614.29911.13.camel@balmora.brq.redhat.com> <4FABA16A.10802@redhat.com> <1336982456.4344.28.camel@balmora.brq.redhat.com> <1336984817.4344.43.camel@balmora.brq.redhat.com> <4FB0FEDB.4020806@redhat.com> <1337068555.10688.20.camel@balmora.brq.redhat.com> <4FB23F9B.10000@redhat.com> <1337155089.2963.10.camel@balmora.brq.redhat.com> <4FB3674A.1010402@redhat.com> <1337170298.2963.16.camel@balmora.brq.redhat.com> Message-ID: <4FBF32D3.1060302@redhat.com> On 05/16/2012 02:11 PM, Martin Kosek wrote: > On Wed, 2012-05-16 at 10:37 +0200, Petr Viktorin wrote: >> On 05/16/2012 09:58 AM, Martin Kosek wrote: >>> On Tue, 2012-05-15 at 13:35 +0200, Petr Viktorin wrote: >>>> On 05/15/2012 09:55 AM, Martin Kosek wrote: >>>>> On Mon, 2012-05-14 at 14:47 +0200, Petr Viktorin wrote: >>>>>> The final part of rejecting unknown Command arguments: enable the >>>>>> validation, add tests. >>>>>> Also fix up things that were changed since the previous patches. >>>>>> >>>>>> https://fedorahosted.org/freeipa/ticket/2509 >>>>>> >>>>> >>>>> The patch looks OK so far. I just found an error in permission/aci >>>>> plugin - --subtree does not work when it matches a result: >>>>> >>>>> # ipa permission-find --subtree=foo >>>>> --------------------- >>>>> 0 permissions matched >>>>> --------------------- >>>>> ---------------------------- >>>>> Number of entries returned 0 >>>>> ---------------------------- >>>>> >>>>> ipa permission-find >>>>> --subtree='ldap:///ipauniqueid=*,cn=hbac,dc=idm,dc=lab,dc=bos,dc=redhat,dc=Com' >>>>> ipa: ERROR: Unknown option: subtree >>>> >>>> Attaching fixed patch. >>>> >>>>> We should not pass **options to aci_show, it is too risky. There may be >>>>> other places where we don't use an option-safe approach that we want to >>>>> have fixed. >>>> >>>> We shouldn't really pass **options to any command; listing everything >>>> explicitly would be much safer. Unfortunately, in a lot of cases where >>>> commands call other commands, it's currently done this way. >>>> >>> >>> >>> Martin >>> >> >> Attaching a rebased patch. >> > > Yup, this one is fine. Now, I did not find issues in the patch itself, > tests are clean. > > However, thanks to this new check I found issues in Web UI (automember, > selfservice, delegation screen) which use illegal options and which > should be fixed before we push your patch: > > https://fedorahosted.org/freeipa/ticket/2760 > > Martin > I found an issue in automountmap_add_indirect. It complains that 'key' is unknown option. -- Petr Vobornik From mkosek at redhat.com Fri May 25 07:26:46 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 25 May 2012 09:26:46 +0200 Subject: [Freeipa-devel] [PATCH] 0050 Fail on unknown Command options In-Reply-To: <4FBF32D3.1060302@redhat.com> References: <4F8E97F6.2070703@redhat.com> <1336392614.29911.13.camel@balmora.brq.redhat.com> <4FABA16A.10802@redhat.com> <1336982456.4344.28.camel@balmora.brq.redhat.com> <1336984817.4344.43.camel@balmora.brq.redhat.com> <4FB0FEDB.4020806@redhat.com> <1337068555.10688.20.camel@balmora.brq.redhat.com> <4FB23F9B.10000@redhat.com> <1337155089.2963.10.camel@balmora.brq.redhat.com> <4FB3674A.1010402@redhat.com> <1337170298.2963.16.camel@balmora.brq.redhat.com> <4FBF32D3.1060302@redhat.com> Message-ID: <1337930806.5899.13.camel@balmora.brq.redhat.com> On Fri, 2012-05-25 at 09:20 +0200, Petr Vobornik wrote: > On 05/16/2012 02:11 PM, Martin Kosek wrote: > > On Wed, 2012-05-16 at 10:37 +0200, Petr Viktorin wrote: > >> On 05/16/2012 09:58 AM, Martin Kosek wrote: > >>> On Tue, 2012-05-15 at 13:35 +0200, Petr Viktorin wrote: > >>>> On 05/15/2012 09:55 AM, Martin Kosek wrote: > >>>>> On Mon, 2012-05-14 at 14:47 +0200, Petr Viktorin wrote: > >>>>>> The final part of rejecting unknown Command arguments: enable the > >>>>>> validation, add tests. > >>>>>> Also fix up things that were changed since the previous patches. > >>>>>> > >>>>>> https://fedorahosted.org/freeipa/ticket/2509 > >>>>>> > >>>>> > >>>>> The patch looks OK so far. I just found an error in permission/aci > >>>>> plugin - --subtree does not work when it matches a result: > >>>>> > >>>>> # ipa permission-find --subtree=foo > >>>>> --------------------- > >>>>> 0 permissions matched > >>>>> --------------------- > >>>>> ---------------------------- > >>>>> Number of entries returned 0 > >>>>> ---------------------------- > >>>>> > >>>>> ipa permission-find > >>>>> --subtree='ldap:///ipauniqueid=*,cn=hbac,dc=idm,dc=lab,dc=bos,dc=redhat,dc=Com' > >>>>> ipa: ERROR: Unknown option: subtree > >>>> > >>>> Attaching fixed patch. > >>>> > >>>>> We should not pass **options to aci_show, it is too risky. There may be > >>>>> other places where we don't use an option-safe approach that we want to > >>>>> have fixed. > >>>> > >>>> We shouldn't really pass **options to any command; listing everything > >>>> explicitly would be much safer. Unfortunately, in a lot of cases where > >>>> commands call other commands, it's currently done this way. > >>>> > >>> > >>> > >>> Martin > >>> > >> > >> Attaching a rebased patch. > >> > > > > Yup, this one is fine. Now, I did not find issues in the patch itself, > > tests are clean. > > > > However, thanks to this new check I found issues in Web UI (automember, > > selfservice, delegation screen) which use illegal options and which > > should be fixed before we push your patch: > > > > https://fedorahosted.org/freeipa/ticket/2760 > > > > Martin > > > > I found an issue in automountmap_add_indirect. It complains that 'key' > is unknown option. I assume this is a cause and would need to be fixed in Petr3's patch: 847 # Add a submount key 848 self.api.Command['automountkey_add']( 849 location, parentmap, automountkey=key, key=key, 850 automountinformation='-fstype=autofs ldap:%s' % map) Martin From mkosek at redhat.com Fri May 25 07:51:37 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 25 May 2012 09:51:37 +0200 Subject: [Freeipa-devel] [PATCH] 1022 normalize uid when in winsync In-Reply-To: <4FBE75F0.3050400@redhat.com> References: <4FBE75F0.3050400@redhat.com> Message-ID: <1337932297.5899.15.camel@balmora.brq.redhat.com> On Thu, 2012-05-24 at 13:54 -0400, Rob Crittenden wrote: > In case the uid that comes from AD is mixed-case we need to normalize it > to all lower. It should be safe using tolower() because we only allow > ASCII characters in uid. > > rob I tested this with a winsync agreement with an AD and it worked fine, I as able to change password and then log in as an AD-mixed-case user. ACK, pushed to master. Martin From atkac at redhat.com Fri May 25 12:41:06 2012 From: atkac at redhat.com (Adam Tkac) Date: Fri, 25 May 2012 14:41:06 +0200 Subject: [Freeipa-devel] DNS zone serial number updates [#2554]: local SOA approach In-Reply-To: <4FA83098.4000800@redhat.com> References: <4F8D9110.3030703@redhat.com> <1334679211.16658.200.camel@willson.li.ssimo.org> <4F8EC1C8.4090204@redhat.com> <1334757864.16658.258.camel@willson.li.ssimo.org> <4F8EDBE0.7090108@redhat.com> <1334764404.16658.277.camel@willson.li.ssimo.org> <4FA7EEB1.1040909@redhat.com> <4FA83098.4000800@redhat.com> Message-ID: <20120525124105.GA11168@redhat.com> On Mon, May 07, 2012 at 10:29:12PM +0200, Petr Spacek wrote: > Hello, > > on the last meeting there was another approach to $SUBJ$ discussed: > > Each DNS server will maintain its own serial number value > independently from other servers. > > Pros: > Should be simpler to implement; no DS plugin required. > > Cons: > Slave DNS servers cannot fall-back to other masters, because of SOA > serial inconsistency. > > > Very basic implementation: > 1) Do not replicate idnsSoaSerial attribute > 2) Use persistent search to watch for incoming changes > 3) After each change increment "local" SOA serial number (and write > it to LDAP - to survive DNS server restart) > > There is a problem with database updates if bind-dyndb-ldap plugin > is not running, but "its" DS runs. In that case DS can replicate > changes from other servers but there is no bind-dyndb-ldap plugin to > modify serial number. > > I think it can happen during startup/shutdown phase or after BIND > crash. In this case zone content can be changed without appropriate > SOA serial update. > > Dumb solution: > Always increment stored SOA serial number after DNS server start. It > causes unnecessary zone transfer, but we are safe. > > Another solution (IPA+389 specific): > Remember max(entryUSN) value computed from all records together with > SOA serial. (Principle is the same as with modifyTimestamp described > earlier in this thread.) It requires new LDAP object with two > attributes: > - assigned value of idnsSOASerial > - highest entryUSN found > DNS server after start can check if max(entryUSN) == recorded max > value. If values do not match plugin bumps idnsSOASerial. > > If entryUSN is not supported by server, we can fall-back to bumping > idnsSOASerial on every start-up. > > Did I miss anything? :-) > > It requires persistent search, but I resigned already. After further discussion this seems like the best approach for me as well. Regards, Adam -- Adam Tkac, Red Hat, Inc. From jdennis at redhat.com Fri May 25 12:45:17 2012 From: jdennis at redhat.com (John Dennis) Date: Fri, 25 May 2012 08:45:17 -0400 Subject: [Freeipa-devel] dns.py question In-Reply-To: <1337928074.5899.12.camel@balmora.brq.redhat.com> References: <4FBE5CD1.4020500@redhat.com> <1337928074.5899.12.camel@balmora.brq.redhat.com> Message-ID: <4FBF7EDD.9070005@redhat.com> On 05/25/2012 02:41 AM, Martin Kosek wrote: > base_dn is defined in dns_is_enabled command class along with filter: Thanks Martin! I missed the fact it was defined in class scope, not method scope. Duh on my part :-( I thought I was nuts, now I know for sure :-) -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From mkosek at redhat.com Fri May 25 12:50:10 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 25 May 2012 14:50:10 +0200 Subject: [Freeipa-devel] [PATCH] 262-265 Enable psearch by default Message-ID: <1337950210.5899.25.camel@balmora.brq.redhat.com> This set of patches handles enabling psearch both for new installations (patch 263) and upgraded IPA servers. For upgraded IPA servers I needed to make sure that psearch is not enabled for every IPA package update, but at most once, when a user updates to IPA with this patch for the first time (patch 264). This is enabled by a new State store located in /var/lib/ipa/sysupgrade (patch 262). I also improved the way we handled SELinux sebool updates (patch 265), this can make ipa-upgradeconfig to finish in 0.4 seconds and not in 150 seconds as previously. Details are in the patches. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-262-add-sysupgrade-state-file.patch Type: text/x-patch Size: 8823 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-263-enable-persistent-search-by-default.patch Type: text/x-patch Size: 13484 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-264-enable-psearch-on-upgrades.patch Type: text/x-patch Size: 9296 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-265-only-set-sebools-when-necessary.patch Type: text/x-patch Size: 3197 bytes Desc: not available URL: From rcritten at redhat.com Fri May 25 13:25:01 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 25 May 2012 09:25:01 -0400 Subject: [Freeipa-devel] [PATCH] 262-265 Enable psearch by default In-Reply-To: <1337950210.5899.25.camel@balmora.brq.redhat.com> References: <1337950210.5899.25.camel@balmora.brq.redhat.com> Message-ID: <4FBF882D.9000909@redhat.com> Martin Kosek wrote: > This set of patches handles enabling psearch both for new installations > (patch 263) and upgraded IPA servers. > > For upgraded IPA servers I needed to make sure that psearch is not > enabled for every IPA package update, but at most once, when a user > updates to IPA with this patch for the first time (patch 264). This is > enabled by a new State store located in /var/lib/ipa/sysupgrade (patch > 262). > > I also improved the way we handled SELinux sebool updates (patch 265), > this can make ipa-upgradeconfig to finish in 0.4 seconds and not in 150 > seconds as previously. Details are in the patches. > > Martin 262: The sysupgrade directory isn't created by the RPM install: mkdir -p %{buildroot}/%{_localstatedir}/cache/ipa/sysupgrade 263: It looks like zone_refresh is simply disabled in bindinstance.py, why not remove it completely? 264: Small nit, worth doing case-insensitive compare of psearch enabled status? We're updating named.conf in place so I don't know that we need to reset permissions. It at least shouldn't get modified by the write. rob From pspacek at redhat.com Fri May 25 13:52:40 2012 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 25 May 2012 15:52:40 +0200 Subject: [Freeipa-devel] routing requests to local servers - DNS SRV [discussion needed] In-Reply-To: <4FBE7723.2060206@redhat.com> References: <4FBE6AC9.4020703@redhat.com> <4FBE7723.2060206@redhat.com> Message-ID: <4FBF8EA8.4040900@redhat.com> On 05/24/2012 08:00 PM, Dmitri Pal wrote: > On 05/24/2012 01:07 PM, Petr Spacek wrote: >> Hello, >> >> some time ago there was a request for DNS to support "routing requests >> to local servers". Any opinions if/when do it and proposals how to do >> it are more than welcome. My best knowledge about this problem follows: >> >> >> This request actually means "differentiate answer to DNS query on >> client's IP address basics". >> Relevant thread on ipa-users-list: >> https://www.redhat.com/archives/freeipa-users/2012-April/msg00070.html >> >> First question is: Do we want to implement it? >> (IMHO it is very important for large-scale deployments.) >> >> >> It is related to IPA ticket #122 at architecture level: >> IPA DNS integration should support GeoIP >> https://fedorahosted.org/freeipa/ticket/122 >> IMHO it is better to support generic "locations" defined by user than >> real Geo-IP: >> - at least private IP addresses will be problematic >> - real Geo-IP database is huge and can slow all things down >> >> >> Bind-dyndb-ldap plugin supports BIND "views" already. It should be >> possible to define multiple plugin instances with different ldap >> search base and let it to serve records. >> >> The main problem problem is managing DNS subtree in LDAP. With current >> implementation you have to copy whole subtree to another place and >> then change necessary records. Problem is with maintaining "base" >> (non-overridden) records - you have to do same change n-times. >> >> >> Adam and I discussed possible approaches. We agreed on "stackable" >> approach: >> - Store whole original DNS tree in one subtree, let say "base". >> - Create "override" subtrees for each "locality" = set of customized >> records. Shared records are only in "base". During DNS query >> processing "override" subtree is searched first. If record does not >> exist in "override" subtree, search will continue in "base" subtree. >> >> >> Second question is how to realize these "overrides": >> - Let Directory Server to do the work. If DS supports this, it is >> mostly done. >> It do not require any change in bind-dyndb-ldap code. All >> merges/overrides will be done on Directory server. Only >> /etc/named.conf has to be adjusted with right search base and "view" >> clause. >> >> I asked on 389 mailing list and I'm waiting for reply: >> http://lists.fedoraproject.org/pipermail/389-users/2012-May/014653.html >> >> >> - Another approach is to add support directly to bind-dyndb-ldap, but >> it is not so nice solution. In that case both LDAP search bases have >> to be defined in /etc/named.conf. For each DNS query is necessary to >> search "override" base first. If required record is not present then >> is necessary to search through "base" subtree. >> With persistent search it should be better, because all records are in >> memory, but basic principle remains same. >> >> >> Thanks for all opinions. >> >> Petr^2 Spacek > > May be I am oversimplifying things but it seems that it would make sense > to just have an extra multi-value attribute added to the DNS schema. > This attribute would contain a value that would allow the entry to be > included into the view. This way the base is the same and what the view > exposes is just a filter. The default view would have a filter that > matches all entries like now. > > > I assume that DNS server makes it decision based on the IP so it has a > call to get a "view" data. The ldap driver can return a filter. The DNS > server them makes a call using this filter to get all the records that > apply. I know it is too ldap centric. There is some abstraction in DNS > server and we do not want to change it. But the point is there are > probably two calls in the DNS server (have not actually confirmed): > lookup data X by IP to understand what the view is. > Pass data X to get the actual DNS entries. > > I am saying that X can be filter. > > Thoughts? Special attribute sounds like a good idea. It is not realizable directly, but I will explore variants with some "view" attribute. Current DNS "name" (name can potentially contain multiple records) structure is following: dn: idnsname=_kerberos._tcp,idnsname=e.localnet,cn=dns,dc=e,dc=org objectClass: idnsrecord objectClass: top idnsName: _kerberos._tcp sRVRecord: 0 100 88 unused-4-107 DNS name is part of DN. It is not possible to have more objects with same DNS name and different attributes. This problem lead me to "stackable" approach. I'm looking into "Views" in DS now. Petr^2 Spacek From simo at redhat.com Fri May 25 14:10:53 2012 From: simo at redhat.com (Simo Sorce) Date: Fri, 25 May 2012 10:10:53 -0400 Subject: [Freeipa-devel] routing requests to local servers - DNS SRV [discussion needed] In-Reply-To: <4FBE6AC9.4020703@redhat.com> References: <4FBE6AC9.4020703@redhat.com> Message-ID: <1337955053.16840.286.camel@willson.li.ssimo.org> On Thu, 2012-05-24 at 19:07 +0200, Petr Spacek wrote: > Hello, > > some time ago there was a request for DNS to support "routing requests to > local servers". Any opinions if/when do it and proposals how to do it are more > than welcome. My best knowledge about this problem follows: > > > This request actually means "differentiate answer to DNS query on client's IP > address basics". > Relevant thread on ipa-users-list: > https://www.redhat.com/archives/freeipa-users/2012-April/msg00070.html > > First question is: Do we want to implement it? > (IMHO it is very important for large-scale deployments.) I am not really sure I like it, as it makes things quite difficult to handle. Please think how we are going to operate if you ahve a view dfined and a client does a dyn DNS update. > It is related to IPA ticket #122 at architecture level: > IPA DNS integration should support GeoIP > https://fedorahosted.org/freeipa/ticket/122 > IMHO it is better to support generic "locations" defined by user than real Geo-IP: > - at least private IP addresses will be problematic > - real Geo-IP database is huge and can slow all things down Geo-IP is good for some services, mostly internet facing high availability stuff. For internal networks we should just use the location code, as Geo-IP tagging makes no sense for private networks. I say we should defer any work on Geo-IP tagging as we do not target public facing high avilability IP serving with our DNS at the moemnt (Ie I expect those people will delegate a zone to a dedicated set of DNS servers instead). Ah incidentally this means we need to start supporting zone delegation from the management UI if we do not already support that. > Bind-dyndb-ldap plugin supports BIND "views" already. It should be possible to > define multiple plugin instances with different ldap search base and let it to > serve records. > The main problem problem is managing DNS subtree in LDAP. With current > implementation you have to copy whole subtree to another place and then change > necessary records. Problem is with maintaining "base" (non-overridden) records > - you have to do same change n-times. Using different bases would require replicating the whole database indeed with common entries you do not change per view. That's a lot of effort, and very easy to screw up. I think we really do not want to take this approach. > Adam and I discussed possible approaches. We agreed on "stackable" approach: > - Store whole original DNS tree in one subtree, let say "base". > - Create "override" subtrees for each "locality" = set of customized records. > Shared records are only in "base". During DNS query processing "override" > subtree is searched first. If record does not exist in "override" subtree, > search will continue in "base" subtree. Yes, this is my first thought too. > Second question is how to realize these "overrides": > - Let Directory Server to do the work. If DS supports this, it is mostly done. Why do you need dirsrv to do anything special ? The simple way is to do subtree searches, if you get back more than one result for a specific name then you know you have views and proceed to filter out the one you want. The bonus here is that you can cache all replies if you keep different caches per view. The alternative is to add a 'viewname' or something in the filter, but then you need to do 2 searches and fallbacks. This sounds more expensive. > It do not require any change in bind-dyndb-ldap code. All merges/overrides > will be done on Directory server. Given we do persistent searches and we also do some caching in bind-dyndb-ldap we almost certainly do not want to 'fool' it by returning different values from DS w/o bind-dyndb-ldap knowledge. > Only /etc/named.conf has to be adjusted with > right search base and "view" clause. > > I asked on 389 mailing list and I'm waiting for reply: > http://lists.fedoraproject.org/pipermail/389-users/2012-May/014653.html > > > - Another approach is to add support directly to bind-dyndb-ldap, but it is > not so nice solution. Why ? > In that case both LDAP search bases have to be defined > in /etc/named.conf. For each DNS query is necessary to search "override" base > first. If required record is not present then is necessary to search through > "base" subtree. No, you do not need to do multiple searches, you just need to filter the results by view. It would make it very easy to manage if we added a 'dnsView' attribute to overriding entries in the subtree. This would allow us to define a new overriding entry and even assign it to multiple views if the 'dnsView' attribute is multivalued. > With persistent search it should be better, because all records are in memory, > but basic principle remains same. psearch is one of the reasons we might want a dnsView attribute base approach instead of playing games with DS. I would also prefer to avoid adding more code to DS where the bind plugin can easily handle this data itself. Another reason is that I really want this plugin code to be generic and usable with other LDAP servers and not be tied so strictly to 389ds. Simo. -- Simo Sorce * Red Hat, Inc * New York From pvoborni at redhat.com Fri May 25 14:40:59 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 25 May 2012 16:40:59 +0200 Subject: [Freeipa-devel] [PATCH] 147 Set network.http.sendRefererHeader to 2 on browser config Message-ID: <4FBF99FB.7090206@redhat.com> IPA web UI isn't functional when browser doesn't send http headers. This patch adds a functionality which sets Firefox network.http.sendRefererHeader configuration option to value '2' which enables it. Possible values: http://kb.mozillazine.org/Network.http.sendRefererHeader https://fedorahosted.org/freeipa/ticket/2778 I was also thinking about upgrading the configure.jar. We had a ticket for it, which ended by documenting the steps. https://fedorahosted.org/freeipa/ticket/2311 http://docs.fedoraproject.org/en-US/Fedora/16/html/FreeIPA_Guide/upgrading.html#ticket-delegation I think the documentation is wrong. In it we are rebuilding the .jar from /usr/share/ipa/html/preferences.html, this file is created on server install and it is never updated therefore the .jar won't be updated. The updated file is its template (the one changed in this patch). The template output is created in httpinstance.__setup_autoconfig() call. For my development purposes I took this code and created a script which rebuilds the .jar file (attached). Do we want to use it? -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0147-Set-network.http.sendRefererHeader-to-2-on-browser-c.patch Type: text/x-patch Size: 3085 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: build-configure-jar.py Type: text/x-python Size: 2557 bytes Desc: not available URL: From pvoborni at redhat.com Fri May 25 14:57:47 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 25 May 2012 16:57:47 +0200 Subject: [Freeipa-devel] [PATCH] 148 Removal of illegal options in JSON-RPC calls Message-ID: <4FBF9DEB.8050905@redhat.com> Ticket https://fedorahosted.org/freeipa/ticket/2509 bans using non existent options. If such option is supplied command ends with error. It uncovered several cases in Web UI. This patch is fixing these cases. Automember, Self-service and Delegation don't support 'pkey-only', 'size-limit' and 'rights' option. Pagination and rights check were disabled for them. Automount map adder dialog was sending options for indirect map even if chosen type was direct (when those for indirect was filled earlier), also it was sending non-existant 'method' option. https://fedorahosted.org/freeipa/ticket/2760 Note for reviewing: #2509 is partially done in Petr Viktorin's patch #35. At this time it has a small issue regarding automountmap_add_indirect command. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0148-Removal-of-illegal-options-in-JSON-RPC-calls.patch Type: text/x-patch Size: 7012 bytes Desc: not available URL: From mkosek at redhat.com Fri May 25 15:14:31 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 25 May 2012 17:14:31 +0200 Subject: [Freeipa-devel] [PATCH] 262-265 Enable psearch by default In-Reply-To: <4FBF882D.9000909@redhat.com> References: <1337950210.5899.25.camel@balmora.brq.redhat.com> <4FBF882D.9000909@redhat.com> Message-ID: <1337958871.5899.32.camel@balmora.brq.redhat.com> On Fri, 2012-05-25 at 09:25 -0400, Rob Crittenden wrote: > Martin Kosek wrote: > > This set of patches handles enabling psearch both for new installations > > (patch 263) and upgraded IPA servers. > > > > For upgraded IPA servers I needed to make sure that psearch is not > > enabled for every IPA package update, but at most once, when a user > > updates to IPA with this patch for the first time (patch 264). This is > > enabled by a new State store located in /var/lib/ipa/sysupgrade (patch > > 262). > > > > I also improved the way we handled SELinux sebool updates (patch 265), > > this can make ipa-upgradeconfig to finish in 0.4 seconds and not in 150 > > seconds as previously. Details are in the patches. > > > > Martin > > 262: > The sysupgrade directory isn't created by the RPM install: > > mkdir -p %{buildroot}/%{_localstatedir}/cache/ipa/sysupgrade Fixed. > > 263: > > It looks like zone_refresh is simply disabled in bindinstance.py, why > not remove it completely? zone_refresh is used by bindinstance.py. ipa-server-install or ipa-dns-install may be configured to use zone refresh instead of persistent search mechanism to update the zones (e.g. --zone-refresh 30). > > 264: > > Small nit, worth doing case-insensitive compare of psearch enabled status? Petr2 told me that arg value for boolean configuration option is case-insensitive, so we can do that - fixed. > > We're updating named.conf in place so I don't know that we need to reset > permissions. It at least shouldn't get modified by the write. Right, I was being too defensive. I removed the check. I made the upgrade more robust, now it won't crash for example when named.conf does not exist. I also made sure the upgrade script works correctly when the IPA is configured without DNS. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-262-2-add-sysupgrade-state-file.patch Type: text/x-patch Size: 9237 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-263-2-enable-persistent-search-by-default.patch Type: text/x-patch Size: 13484 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-264-2-enable-psearch-on-upgrades.patch Type: text/x-patch Size: 11545 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-265-2-only-set-sebools-when-necessary.patch Type: text/x-patch Size: 3197 bytes Desc: not available URL: From pviktori at redhat.com Fri May 25 15:17:58 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 25 May 2012 17:17:58 +0200 Subject: [Freeipa-devel] [PATCH] 0054 Provide a better error message when deleting nonexistent attributes Message-ID: <4FBFA2A6.7050804@redhat.com> This fixes "misleading/invalid" error messages given when using --delattr to delete values from an attribute that doesn't exist on the entry. Please see the trac comment for details. https://fedorahosted.org/freeipa/ticket/2699 -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0054-Provide-a-better-error-message-when-deleting-nonexis.patch Type: text/x-patch Size: 3442 bytes Desc: not available URL: From pviktori at redhat.com Fri May 25 15:39:59 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 25 May 2012 17:39:59 +0200 Subject: [Freeipa-devel] [PATCH] 0055 Add more automount tests Message-ID: <4FBFA7CF.4050407@redhat.com> Martin and Petr Voborn?k found an automount plugin bug in my patch 50. I checked the automount test coverage and found that it isn't that great ? not only automount-key-add-indirect with the parentmap flag, but the import and tofiles commands weren't tested at all. This patch makes the situation a bit better. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0055-Add-more-automount-tests.patch Type: text/x-patch Size: 13899 bytes Desc: not available URL: From pviktori at redhat.com Fri May 25 15:44:18 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 25 May 2012 17:44:18 +0200 Subject: [Freeipa-devel] [PATCH] 0050 Fail on unknown Command options In-Reply-To: <1337930806.5899.13.camel@balmora.brq.redhat.com> References: <4F8E97F6.2070703@redhat.com> <1336392614.29911.13.camel@balmora.brq.redhat.com> <4FABA16A.10802@redhat.com> <1336982456.4344.28.camel@balmora.brq.redhat.com> <1336984817.4344.43.camel@balmora.brq.redhat.com> <4FB0FEDB.4020806@redhat.com> <1337068555.10688.20.camel@balmora.brq.redhat.com> <4FB23F9B.10000@redhat.com> <1337155089.2963.10.camel@balmora.brq.redhat.com> <4FB3674A.1010402@redhat.com> <1337170298.2963.16.camel@balmora.brq.redhat.com> <4FBF32D3.1060302@redhat.com> <1337930806.5899.13.camel@balmora.brq.redhat.com> Message-ID: <4FBFA8D2.5010105@redhat.com> On 05/25/2012 09:26 AM, Martin Kosek wrote: > On Fri, 2012-05-25 at 09:20 +0200, Petr Vobornik wrote: >> On 05/16/2012 02:11 PM, Martin Kosek wrote: >>> On Wed, 2012-05-16 at 10:37 +0200, Petr Viktorin wrote: >>>> On 05/16/2012 09:58 AM, Martin Kosek wrote: >>>>> On Tue, 2012-05-15 at 13:35 +0200, Petr Viktorin wrote: >>>>>> On 05/15/2012 09:55 AM, Martin Kosek wrote: >>>>>>> On Mon, 2012-05-14 at 14:47 +0200, Petr Viktorin wrote: >>>>>>>> The final part of rejecting unknown Command arguments: enable the >>>>>>>> validation, add tests. >>>>>>>> Also fix up things that were changed since the previous patches. >>>>>>>> >>>>>>>> https://fedorahosted.org/freeipa/ticket/2509 >>>>>>>> >>>>>>> >>>>>>> The patch looks OK so far. I just found an error in permission/aci >>>>>>> plugin - --subtree does not work when it matches a result: >>>>>>> >>>>>>> # ipa permission-find --subtree=foo >>>>>>> --------------------- >>>>>>> 0 permissions matched >>>>>>> --------------------- >>>>>>> ---------------------------- >>>>>>> Number of entries returned 0 >>>>>>> ---------------------------- >>>>>>> >>>>>>> ipa permission-find >>>>>>> --subtree='ldap:///ipauniqueid=*,cn=hbac,dc=idm,dc=lab,dc=bos,dc=redhat,dc=Com' >>>>>>> ipa: ERROR: Unknown option: subtree >>>>>> >>>>>> Attaching fixed patch. >>>>>> >>>>>>> We should not pass **options to aci_show, it is too risky. There may be >>>>>>> other places where we don't use an option-safe approach that we want to >>>>>>> have fixed. >>>>>> >>>>>> We shouldn't really pass **options to any command; listing everything >>>>>> explicitly would be much safer. Unfortunately, in a lot of cases where >>>>>> commands call other commands, it's currently done this way. >>>>>> >>>>> >>>>> >>>>> Martin >>>>> >>>> >>>> Attaching a rebased patch. >>>> >>> >>> Yup, this one is fine. Now, I did not find issues in the patch itself, >>> tests are clean. >>> >>> However, thanks to this new check I found issues in Web UI (automember, >>> selfservice, delegation screen) which use illegal options and which >>> should be fixed before we push your patch: >>> >>> https://fedorahosted.org/freeipa/ticket/2760 >>> >>> Martin >>> >> >> I found an issue in automountmap_add_indirect. It complains that 'key' >> is unknown option. > > I assume this is a cause and would need to be fixed in Petr3's patch: > > 847 # Add a submount key > 848 self.api.Command['automountkey_add']( > 849 location, parentmap, automountkey=key, key=key, > 850 automountinformation='-fstype=autofs ldap:%s' % > map) > > Martin > Thanks! Fixed and rebased. I'll test this code path in a separate patch. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0050-04-Fail-on-unknown-Command-options.patch Type: text/x-patch Size: 16076 bytes Desc: not available URL: From pvoborni at redhat.com Fri May 25 16:04:50 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 25 May 2012 18:04:50 +0200 Subject: [Freeipa-devel] [PATCH] 148 Removal of illegal options in JSON-RPC calls In-Reply-To: <4FBF9DEB.8050905@redhat.com> References: <4FBF9DEB.8050905@redhat.com> Message-ID: <4FBFADA2.6010404@redhat.com> On 05/25/2012 04:57 PM, Petr Vobornik wrote: > Ticket https://fedorahosted.org/freeipa/ticket/2509 bans using non > existent options. If such option is supplied command ends with error. It > uncovered several cases in Web UI. This patch is fixing these cases. > > Automember, Self-service and Delegation don't support 'pkey-only', > 'size-limit' and 'rights' option. Pagination and rights check were > disabled for them. > > Automount map adder dialog was sending options for indirect map even if > chosen type was direct (when those for indirect was filled earlier), > also it was sending non-existant 'method' option. > > https://fedorahosted.org/freeipa/ticket/2760 > > Note for reviewing: #2509 is partially done in Petr Viktorin's patch > #35. At this time it has a small issue regarding > automountmap_add_indirect command. > I found more used non existing options in association adder dialog code: in hbac hbacsvcgroup_find: no_hbacsvc. I will investigate these cases more deeply. -- Petr Vobornik From mkosek at redhat.com Fri May 25 16:09:54 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 25 May 2012 18:09:54 +0200 Subject: [Freeipa-devel] [PATCH] 79 SSH configuration fixes In-Reply-To: <4FBCAAEB.9000100@redhat.com> References: <4FBCAAEB.9000100@redhat.com> Message-ID: <1337962194.5899.37.camel@balmora.brq.redhat.com> On Wed, 2012-05-23 at 11:16 +0200, Jan Cholasta wrote: > Hi, > > this fixes https://fedorahosted.org/freeipa/ticket/2769 as well as some > other issues with SSH configuration in ipa-client-install. > > Honza > This fixed the basic functionality, but I discovered another issue (quite serious one). With /usr/bin/sss_ssh_knownhostsproxy as ssh ProxyCommand, I cannot connect to remove client which does not have valid reverse record. Without the proxy command, it works fine. I logged a Bugzilla for this issue: https://bugzilla.redhat.com/show_bug.cgi?id=825316 Martin From pvoborni at redhat.com Fri May 25 16:23:26 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 25 May 2012 18:23:26 +0200 Subject: [Freeipa-devel] [PATCH] 149 Added links to netgroup member tables Message-ID: <4FBFB1FE.3070703@redhat.com> Tables with members in netgroup were missing links for navigation to associated details pages. This patch adds these links. https://fedorahosted.org/freeipa/ticket/2670 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0149-Added-links-to-netgroup-member-tables.patch Type: text/x-patch Size: 2788 bytes Desc: not available URL: From simo at redhat.com Fri May 25 19:20:49 2012 From: simo at redhat.com (Simo Sorce) Date: Fri, 25 May 2012 15:20:49 -0400 Subject: [Freeipa-devel] routing requests to local servers - DNS SRV [discussion needed] In-Reply-To: <4FBF8EA8.4040900@redhat.com> References: <4FBE6AC9.4020703@redhat.com> <4FBE7723.2060206@redhat.com> <4FBF8EA8.4040900@redhat.com> Message-ID: <1337973649.16840.627.camel@willson.li.ssimo.org> On Fri, 2012-05-25 at 15:52 +0200, Petr Spacek wrote: > On 05/24/2012 08:00 PM, Dmitri Pal wrote: > > On 05/24/2012 01:07 PM, Petr Spacek wrote: > >> Hello, > >> > >> some time ago there was a request for DNS to support "routing requests > >> to local servers". Any opinions if/when do it and proposals how to do > >> it are more than welcome. My best knowledge about this problem follows: > >> > >> > >> This request actually means "differentiate answer to DNS query on > >> client's IP address basics". > >> Relevant thread on ipa-users-list: > >> https://www.redhat.com/archives/freeipa-users/2012-April/msg00070.html > >> > >> First question is: Do we want to implement it? > >> (IMHO it is very important for large-scale deployments.) > >> > >> > >> It is related to IPA ticket #122 at architecture level: > >> IPA DNS integration should support GeoIP > >> https://fedorahosted.org/freeipa/ticket/122 > >> IMHO it is better to support generic "locations" defined by user than > >> real Geo-IP: > >> - at least private IP addresses will be problematic > >> - real Geo-IP database is huge and can slow all things down > >> > >> > >> Bind-dyndb-ldap plugin supports BIND "views" already. It should be > >> possible to define multiple plugin instances with different ldap > >> search base and let it to serve records. > >> > >> The main problem problem is managing DNS subtree in LDAP. With current > >> implementation you have to copy whole subtree to another place and > >> then change necessary records. Problem is with maintaining "base" > >> (non-overridden) records - you have to do same change n-times. > >> > >> > >> Adam and I discussed possible approaches. We agreed on "stackable" > >> approach: > >> - Store whole original DNS tree in one subtree, let say "base". > >> - Create "override" subtrees for each "locality" = set of customized > >> records. Shared records are only in "base". During DNS query > >> processing "override" subtree is searched first. If record does not > >> exist in "override" subtree, search will continue in "base" subtree. > >> > >> > >> Second question is how to realize these "overrides": > >> - Let Directory Server to do the work. If DS supports this, it is > >> mostly done. > >> It do not require any change in bind-dyndb-ldap code. All > >> merges/overrides will be done on Directory server. Only > >> /etc/named.conf has to be adjusted with right search base and "view" > >> clause. > >> > >> I asked on 389 mailing list and I'm waiting for reply: > >> http://lists.fedoraproject.org/pipermail/389-users/2012-May/014653.html > >> > >> > >> - Another approach is to add support directly to bind-dyndb-ldap, but > >> it is not so nice solution. In that case both LDAP search bases have > >> to be defined in /etc/named.conf. For each DNS query is necessary to > >> search "override" base first. If required record is not present then > >> is necessary to search through "base" subtree. > >> With persistent search it should be better, because all records are in > >> memory, but basic principle remains same. > >> > >> > >> Thanks for all opinions. > >> > >> Petr^2 Spacek > > > > May be I am oversimplifying things but it seems that it would make sense > > to just have an extra multi-value attribute added to the DNS schema. > > This attribute would contain a value that would allow the entry to be > > included into the view. This way the base is the same and what the view > > exposes is just a filter. The default view would have a filter that > > matches all entries like now. > > > > > > I assume that DNS server makes it decision based on the IP so it has a > > call to get a "view" data. The ldap driver can return a filter. The DNS > > server them makes a call using this filter to get all the records that > > apply. I know it is too ldap centric. There is some abstraction in DNS > > server and we do not want to change it. But the point is there are > > probably two calls in the DNS server (have not actually confirmed): > > lookup data X by IP to understand what the view is. > > Pass data X to get the actual DNS entries. > > > > I am saying that X can be filter. > > > > Thoughts? > > Special attribute sounds like a good idea. It is not realizable directly, but > I will explore variants with some "view" attribute. > > Current DNS "name" (name can potentially contain multiple records) structure > is following: > > dn: idnsname=_kerberos._tcp,idnsname=e.localnet,cn=dns,dc=e,dc=org > objectClass: idnsrecord > objectClass: top > idnsName: _kerberos._tcp > sRVRecord: 0 100 88 unused-4-107 > > DNS name is part of DN. It is not possible to have more objects with same DNS > name and different attributes. This problem lead me to "stackable" approach. Yes, and we can also use multiple attributes in the same tree, although for clarity I probably prefer the subtree approach. So a few options: 1. all in the same subtree: # Normal object dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org objectClass: idnsrecord objectClass: top idnsName: bar aRecord: 192.168.12.34 dNSTTL: 1200 # Object belongin to the 'DMZ' view dn:cn=DMZ-bar,idnsname=foo.org,cn=dns,dc=foo,dc=org objectClass: idnsrecord objectClass: top objectClass: nsContainer cn: DMZ-bar idnsName: bar aRecord: 5.6.7.8 dNSTTL: 3600 idnsView: DMZ NOTE: I had to add nsContainer here in order to give the object a way to have a unique name by using the CN attribute. I am not very fond of this arrangement though. It is also ugly to parse out using a LDAP browser. It make one thing simpler in that using multiple values for dnsView you can assign the same entry to multiple views. 2. using per view subtrees # Normal object dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org objectClass: idnsrecord objectClass: top idnsName: bar aRecord: 192.168.12.34 dNSTTL: 1200 # Object belongin to the 'DMZ' view dn:idnsname=bar,cn=DMZ,idnsname=foo.org,cn=dns,dc=foo,dc=org objectClass: idnsrecord objectClass: top idnsName: bar aRecord: 5.6.7.8 dNSTTL: 3600 NOTE: I prefer this method as it makes things a lot easier to manage and view through an LDAP broiwser, however it makes sharing entries between multiple views a bit awkward. 3. using only one 'views' subtree pr zone and dnsView to discrimnate # Normal object dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org objectClass: idnsrecord objectClass: top idnsName: bar aRecord: 192.168.12.34 dNSTTL: 1200 # Object belongin to the 'DMZ' view dn:idnsUniqueID=F6A1245-bar,cn=views,idnsname=foo.org,cn=dns,dc=foo,dc=org objectClass: idnsrecord objectClass: top idnsUniqueID: F6A1245-bar idnsName: bar aRecord: 5.6.7.8 dNSTTL: 3600 idnsView: DMZ idnsView: VPN NOTE: here I added also a idnsUniqueID as a way to have unique names so we can have multiple entries for the same record. This is so that you can have 3 different entries for the same record belonging to 3 different views. The reason why I added the actual name after a random id is that this way it is simpler to recognize what it is when looking at an ldap browser w/o having to read the actual object attributes, it also make collisions a lot less likely and so it allows to keep the random part smaller (and thus more readable). Also note that I've put 2 values in idnsView, meaning that this record belongs to 2 separate views. This allows compact representation when multiple views want to redefine some records in the same way (an dothers in a different way, thus why 2 separate views)/ > I'm looking into "Views" in DS now. We do not want to make this code 389DS specific, anyways, so I think we should not go there. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Fri May 25 22:36:20 2012 From: simo at redhat.com (Simo Sorce) Date: Fri, 25 May 2012 18:36:20 -0400 Subject: [Freeipa-devel] [PATCH] 492 Add options to reduce writes from KDC Message-ID: <1337985380.16840.643.camel@willson.li.ssimo.org> The original ldap driver we used up to 2.2 had 2 options admins could set to limit the amount of writes to the database on certain auditing related operations. In particular disable_last_success is really important to reduce the load on database servers. I have implemented ticket #2734 with a little twist. Instead of adding local options in krb5.conf I create global options in the LDAP tree, so that all KDCs in the domain have the same configuration. The 2 new options can be set in ipaConfigString attribute of the cn=ipaConfig object under cn=etc,$SUFFIX These are: KDC:Disable Last Success KDC:Disable Lockout The first string if set will disable updating the krbLastSuccessfulAuth field in the service/user entry. The second one will prevent changing any of the Lockout related fields and will effectively disable lockout policies. I think we may want to set the first one by default in future. The last successful auth field is not very interesting in general and is cause for a lot of writes that pressure a lot the LDAP server and get replicated everywhere with a storm multiplier effect we'd like to avoid. The lockout one instead happen only when there are failed authentication attempt, this means it never happens when keytabs are used for example. And even with users it should happen rarely enough that traking lockouts by default make leaving these writes on by default is a good tradeoff. Note that simply setting the lockout policy to never lockout is *not* equivalent to setting KDC:Disable Lockout, as it does not prevent writes to the database. I've tested setting KDC:Disable Last Success and it effectively prevent MOD operation from showing up in the server access log. Any change to these configuration options requires a reconnection from the KDC to the LDAP server, the simplest way to cause that is to restart the KDC service. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-492-1-Add-support-for-disabling-KDC-writes.patch Type: text/x-patch Size: 6815 bytes Desc: not available URL: From pvoborni at redhat.com Mon May 28 11:44:07 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 28 May 2012 13:44:07 +0200 Subject: [Freeipa-devel] [PATCH] 150 Text widget's dirty state is changed on various input methods Message-ID: <4FC36507.6000305@redhat.com> on_value_changed event in textboxes and textareas was raised only on keyboard input. If user used different input method such as paste or browser undo and redo functions widget's on_value_changed event wasn't raised and so dirty state wasn't changed as well. This patch adds listener to text's and textarea's 'input' event. Input is a HTML 5 event which is raises on user initiated action. Some of user initiated actions : * Cut * Copy * Paste * Undo * Redo * Clear * Typing (like keyup) * Form AutoFill * User-invoked spellcheck corrections * Input from Input Method Editor It should be supported by all recent versions of major browsers. IE doesn't support it up to version 8. Listener for 'keyup' event was left in implementation for backward compatibility with older browsers. This may cause firing on_value_change twice but so far it shouldn't cause troubles. https://fedorahosted.org/freeipa/ticket/2647 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0150-Text-widget-s-dirty-state-is-changed-on-various-inpu.patch Type: text/x-patch Size: 3101 bytes Desc: not available URL: From mkosek at redhat.com Mon May 28 13:41:59 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 28 May 2012 15:41:59 +0200 Subject: [Freeipa-devel] [PATCH] 1018 enforce sizelimit when searching for permissions In-Reply-To: <4FBD2483.7090101@redhat.com> References: <4FB55B69.5080506@redhat.com> <1337327599.25787.10.camel@balmora.brq.redhat.com> <4FB65045.5060706@redhat.com> <4FB65A0C.5030306@redhat.com> <1337692317.17468.8.camel@balmora.brq.redhat.com> <4FBBFAF3.2070106@redhat.com> <4FBD2483.7090101@redhat.com> Message-ID: <1338212519.5538.5.camel@balmora.brq.redhat.com> On Wed, 2012-05-23 at 13:55 -0400, Rob Crittenden wrote: > Rob Crittenden wrote: > > Martin Kosek wrote: > >> On Fri, 2012-05-18 at 10:17 -0400, Rob Crittenden wrote: > >>> Rob Crittenden wrote: > >>>> Martin Kosek wrote: > >>>>> On Thu, 2012-05-17 at 16:11 -0400, Rob Crittenden wrote: > >>>>>> We do two searches when looking for permissions. One within the > >>>>>> permission object itself and a second in the ACIs. We weren't > >>>>>> enforcing > >>>>>> a sizelimit on either search. > >>>>>> > >>>>>> rob > >>>>> > >>>>> This returns the right result, but I don't think it is right with > >>>>> respect to "truncated" flag because of several reasons: > >>>>> > >>>>> 1) You manipulate and set "truncated" flag in post_callback but this > >>>>> won't affect the flag in the returned result because the new value is > >>>>> not propagated outside of the post_callback function. I.e. truncated > >>>>> flag will be set correctly only when it was raised during original > >>>>> permission_find. > >>>> > >>>> Truncated is still honored as expected. I even added a test case for > >>>> this. > >> > >> Yes, but it only works when the truncated flag is raised by the base > >> LDAP search, i.e. the search for permission objects (which is a case of > >> your unit test). If the search does not raise the flag and it is set > >> later in post callback, it is never propagated to the user. Please check > >> the attached (crappy) test that shows this issue: > >> > >> ====================================================================== > >> FAIL: test_permission[20]: permission_find: Search for permissions by > >> attr with a limit of 1 (truncated) > >> ---------------------------------------------------------------------- > >> ... > >> AssertionError: assert_deepequal: expected != got. > >> test_permission[20]: permission_find: Search for permissions by attr > >> with a limit of 1 (truncated) > >> expected = True > >> got = False > >> path = ('truncated',) > >> > >> I am not sure how to solve this right, we may need to add a new return > >> attribute (truncated) to all LDAPSearch post callbacks so that the post > >> callback can really modify it - something similar we already do with pre > >> callbacks which are able to change LDAP search filter, scope etc. > >> > >>>> > >>>>> 2) The part with "ind" is strange: > >>>>> > >>>>> + # enforce --sizelimit > >>>>> + if len(entries) == max_entries: > >>>>> + if ind + 1< len(results): > >>>>> + truncated = True > >>>>> + break > >>>>> > >>>>> I think it would be much easier to just do > >>>>> > >>>>> ... > >>>>> if (dn, permission) not in entries: > >>>>> if len(entries)< max_entries: > >>>>> entries.append((dn, permission)) > >>>>> else: > >>>>> truncated = True > >>>>> break > >>>>> > >>>>> Otherwise you would rise "truncated" even when the rest of "results" > >>>>> does not contain relevant entries that would have not been added > >>>>> anyway. > >>>> > >>>> Yes, that makes sense. > >>> > >>> And now updated patch. > >> > >> We can now also remove the "enumerate" part, "ind" is no longer needed. > >> > >> Martin > > > > You're right, I'd have sworn I tested that... > > > > The only solution is going to be to have the post_callback return > > truncated. This is going to be a rather intrusive change. > > > > rob > > Updated patch against master that adds a return value to post_callback. > > I also included Martin's test. It relies on the ordering of data in LDAP > but it is better than nothing right now. > > rob Yeah, returning a value in LDAPSearch post_callback is OK. I just see you missed post_callback in dnsrecord_find function, this makes "dnsrecord-find" command and also other DNS command in the unit tests crash. Martin From pvoborni at redhat.com Mon May 28 13:46:11 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 28 May 2012 15:46:11 +0200 Subject: [Freeipa-devel] [PATCH] 0050 Fail on unknown Command options In-Reply-To: <4FBF32D3.1060302@redhat.com> References: <4F8E97F6.2070703@redhat.com> <1336392614.29911.13.camel@balmora.brq.redhat.com> <4FABA16A.10802@redhat.com> <1336982456.4344.28.camel@balmora.brq.redhat.com> <1336984817.4344.43.camel@balmora.brq.redhat.com> <4FB0FEDB.4020806@redhat.com> <1337068555.10688.20.camel@balmora.brq.redhat.com> <4FB23F9B.10000@redhat.com> <1337155089.2963.10.camel@balmora.brq.redhat.com> <4FB3674A.1010402@redhat.com> <1337170298.2963.16.camel@balmora.brq.redhat.com> <4FBF32D3.1060302@redhat.com> Message-ID: <4FC381A3.7020000@redhat.com> On 05/25/2012 09:20 AM, Petr Vobornik wrote: > On 05/16/2012 02:11 PM, Martin Kosek wrote: >> On Wed, 2012-05-16 at 10:37 +0200, Petr Viktorin wrote: >>> On 05/16/2012 09:58 AM, Martin Kosek wrote: >>>> On Tue, 2012-05-15 at 13:35 +0200, Petr Viktorin wrote: >>>>> On 05/15/2012 09:55 AM, Martin Kosek wrote: >>>>>> On Mon, 2012-05-14 at 14:47 +0200, Petr Viktorin wrote: >>>>>>> The final part of rejecting unknown Command arguments: enable the >>>>>>> validation, add tests. >>>>>>> Also fix up things that were changed since the previous patches. >>>>>>> >>>>>>> https://fedorahosted.org/freeipa/ticket/2509 8><------------------ >>> >>> Attaching a rebased patch. >>> >> >> Yup, this one is fine. Now, I did not find issues in the patch itself, >> tests are clean. >> >> However, thanks to this new check I found issues in Web UI (automember, >> selfservice, delegation screen) which use illegal options and which >> should be fixed before we push your patch: >> >> https://fedorahosted.org/freeipa/ticket/2760 >> >> Martin >> > > I found an issue in automountmap_add_indirect. It complains that 'key' > is unknown option. I found another options which were functional and now it complains: * hbacsvcgroup_find: no_hbacsvc * hbacsvc_find: not_in_hbacsvcgroup * same issue in sudo commands and sudo command groups. I didn't check all relationships, so it may be broken elsewhere as well. -- Petr Vobornik From mkosek at redhat.com Mon May 28 14:05:42 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 28 May 2012 16:05:42 +0200 Subject: [Freeipa-devel] [PATCH] 0052 Fix the pwpolicy_find post_callback In-Reply-To: <4FB65F7E.8090509@redhat.com> References: <4FB64C37.1080304@redhat.com> <4FB65694.8000505@redhat.com> <1337351066.25787.51.camel@balmora.brq.redhat.com> <4FB65F7E.8090509@redhat.com> Message-ID: <1338213942.5538.7.camel@balmora.brq.redhat.com> On Fri, 2012-05-18 at 16:41 +0200, Petr Viktorin wrote: > On 05/18/2012 04:24 PM, Martin Kosek wrote: > > On Fri, 2012-05-18 at 10:03 -0400, Rob Crittenden wrote: > >> Petr Viktorin wrote: > >>> The pwpolicy-find command had a bad condition that made IPA convert > >>> lifetime values for output only if those values were *not* being retrieved. > >>> With this patch, the conversion routine is called unconditionally (it > >>> has its own check for presence of the attributes). > >>> > >>> https://fedorahosted.org/freeipa/ticket/2726 > >>> > >>> Also, have the sorting code use a key function instead of a cmp; this is > >>> a Python best practice. > >>> Improve a comment -- policies with higher priority are and always were > >>> at the *end* of the list, not the beginning. (I'm not sure if we want it > >>> this way?) > >>> > >>> Regression test included. > >> > >> I haven't checked this in detail yet but the comment about priority may > >> be wrong. The priority is such that the lower the number the higher the > >> priority. I think that is what it meant. > >> > >> rob > >> > > > > Yup, the comment was right. Lower numbers, i.e. password policies with > > higher priority, were sorted in the top. > > > > Martin > > > > Ah, I see. Made it clear so the next person is not confused. > It is OK now. The solution with key instead of cmp is indeed more readable. ACK. Pushed to master. Martin From mkosek at redhat.com Mon May 28 14:16:06 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 28 May 2012 16:16:06 +0200 Subject: [Freeipa-devel] [PATCH] 0050 Fail on unknown Command options In-Reply-To: <4FC381A3.7020000@redhat.com> References: <4F8E97F6.2070703@redhat.com> <1336392614.29911.13.camel@balmora.brq.redhat.com> <4FABA16A.10802@redhat.com> <1336982456.4344.28.camel@balmora.brq.redhat.com> <1336984817.4344.43.camel@balmora.brq.redhat.com> <4FB0FEDB.4020806@redhat.com> <1337068555.10688.20.camel@balmora.brq.redhat.com> <4FB23F9B.10000@redhat.com> <1337155089.2963.10.camel@balmora.brq.redhat.com> <4FB3674A.1010402@redhat.com> <1337170298.2963.16.camel@balmora.brq.redhat.com> <4FBF32D3.1060302@redhat.com> <4FC381A3.7020000@redhat.com> Message-ID: <1338214566.5538.11.camel@balmora.brq.redhat.com> On Mon, 2012-05-28 at 15:46 +0200, Petr Vobornik wrote: > On 05/25/2012 09:20 AM, Petr Vobornik wrote: > > On 05/16/2012 02:11 PM, Martin Kosek wrote: > >> On Wed, 2012-05-16 at 10:37 +0200, Petr Viktorin wrote: > >>> On 05/16/2012 09:58 AM, Martin Kosek wrote: > >>>> On Tue, 2012-05-15 at 13:35 +0200, Petr Viktorin wrote: > >>>>> On 05/15/2012 09:55 AM, Martin Kosek wrote: > >>>>>> On Mon, 2012-05-14 at 14:47 +0200, Petr Viktorin wrote: > >>>>>>> The final part of rejecting unknown Command arguments: enable the > >>>>>>> validation, add tests. > >>>>>>> Also fix up things that were changed since the previous patches. > >>>>>>> > >>>>>>> https://fedorahosted.org/freeipa/ticket/2509 > > 8><------------------ > > >>> > >>> Attaching a rebased patch. > >>> > >> > >> Yup, this one is fine. Now, I did not find issues in the patch itself, > >> tests are clean. > >> > >> However, thanks to this new check I found issues in Web UI (automember, > >> selfservice, delegation screen) which use illegal options and which > >> should be fixed before we push your patch: > >> > >> https://fedorahosted.org/freeipa/ticket/2760 > >> > >> Martin > >> > > > > I found an issue in automountmap_add_indirect. It complains that 'key' > > is unknown option. > > I found another options which were functional and now it complains: > * hbacsvcgroup_find: no_hbacsvc > * hbacsvc_find: not_in_hbacsvcgroup > * same issue in sudo commands and sudo command groups. > > I didn't check all relationships, so it may be broken elsewhere as well. > I don't think this is an error on server side - it never had filter options like these in the modules you referenced (though we may add them as an RFE when needed). When you pass these options in the UI to the server side, its just NOOP - or an error when Petr's patch is applied. Martin From mkosek at redhat.com Mon May 28 15:11:19 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 28 May 2012 17:11:19 +0200 Subject: [Freeipa-devel] [PATCH] 22 Always set ipa_hostname for sssd.conf In-Reply-To: <4F86DDEF.10707@redhat.com> References: <4F86DDEF.10707@redhat.com> Message-ID: <1338217879.5538.13.camel@balmora.brq.redhat.com> On Thu, 2012-04-12 at 15:51 +0200, Ondrej Hamada wrote: > https://fedorahosted.org/freeipa/ticket/2527 > > ipa-client-install will always set ipa_hostname for sssd.conf in order > to prevent the client from getting into weird state. ACK. Pushed to master. Martin From mkosek at redhat.com Mon May 28 15:24:38 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 28 May 2012 17:24:38 +0200 Subject: [Freeipa-devel] [PATCH] 266 Reset krbtpolicy when a unit test is finished Message-ID: <1338218678.5538.15.camel@balmora.brq.redhat.com> Pushed to master under the one-liner rule. --- Kerberos ticket maximum life was being set to 1 hour which then affected lifetime of Kerberos tickets returned by IPA server under the test. Make sure that the policy is reset before and after the unit test to keep the IPA server settings clean and not to disrupt development environment. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-266-reset-krbtpolicy-when-a-unit-test-is-finished.patch Type: text/x-patch Size: 1087 bytes Desc: not available URL: From mkosek at redhat.com Tue May 29 07:26:01 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 May 2012 09:26:01 +0200 Subject: [Freeipa-devel] [PATCH] 0053 Disallow setattr on no_update/no_create params In-Reply-To: <4FBA2DDC.7010607@redhat.com> References: <4FBA2DDC.7010607@redhat.com> Message-ID: <1338276361.30643.9.camel@balmora.brq.redhat.com> On Mon, 2012-05-21 at 13:58 +0200, Petr Viktorin wrote: > Only use no_create/no_update for things we really don't want the user to > change (even through setattr). This is stuff like ipacertificatesubjectbase. > Make --{set,add,del}attr refuse to modify these params. > > For things we just don't advertise in the because there's a different > way to do change them, there is the "no_option" flag (undocumented > before this patch). This only causes the option to be hidden from the > CLI; XML-RPC will still happily take it (and it will appear in API.txt). > Use this for ipaenabledflag, nsacconuntlock, and externalhost. > > > https://fedorahosted.org/freeipa/ticket/2580 Yeah, I think this approach is OK, everything worked fine for me. ACK. Pushed to master. Martin From mkosek at redhat.com Tue May 29 08:19:56 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 May 2012 10:19:56 +0200 Subject: [Freeipa-devel] [PATCH] 1019 require policycoreutils if SELinux is enabled In-Reply-To: <4FB67069.1040901@redhat.com> References: <4FB67069.1040901@redhat.com> Message-ID: <1338279596.30643.15.camel@balmora.brq.redhat.com> On Fri, 2012-05-18 at 11:53 -0400, Rob Crittenden wrote: > We don't have an explicit requires on the policycoreutils package in the > client because SELinux is not required (just recommended). > > SELinux can be enabled without this package so check for that condition > and don't allow installation if it is the case. The resulting install > will be rather broken. > > Also check on the server when installing. This should never happen but > in theory it could do the server install then fail in the client because > of this. > > rob This works fine. I am just thinking if we should not rather use paths in /usr/ for the check if a binary exists, i.e. check for /usr/sbin/restorecon instead of /sbin/restorecon on Fedora. If we don't do this we need to be sure that the /sbin -> /usr/sbin symlink created during UsrMove will stay on the system. Martin From mkosek at redhat.com Tue May 29 10:46:38 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 May 2012 12:46:38 +0200 Subject: [Freeipa-devel] [PATCH] 0040 Move install script error handling to a common function In-Reply-To: <4FBB9896.7020908@redhat.com> References: <4F951EB7.9050908@redhat.com> <4F956FB7.3040000@redhat.com> <4FBB9896.7020908@redhat.com> Message-ID: <1338288398.30643.35.camel@balmora.brq.redhat.com> On Tue, 2012-05-22 at 15:45 +0200, Petr Viktorin wrote: > On 2012-04-23 17:05, John Dennis wrote: > > On 04/23/2012 05:19 AM, Petr Viktorin wrote: > >> This fixes https://fedorahosted.org/freeipa/ticket/2071 (Add final debug > >> message in installers). > >> > >> I submitted an earlier version of this patch before (0014), but it was > >> too much to include in 2.2. Hopefully now there's more space for > >> restructuring. I think it's better to start a new thread with this > >> approach. > >> > >> The try/except blocks at the end of installers/management scripts are > >> replaced by a call to a common function, which includes the final > >> message. > >> For each specific error, the error handlers in all scripts was almost > >> the same, but each script handled a different selection of errors. > >> Instead of having this copy/pasted code (with subtle differences > >> creeping in over time), this patch consolidates it all in one place. > > > > I like this approach much better than the earlier patch, great, thanks. > > I'm a big fan of calling into common code instead of copying code to my > > mind the refactoring to utilize common code is great approach. I also > > like the fact the logging configuration is not modified after it's > > established. > > > > At some point we may want to revist how the log messages are generated. > > For example should all communication to the console pass through the > > console handler? Is there a logger established for the script? Should > > the format of messages emitted to the console be altered? Should all > > command line utilities accept the both the verbose and debug flag? Etc. > > But for now this is fantastic start in the right direction. > > > > I have not installed and exercised the patch so I can't comment on any > > runtime time issues that might be present, but from code inspection only > > it has my ACK. > > > > > Thanks John! > Yes, this is just a start. > > > Patch rebased to curent master > The patch needs another rebase. Besides that, the approach is fine (and removes a lot of redundant code) but I found several issues with the patches: 1) ipa-server-install: Fails when option parser error is raised: # ipa-server-install -p foo Usage: ipa-server-install [options] ipa-server-install: error: DS admin password: Password must be at least 8 characters long Traceback (most recent call last): File "/sbin/ipa-server-install", line 1131, in if not success and installation_cleanup: NameError: name 'success' is not defined 2) BadSyntax error is not caught properly: # ipa-ldap-updater Directory Manager password: ipa : INFO PRE_UPDATE ipa : INFO Parsing file /usr/share/ipa/updates/10-60basev2.update ipa : INFO Parsing file /usr/share/ipa/updates/10-60basev3.update ipa : INFO Parsing file /usr/share/ipa/updates/10-RFC2307bis.update ipa : INFO Parsing file /usr/share/ipa/updates/10-RFC4876.update ipa : INFO Parsing file /usr/share/ipa/updates/10-bind-schema.update ipa : INFO Parsing file /usr/share/ipa/updates/10-config.update ipa : INFO Parsing file /usr/share/ipa/updates/10-schema_compat.update ipa : INFO Parsing file /usr/share/ipa/updates/10-selinuxusermap.update ipa : INFO Parsing file /usr/share/ipa/updates/10-ssh.update ipa : INFO File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 695, in run_script return_value = main_function() File "/sbin/ipa-ldap-updater", line 142, in main modified = ld.update(files) File "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", line 830, in update (all_updates, dn_list) = self.parse_update_file(data, all_updates, dn_list) File "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", line 284, in parse_update_file raise BadSyntax, "dn is not defined in the update" ipa : INFO The ipa-ldap-updater command failed, exception: BadSyntax: 'dn is not defined in the update' There is a syntax error in this update file: dn is not defined in the update 3) Inappropriate debug message in ipa-replica-manage: # ipa-replica-manage list Directory Manager password: vm-125.idm.lab.bos.redhat.com: master ipa: INFO: The ipa-replica-manage command was successful <<< 4) ipa-ca-install emits failed_message when option error is detected: # ipa-ca-install Usage: ipa-ca-install [options] REPLICA_FILE ipa-ca-install: error: you must provide a file generated by ipa-replica-prepare >>> Your system may be partly configured. >>> Run /usr/sbin/ipa-server-install --uninstall to clean up. 5) ipa-cs-replica-manage - the same issue as with 3) 6) ipa-replica-prepare - the same issue as with 3) 7) ipa-replica-prepare - the same issue as with 4) 8) ldap.SERVER_DOWN is not caught: # ipactl stop Stopping HTTP Service Stopping MEMCACHE Service Stopping KPASSWD Service Stopping KDC Service Stopping Directory Service # ipa-replica-manage list ipa: INFO: File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 695, in run_script return_value = main_function() ... ipa: INFO: The ipa-replica-manage command failed, exception: SERVER_DOWN: {'desc': "Can't contact LDAP server"} Can't contact LDAP server IMO we should add a handler both for ldap.SERVER_DOWN and also a general exception ldap.LDAPError to catch other potential LDAP errors. 9) KeyboardInterrupt is not processed properly in ipa-replica-manage: # ipa-replica-manage list ^Cipa: INFO: File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 695, in run_script return_value = main_function() File "/sbin/ipa-replica-manage", line 477, in main api.finalize() ... File "/usr/lib/python2.7/site-packages/ipalib/parameters.py", line 467, in __init__ type(kind) is tuple and not isinstance(value, kind) ipa: INFO: The ipa-replica-manage command failed, exception: KeyboardInterrupt: Cancelled. I think it would be nicer if such traceback would only be printed either to log (when the command has a log file) or only when --debug option is passed (some of our commands miss that option). 10) Traceback in ipa-csreplica-manage: # ipa-csreplica-manage list Directory Manager password: ipa: INFO: File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 695, in run_script return_value = main_function() File "/sbin/ipa-csreplica-manage", line 415, in main list_replicas(realm, host, replica, dirman_passwd, options.verbose) File "/sbin/ipa-csreplica-manage", line 180, in list_replicas sys.exit("Failed to get data from '%s': %s" % (host, convert_error(e))) ipa: INFO: The ipa-replica-manage command failed, exception: SystemExit: Failed to get data from 'vm-057.idm.lab.bos.redhat.com': Can't contact LDAP server Failed to get data from 'vm-057.idm.lab.bos.redhat.com': Can't contact LDAP server The command also has a wrong operation_name. This is what happens when a script name is hard-coded and it is not detected automatically, e.g. from __file__. Martin From mkosek at redhat.com Tue May 29 12:04:35 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 May 2012 14:04:35 +0200 Subject: [Freeipa-devel] [PATCH] 1014 configurable service timeout In-Reply-To: <4FBE560E.1080108@redhat.com> References: <4FB3F267.2070104@redhat.com> <4FB6AAF6.9080800@redhat.com> <4FBE2654.4030207@redhat.com> <4FBE560E.1080108@redhat.com> Message-ID: <1338293075.30643.52.camel@balmora.brq.redhat.com> On Thu, 2012-05-24 at 11:38 -0400, Rob Crittenden wrote: > Petr Viktorin wrote: > > On 05/18/2012 10:03 PM, Rob Crittenden wrote: > >> Rob Crittenden wrote: > >>> A hardcoded timeout was used in ipactl for service restarts, set rather > >>> low. A separate timeout was hardcoded into the installer. > >>> > >>> I centralized them into a single timeout, configurable in the standard > >>> way in /etc/ipa/*.conf. > >>> > >>> On install it will always default to 120 seconds and remain there unless > >>> changed in default.conf (not replicated either). > >>> > >>> I tested this on systemd systems and sysV systems and it works ok for > >>> me. You'll also want to double-check that this works when other 389-ds > >>> instances are installed. > >>> > >>> Getting the naming of instances right was a bit tricky. > >> > >> Noticed a problem on upgrades and fixed that. Updated patch attached. > >> > >> rob > >> > >> > > > > Please rebase the patch onto current master. > > > > > > Done This is a good start. I just found few places where I found that the remaining wait function calls are redundant: 1) install/tools/ipactl: if lurl.urlscheme == 'ldapi': - wait_for_open_socket(lurl.hostport, timeout=6) + wait_for_open_socket(lurl.hostport, timeout=api.env.startup_timeout) else: (host,port) = lurl.hostport.split(':') - wait_for_open_ports(host, [int(port)], timeout=6) + wait_for_open_ports(host, [int(port)], timeout=api.env.startup_timeout) Aren't these calls redundant? We already wait for ports when dirsrv is started (dirsrv.start()) or restarted (dirsrv.restart()). 2) ipaserver/install/replication.py: - installutils.wait_for_open_ports('localhost', [389, 636], 300) + ipautil.wait_for_open_ports('localhost', [389, 636], 300) Isn't this now redundant? Port check should be done in service restart. 3) ipaserver/install/plugins/updateclient.py: - installutils.wait_for_open_socket(socket_name) + wait_for_open_socket(socket_name) Also seems redundant, dirsrv should be already up as it was restarted via our Service framework. Though we only check for ports in the Service framework, I wonder if this is enough and we can be sure that when ports are up, the LDAPI socket is also up. Martin From mkosek at redhat.com Tue May 29 13:11:45 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 May 2012 15:11:45 +0200 Subject: [Freeipa-devel] [PATCH] 267 Allow relative DNS name in NS validator Message-ID: <1338297105.30643.54.camel@balmora.brq.redhat.com> Precallback validator was failing when a zone-relative name was used as a NS record (for example record "ns" in a zone "example.com"). However, this is valid in BIND and we should allow it as well. Imports in dns module had to be switched to absolute imports (available from Python 2.5) to deal with a conflict of IPA dns module and dnspython module. https://fedorahosted.org/freeipa/ticket/2630 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-267-allow-relative-dns-name-in-ns-validator.patch Type: text/x-patch Size: 2804 bytes Desc: not available URL: From rcritten at redhat.com Tue May 29 13:47:16 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 29 May 2012 09:47:16 -0400 Subject: [Freeipa-devel] [PATCH] 266 Reset krbtpolicy when a unit test is finished In-Reply-To: <1338218678.5538.15.camel@balmora.brq.redhat.com> References: <1338218678.5538.15.camel@balmora.brq.redhat.com> Message-ID: <4FC4D364.9080301@redhat.com> Martin Kosek wrote: > Pushed to master under the one-liner rule. > > --- > > Kerberos ticket maximum life was being set to 1 hour which then > affected lifetime of Kerberos tickets returned by IPA server under > the test. > > Make sure that the policy is reset before and after the unit test to > keep the IPA server settings clean and not to disrupt development > environment. ACK, thanks for fixing this, it was quite annoying. rob From mkosek at redhat.com Tue May 29 13:53:44 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 May 2012 15:53:44 +0200 Subject: [Freeipa-devel] [PATCH] 266 Reset krbtpolicy when a unit test is finished In-Reply-To: <4FC4D364.9080301@redhat.com> References: <1338218678.5538.15.camel@balmora.brq.redhat.com> <4FC4D364.9080301@redhat.com> Message-ID: <1338299624.30643.55.camel@balmora.brq.redhat.com> On Tue, 2012-05-29 at 09:47 -0400, Rob Crittenden wrote: > Martin Kosek wrote: > > Pushed to master under the one-liner rule. > > > > --- > > > > Kerberos ticket maximum life was being set to 1 hour which then > > affected lifetime of Kerberos tickets returned by IPA server under > > the test. > > > > Make sure that the policy is reset before and after the unit test to > > keep the IPA server settings clean and not to disrupt development > > environment. > > ACK, thanks for fixing this, it was quite annoying. > > rob > I already pushed that as one-liner change as I wrote above. But you are right, it was indeed very annoying. Martin From mkosek at redhat.com Tue May 29 14:01:37 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 May 2012 16:01:37 +0200 Subject: [Freeipa-devel] [PATCH] 268 Add rename option for DNS records Message-ID: <1338300097.30643.56.camel@balmora.brq.redhat.com> This option will make renaming DNS records much easier. Add a unit test for this new functionality. https://fedorahosted.org/freeipa/ticket/2600 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-268-add-rename-option-for-dns-records.patch Type: text/x-patch Size: 6557 bytes Desc: not available URL: From rcritten at redhat.com Tue May 29 14:31:23 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 29 May 2012 10:31:23 -0400 Subject: [Freeipa-devel] [PATCH] 1014 configurable service timeout In-Reply-To: <1338293075.30643.52.camel@balmora.brq.redhat.com> References: <4FB3F267.2070104@redhat.com> <4FB6AAF6.9080800@redhat.com> <4FBE2654.4030207@redhat.com> <4FBE560E.1080108@redhat.com> <1338293075.30643.52.camel@balmora.brq.redhat.com> Message-ID: <4FC4DDBB.7070902@redhat.com> Martin Kosek wrote: > On Thu, 2012-05-24 at 11:38 -0400, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> On 05/18/2012 10:03 PM, Rob Crittenden wrote: >>>> Rob Crittenden wrote: >>>>> A hardcoded timeout was used in ipactl for service restarts, set rather >>>>> low. A separate timeout was hardcoded into the installer. >>>>> >>>>> I centralized them into a single timeout, configurable in the standard >>>>> way in /etc/ipa/*.conf. >>>>> >>>>> On install it will always default to 120 seconds and remain there unless >>>>> changed in default.conf (not replicated either). >>>>> >>>>> I tested this on systemd systems and sysV systems and it works ok for >>>>> me. You'll also want to double-check that this works when other 389-ds >>>>> instances are installed. >>>>> >>>>> Getting the naming of instances right was a bit tricky. >>>> >>>> Noticed a problem on upgrades and fixed that. Updated patch attached. >>>> >>>> rob >>>> >>>> >>> >>> Please rebase the patch onto current master. >>> >>> >> >> Done > > This is a good start. I just found few places where I found that the > remaining wait function calls are redundant: > > 1) install/tools/ipactl: > > if lurl.urlscheme == 'ldapi': > - wait_for_open_socket(lurl.hostport, timeout=6) > + wait_for_open_socket(lurl.hostport, > timeout=api.env.startup_timeout) > else: > (host,port) = lurl.hostport.split(':') > - wait_for_open_ports(host, [int(port)], timeout=6) > + wait_for_open_ports(host, [int(port)], > timeout=api.env.startup_timeout) > > Aren't these calls redundant? We already wait for ports when dirsrv is > started (dirsrv.start()) or restarted (dirsrv.restart()). It is redundant in some cases but there are some calls we make where this is used to determine the availability of the service. This call is needed. > 2) ipaserver/install/replication.py: > - installutils.wait_for_open_ports('localhost', [389, 636], 300) > + ipautil.wait_for_open_ports('localhost', [389, 636], 300) > > Isn't this now redundant? Port check should be done in service restart. Yes, looks like this call can go. > 3) ipaserver/install/plugins/updateclient.py: > > - installutils.wait_for_open_socket(socket_name) > + wait_for_open_socket(socket_name) > > Also seems redundant, dirsrv should be already up as it was restarted > via our Service framework. Though we only check for ports in the Service > framework, I wonder if this is enough and we can be sure that when ports > are up, the LDAPI socket is also up. No, sockets and ports are separate, particularly when updating. In fact, we disable the ports so a wait_for_port() will always fail which is why I added the wait flag. This may be a case I missed with upgrades. Let me test upgrades again... rob From jcholast at redhat.com Tue May 29 14:40:32 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 29 May 2012 16:40:32 +0200 Subject: [Freeipa-devel] [PATCH] 268 Add rename option for DNS records In-Reply-To: <1338300097.30643.56.camel@balmora.brq.redhat.com> References: <1338300097.30643.56.camel@balmora.brq.redhat.com> Message-ID: <4FC4DFE0.5010508@redhat.com> On 29.5.2012 16:01, Martin Kosek wrote: > This option will make renaming DNS records much easier. > Add a unit test for this new functionality. > > https://fedorahosted.org/freeipa/ticket/2600 > I wonder, how hard would it be to modify the patch to allow --rename on all objects, as requested in ? Honza -- Jan Cholasta From mkosek at redhat.com Tue May 29 14:59:21 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 29 May 2012 16:59:21 +0200 Subject: [Freeipa-devel] [PATCH] 268 Add rename option for DNS records In-Reply-To: <4FC4DFE0.5010508@redhat.com> References: <1338300097.30643.56.camel@balmora.brq.redhat.com> <4FC4DFE0.5010508@redhat.com> Message-ID: <1338303561.10122.2.camel@balmora.brq.redhat.com> On Tue, 2012-05-29 at 16:40 +0200, Jan Cholasta wrote: > On 29.5.2012 16:01, Martin Kosek wrote: > > This option will make renaming DNS records much easier. > > Add a unit test for this new functionality. > > > > https://fedorahosted.org/freeipa/ticket/2600 > > > > I wonder, how hard would it be to modify the patch to allow --rename on > all objects, as requested in ? > > Honza > Not sure, but it would require a lot of testing. As you can see in a case of dnsrecord object, it may need more changes than just adding rdn_is_primary_key=True to LDAPObjects. I assume there is a reason why we deferred this effort for now. Martin From pspacek at redhat.com Tue May 29 15:16:06 2012 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 29 May 2012 17:16:06 +0200 Subject: [Freeipa-devel] routing requests to local servers - DNS SRV [discussion needed] In-Reply-To: <1337955053.16840.286.camel@willson.li.ssimo.org> References: <4FBE6AC9.4020703@redhat.com> <1337955053.16840.286.camel@willson.li.ssimo.org> Message-ID: <4FC4E836.6080804@redhat.com> Hello, for clarity: I'm not going to implement it (now). There are another features on the table. I'm trying to find simplest solution/workaround, because several people asked for this feature and I think it is quite important. (Besides load-balancing purpose it can be handy for environments with NAT in place - as Amazon EC2.) Discussion about major changes should be read as "design for far future". On 05/25/2012 04:10 PM, Simo Sorce wrote: > On Thu, 2012-05-24 at 19:07 +0200, Petr Spacek wrote: >> Hello, >> >> some time ago there was a request for DNS to support "routing requests to >> local servers". Any opinions if/when do it and proposals how to do it are more >> than welcome. My best knowledge about this problem follows: >> >> >> This request actually means "differentiate answer to DNS query on client's IP >> address basics". >> Relevant thread on ipa-users-list: >> https://www.redhat.com/archives/freeipa-users/2012-April/msg00070.html >> >> First question is: Do we want to implement it? >> (IMHO it is very important for large-scale deployments.) > > I am not really sure I like it, as it makes things quite difficult to > handle. > > Please think how we are going to operate if you ahve a view dfined and a > client does a dyn DNS update. Yes, it's a bit difficult. I propose this approach: - add new record -> add it to "base" (shared) part - delete record -> delete "visible" record: i.e. delete override record if it is present, delete "base" record otherwise. It allows clients to update own records as usual and changes will be visible to the whole world. One potential problem are update scripts from monitoring systems. Monitoring system can test availability of server and dynamically adjust DNS records, if server status has changed. I'm not sure if some monitoring systems supports this feature directly, I always used my own scripts. These scripts have to be changed from nsupdate/another tool to ldapmodify, but I think it is acceptable. ...snip ... >> Adam and I discussed possible approaches. We agreed on "stackable" approach: >> - Store whole original DNS tree in one subtree, let say "base". >> - Create "override" subtrees for each "locality" = set of customized records. >> Shared records are only in "base". During DNS query processing "override" >> subtree is searched first. If record does not exist in "override" subtree, >> search will continue in "base" subtree. > > Yes, this is my first thought too. > >> Second question is how to realize these "overrides": >> - Let Directory Server to do the work. If DS supports this, it is mostly done. > Why do you need dirsrv to do anything special ? I can save bugs in the new code (and time) if dirsrv can do it. I played with 389 last days and unfortunately I can't find support for this use case. (I'm pretty sure that OpenLDAP can support this scenario - that was a reason why I started to examine this possibility in 389.) > The simple way is to do subtree searches, if you get back more than one > result for a specific name then you know you have views and proceed to > filter out the one you want. The bonus here is that you can cache all > replies if you keep different caches per view. > > The alternative is to add a 'viewname' or something in the filter, but > then you need to do 2 searches and fallbacks. This sounds more > expensive. > >> It do not require any change in bind-dyndb-ldap code. All merges/overrides >> will be done on Directory server. > > Given we do persistent searches and we also do some caching in > bind-dyndb-ldap we almost certainly do not want to 'fool' it by > returning different values from DS w/o bind-dyndb-ldap knowledge. I probably missed something, I don't see the problem. One bind-dyndb-ldap instance is run for each "view" separately. Each instance has own caches and other data structures. They don't know about each other. >> Only /etc/named.conf has to be adjusted with >> right search base and "view" clause. >> >> I asked on 389 mailing list and I'm waiting for reply: >> http://lists.fedoraproject.org/pipermail/389-users/2012-May/014653.html >> >> >> - Another approach is to add support directly to bind-dyndb-ldap, but it is >> not so nice solution. > > Why ? > >> In that case both LDAP search bases have to be defined >> in /etc/named.conf. For each DNS query is necessary to search "override" base >> first. If required record is not present then is necessary to search through >> "base" subtree. > > No, you do not need to do multiple searches, you just need to filter the > results by view. You are right, if we change the way how DNS data are stored in LDAP (a bit). Mentioned idea was to create new subtree ("cn=dns-viewname") and re-use existing driver and all tools for managing it with minimal changes. > It would make it very easy to manage if we added a 'dnsView' attribute > to overriding entries in the subtree. This would allow us to define a > new overriding entry and even assign it to multiple views if the > 'dnsView' attribute is multivalued. > >> With persistent search it should be better, because all records are in memory, >> but basic principle remains same. > > psearch is one of the reasons we might want a dnsView attribute base > approach instead of playing games with DS. > I would also prefer to avoid adding more code to DS where the bind > plugin can easily handle this data itself. Another reason is that I > really want this plugin code to be generic and usable with other LDAP > servers and not be tied so strictly to 389ds. > > Simo. I'm not talking about new DS plugin, I wanted to use existing features. IMHO OpenLDAP with rewrite/remap and rwm overlays can do all the work for us. Unfortunately 389 is poor in this case (or I can't find it in manual). Merging with another thread: On 05/25/2012 09:20 PM, Simo Sorce wrote: > On Fri, 2012-05-25 at 15:52 +0200, Petr Spacek wrote: >> > On 05/24/2012 08:00 PM, Dmitri Pal wrote: >>> > > On 05/24/2012 01:07 PM, Petr Spacek wrote: >>> > > >>> > > May be I am oversimplifying things but it seems that it would make >>> > > sense >>> > > to just have an extra multi-value attribute added to the DNS schema. >>> > > This attribute would contain a value that would allow the entry to be >>> > > included into the view. This way the base is the same and what the view >>> > > exposes is just a filter. The default view would have a filter that >>> > > matches all entries like now. >>> > > >>> > > >>> > > I assume that DNS server makes it decision based on the IP so it has a >>> > > call to get a "view" data. The ldap driver can return a filter. The DNS >>> > > server them makes a call using this filter to get all the records that >>> > > apply. I know it is too ldap centric. There is some abstraction in DNS >>> > > server and we do not want to change it. But the point is there are >>> > > probably two calls in the DNS server (have not actually confirmed): >>> > > lookup data X by IP to understand what the view is. >>> > > Pass data X to get the actual DNS entries. >>> > > >>> > > I am saying that X can be filter. >>> > > >>> > > Thoughts? >> > >> > Special attribute sounds like a good idea. It is not realizable >> > directly, but >> > I will explore variants with some "view" attribute. >> > >> > Current DNS "name" (name can potentially contain multiple records) >> > structure >> > is following: >> > >> > dn: idnsname=_kerberos._tcp,idnsname=e.localnet,cn=dns,dc=e,dc=org >> > objectClass: idnsrecord >> > objectClass: top >> > idnsName: _kerberos._tcp >> > sRVRecord: 0 100 88 unused-4-107 >> > >> > DNS name is part of DN. It is not possible to have more objects with >> > same DNS >> > name and different attributes. This problem lead me to "stackable" >> > approach. > Yes, and we can also use multiple attributes in the same tree, although > for clarity I probably prefer the subtree approach. > > So a few options: > > 1. all in the same subtree: > # Normal object > dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org > objectClass: idnsrecord > objectClass: top > idnsName: bar > aRecord: 192.168.12.34 > dNSTTL: 1200 > > # Object belongin to the 'DMZ' view > dn:cn=DMZ-bar,idnsname=foo.org,cn=dns,dc=foo,dc=org > objectClass: idnsrecord > objectClass: top > objectClass: nsContainer > cn: DMZ-bar > idnsName: bar > aRecord: 5.6.7.8 > dNSTTL: 3600 > idnsView: DMZ > > > NOTE: I had to add nsContainer here in order to give the object a way to > have a unique name by using the CN attribute. I am not very fond of this > arrangement though. It is also ugly to parse out using a LDAP browser. > It make one thing simpler in that using multiple values for dnsView you > can assign the same entry to multiple views. > > 2. using per view subtrees Generally - I like this idea. > # Normal object > dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org > objectClass: idnsrecord > objectClass: top > idnsName: bar > aRecord: 192.168.12.34 > dNSTTL: 1200 I prefer to create "_default" view for normal records (as BIND does). e.g. dn: idnsname=bar,cn=_default,idnsname=foo.org,cn=dns,dc=foo,dc=org It allows to treat "normal" and override records in the same way. Also it allows to optimize query with extensibleMatch filter. E.g. plugin is configured to serve records for "view1": Filter can be (|(cn:dn:=view1)(cn:dn:=_default)) so LDAP server will not return unnecessary records for view2, view3, view4 ... Another problem to solve is how to modify SOA record/hide zone/add new zone to view. With this view-zone-dnssubtree ordering you cannot modify SOA record or hide the zone from certain views. With ordering zone-view-dnssubtree situation is better. We still can run subtree query in 'cn=dns' and as bonus it is possible to define zone only in some views or create modified version of zone's SOA record in another views. Problem is how to *not* override zone SOA record. One (not very nice) approach: We can create extensibleObject and define container only with idnsName attribute (necessary for RDN construction). "Real" zones from "fake" ones can be distinguished by objectClass. Records are leaves under this zone record as usual. > # Object belongin to the 'DMZ' view > dn:idnsname=bar,cn=DMZ,idnsname=foo.org,cn=dns,dc=foo,dc=org > objectClass: idnsrecord > objectClass: top > idnsName: bar > aRecord: 5.6.7.8 > dNSTTL: 3600 > > > NOTE: I prefer this method as it makes things a lot easier to manage and > view through an LDAP broiwser, however it makes sharing entries between > multiple views a bit awkward. I like this idea. What about LDAP aliases? (I never used them, so I don't know.) It is not very nice, but for limited amount of "override" records it can be sufficient. > 3. using only one 'views' subtree pr zone and dnsView to discrimnate > > # Normal object > dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org > objectClass: idnsrecord > objectClass: top > idnsName: bar > aRecord: 192.168.12.34 > dNSTTL: 1200 > > # Object belongin to the 'DMZ' view > dn:idnsUniqueID=F6A1245-bar,cn=views,idnsname=foo.org,cn=dns,dc=foo,dc=org > objectClass: idnsrecord > objectClass: top > idnsUniqueID: F6A1245-bar > idnsName: bar > aRecord: 5.6.7.8 > dNSTTL: 3600 > idnsView: DMZ > idnsView: VPN > > > NOTE: here I added also a idnsUniqueID as a way to have unique names so > we can have multiple entries for the same record. This is so that you > can have 3 different entries for the same record belonging to 3 > different views. The reason why I added the actual name after a random > id is that this way it is simpler to recognize what it is when looking > at an ldap browser w/o having to read the actual object attributes, it > also make collisions a lot less likely and so it allows to keep the > random part smaller (and thus more readable). > Also note that I've put 2 values in idnsView, meaning that this record > belongs to 2 separate views. This allows compact representation when > multiple views want to redefine some records in the same way (an dothers > in a different way, thus why 2 separate views)/ I also prefer "way 2", as I said above. ... snip ... Thanks for your time. Petr^2 Spacek From jcholast at redhat.com Tue May 29 15:21:02 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 29 May 2012 17:21:02 +0200 Subject: [Freeipa-devel] [PATCH] 79 SSH configuration fixes In-Reply-To: <1337962194.5899.37.camel@balmora.brq.redhat.com> References: <4FBCAAEB.9000100@redhat.com> <1337962194.5899.37.camel@balmora.brq.redhat.com> Message-ID: <4FC4E95E.4030504@redhat.com> On 25.5.2012 18:09, Martin Kosek wrote: > On Wed, 2012-05-23 at 11:16 +0200, Jan Cholasta wrote: >> Hi, >> >> this fixes https://fedorahosted.org/freeipa/ticket/2769 as well as some >> other issues with SSH configuration in ipa-client-install. >> >> Honza >> > > This fixed the basic functionality, but I discovered another issue > (quite serious one). > > With /usr/bin/sss_ssh_knownhostsproxy as ssh ProxyCommand, I cannot > connect to remove client which does not have valid reverse record. > Without the proxy command, it works fine. I logged a Bugzilla for this > issue: > https://bugzilla.redhat.com/show_bug.cgi?id=825316 > > Martin > I have sent a fix for this issue to sssd-devel. Honza -- Jan Cholasta From simo at redhat.com Tue May 29 15:36:02 2012 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 May 2012 11:36:02 -0400 Subject: [Freeipa-devel] routing requests to local servers - DNS SRV [discussion needed] In-Reply-To: <4FC4E836.6080804@redhat.com> References: <4FBE6AC9.4020703@redhat.com> <1337955053.16840.286.camel@willson.li.ssimo.org> <4FC4E836.6080804@redhat.com> Message-ID: <1338305762.21387.56.camel@willson.li.ssimo.org> On Tue, 2012-05-29 at 17:16 +0200, Petr Spacek wrote: > Hello, > > for clarity: I'm not going to implement it (now). There are another features > on the table. > > I'm trying to find simplest solution/workaround, because several people asked > for this feature and I think it is quite important. (Besides load-balancing > purpose it can be handy for environments with NAT in place - as Amazon EC2.) > Discussion about major changes should be read as "design for far future". > > On 05/25/2012 04:10 PM, Simo Sorce wrote: > > On Thu, 2012-05-24 at 19:07 +0200, Petr Spacek wrote: > >> Hello, > >> > >> some time ago there was a request for DNS to support "routing requests to > >> local servers". Any opinions if/when do it and proposals how to do it are more > >> than welcome. My best knowledge about this problem follows: > >> > >> > >> This request actually means "differentiate answer to DNS query on client's IP > >> address basics". > >> Relevant thread on ipa-users-list: > >> https://www.redhat.com/archives/freeipa-users/2012-April/msg00070.html > >> > >> First question is: Do we want to implement it? > >> (IMHO it is very important for large-scale deployments.) > > > > I am not really sure I like it, as it makes things quite difficult to > > handle. > > > > Please think how we are going to operate if you ahve a view dfined and a > > client does a dyn DNS update. > Yes, it's a bit difficult. I propose this approach: > > - add new record -> add it to "base" (shared) part > - delete record -> delete "visible" record: i.e. delete override record if it > is present, delete "base" record otherwise. not sure this is the right thing, when you delete you should probably delete all, or adding back will fail. > It allows clients to update own records as usual and changes will be visible > to the whole world. Which may not be the desired outcome actually :-/ > One potential problem are update scripts from monitoring systems. Monitoring > system can test availability of server and dynamically adjust DNS records, if > server status has changed. > I'm not sure if some monitoring systems supports this feature directly, I > always used my own scripts. These scripts have to be changed from > nsupdate/another tool to ldapmodify, but I think it is acceptable. Custom systems can adapt indeed. > ...snip ... > > >> Adam and I discussed possible approaches. We agreed on "stackable" approach: > >> - Store whole original DNS tree in one subtree, let say "base". > >> - Create "override" subtrees for each "locality" = set of customized records. > >> Shared records are only in "base". During DNS query processing "override" > >> subtree is searched first. If record does not exist in "override" subtree, > >> search will continue in "base" subtree. > > > > Yes, this is my first thought too. > > > >> Second question is how to realize these "overrides": > >> - Let Directory Server to do the work. If DS supports this, it is mostly done. > > Why do you need dirsrv to do anything special ? > I can save bugs in the new code (and time) if dirsrv can do it. > > I played with 389 last days and unfortunately I can't find support for this > use case. (I'm pretty sure that OpenLDAP can support this scenario - that was > a reason why I started to examine this possibility in 389.) Not sure what you wanted to do, but I would avoid implementation specific stuff. Make some things simpler and other more complex. > > The simple way is to do subtree searches, if you get back more than one > > result for a specific name then you know you have views and proceed to > > filter out the one you want. The bonus here is that you can cache all > > replies if you keep different caches per view. > > > > The alternative is to add a 'viewname' or something in the filter, but > > then you need to do 2 searches and fallbacks. This sounds more > > expensive. > > > >> It do not require any change in bind-dyndb-ldap code. All merges/overrides > >> will be done on Directory server. > > > > Given we do persistent searches and we also do some caching in > > bind-dyndb-ldap we almost certainly do not want to 'fool' it by > > returning different values from DS w/o bind-dyndb-ldap knowledge. > I probably missed something, I don't see the problem. One bind-dyndb-ldap > instance is run for each "view" separately. Each instance has own caches and > other data structures. They don't know about each other. This is a problem, 389DS uses a lot fo resources for persistent searches. It means if you have views we'd have to disable persistent searches or enable them only in some views. Why do you need to run multiple instances ? Do each of them have its own in memory cache ? How much memory are we going to sue with many views ? I think we really need to discuss also these architectural issues. > >> Only /etc/named.conf has to be adjusted with > >> right search base and "view" clause. > >> > >> I asked on 389 mailing list and I'm waiting for reply: > >> http://lists.fedoraproject.org/pipermail/389-users/2012-May/014653.html > >> > >> > >> - Another approach is to add support directly to bind-dyndb-ldap, but it is > >> not so nice solution. > > > > Why ? > > > >> In that case both LDAP search bases have to be defined > >> in /etc/named.conf. For each DNS query is necessary to search "override" base > >> first. If required record is not present then is necessary to search through > >> "base" subtree. > > > > No, you do not need to do multiple searches, you just need to filter the > > results by view. > You are right, if we change the way how DNS data are stored in LDAP (a bit). > Mentioned idea was to create new subtree ("cn=dns-viewname") and re-use > existing driver and all tools for managing it with minimal changes. I see too many pitfalls, seem like not a good tradeoff in this case. > > It would make it very easy to manage if we added a 'dnsView' attribute > > to overriding entries in the subtree. This would allow us to define a > > new overriding entry and even assign it to multiple views if the > > 'dnsView' attribute is multivalued. > > > >> With persistent search it should be better, because all records are in memory, > >> but basic principle remains same. > > > > psearch is one of the reasons we might want a dnsView attribute base > > approach instead of playing games with DS. > > I would also prefer to avoid adding more code to DS where the bind > > plugin can easily handle this data itself. Another reason is that I > > really want this plugin code to be generic and usable with other LDAP > > servers and not be tied so strictly to 389ds. > > > > Simo. > > I'm not talking about new DS plugin, I wanted to use existing features. > > IMHO OpenLDAP with rewrite/remap and rwm overlays can do all the work for us. > Unfortunately 389 is poor in this case (or I can't find it in manual). Overlay == Plugin, it'a a custom feature specific of one implementation and not defined in a standard (afaik). > Merging with another thread: > > On 05/25/2012 09:20 PM, Simo Sorce wrote: > > On Fri, 2012-05-25 at 15:52 +0200, Petr Spacek wrote: > >> > On 05/24/2012 08:00 PM, Dmitri Pal wrote: > >>> > > On 05/24/2012 01:07 PM, Petr Spacek wrote: > >>> > > > >>> > > May be I am oversimplifying things but it seems that it would make > >>> > > sense > >>> > > to just have an extra multi-value attribute added to the DNS schema. > >>> > > This attribute would contain a value that would allow the entry to be > >>> > > included into the view. This way the base is the same and what the view > >>> > > exposes is just a filter. The default view would have a filter that > >>> > > matches all entries like now. > >>> > > > >>> > > > >>> > > I assume that DNS server makes it decision based on the IP so it has a > >>> > > call to get a "view" data. The ldap driver can return a filter. The DNS > >>> > > server them makes a call using this filter to get all the records that > >>> > > apply. I know it is too ldap centric. There is some abstraction in DNS > >>> > > server and we do not want to change it. But the point is there are > >>> > > probably two calls in the DNS server (have not actually confirmed): > >>> > > lookup data X by IP to understand what the view is. > >>> > > Pass data X to get the actual DNS entries. > >>> > > > >>> > > I am saying that X can be filter. > >>> > > > >>> > > Thoughts? > >> > > >> > Special attribute sounds like a good idea. It is not realizable > >> > directly, but > >> > I will explore variants with some "view" attribute. > >> > > >> > Current DNS "name" (name can potentially contain multiple records) > >> > structure > >> > is following: > >> > > >> > dn: idnsname=_kerberos._tcp,idnsname=e.localnet,cn=dns,dc=e,dc=org > >> > objectClass: idnsrecord > >> > objectClass: top > >> > idnsName: _kerberos._tcp > >> > sRVRecord: 0 100 88 unused-4-107 > >> > > >> > DNS name is part of DN. It is not possible to have more objects with > >> > same DNS > >> > name and different attributes. This problem lead me to "stackable" > >> > approach. > > Yes, and we can also use multiple attributes in the same tree, although > > for clarity I probably prefer the subtree approach. > > > > So a few options: > > > > 1. all in the same subtree: > > # Normal object > > dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org > > objectClass: idnsrecord > > objectClass: top > > idnsName: bar > > aRecord: 192.168.12.34 > > dNSTTL: 1200 > > > > # Object belongin to the 'DMZ' view > > dn:cn=DMZ-bar,idnsname=foo.org,cn=dns,dc=foo,dc=org > > objectClass: idnsrecord > > objectClass: top > > objectClass: nsContainer > > cn: DMZ-bar > > idnsName: bar > > aRecord: 5.6.7.8 > > dNSTTL: 3600 > > idnsView: DMZ > > > > > > NOTE: I had to add nsContainer here in order to give the object a way to > > have a unique name by using the CN attribute. I am not very fond of this > > arrangement though. It is also ugly to parse out using a LDAP browser. > > It make one thing simpler in that using multiple values for dnsView you > > can assign the same entry to multiple views. > > > > 2. using per view subtrees > Generally - I like this idea. > > > # Normal object > > dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org > > objectClass: idnsrecord > > objectClass: top > > idnsName: bar > > aRecord: 192.168.12.34 > > dNSTTL: 1200 > I prefer to create "_default" view for normal records (as BIND does). > e.g. dn: idnsname=bar,cn=_default,idnsname=foo.org,cn=dns,dc=foo,dc=org > > It allows to treat "normal" and override records in the same way. Also it > allows to optimize query with extensibleMatch filter. > E.g. plugin is configured to serve records for "view1": Filter can be > (|(cn:dn:=view1)(cn:dn:=_default)) so LDAP server will not return unnecessary > records for view2, view3, view4 ... I don;t think that filter is standard LDAP, I do not want to rely on something like that. It is preferrable to have a idnsView attribute in the entry, so you can use a normal filter and index on it. > Another problem to solve is how to modify SOA record/hide zone/add new zone to > view. With this view-zone-dnssubtree ordering you cannot modify SOA record or > hide the zone from certain views. Please provide more info, it is not clear to me why you can't do it. > With ordering zone-view-dnssubtree situation is better. We still can run > subtree query in 'cn=dns' and as bonus it is possible to define zone only in > some views or create modified version of zone's SOA record in another views. > > Problem is how to *not* override zone SOA record. One (not very nice) approach: > We can create extensibleObject and define container only with idnsName extensibleObject is not ok to use except for experimentation. > attribute (necessary for RDN construction). "Real" zones from "fake" ones can > be distinguished by objectClass. Records are leaves under this zone record as > usual. Create a idnsViewZone objectclass or something like that. > > # Object belongin to the 'DMZ' view > > dn:idnsname=bar,cn=DMZ,idnsname=foo.org,cn=dns,dc=foo,dc=org > > objectClass: idnsrecord > > objectClass: top > > idnsName: bar > > aRecord: 5.6.7.8 > > dNSTTL: 3600 > > > > > > NOTE: I prefer this method as it makes things a lot easier to manage and > > view through an LDAP broiwser, however it makes sharing entries between > > multiple views a bit awkward. > I like this idea. What about LDAP aliases? (I never used them, so I don't > know.) It is not very nice, but for limited amount of "override" records it > can be sufficient. Obviously this feature will be abused, so if you start with assuming "limited" you are already on the wrong path IMO :-) > > 3. using only one 'views' subtree pr zone and dnsView to discrimnate > > > > # Normal object > > dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org > > objectClass: idnsrecord > > objectClass: top > > idnsName: bar > > aRecord: 192.168.12.34 > > dNSTTL: 1200 > > > > # Object belongin to the 'DMZ' view > > dn:idnsUniqueID=F6A1245-bar,cn=views,idnsname=foo.org,cn=dns,dc=foo,dc=org > > objectClass: idnsrecord > > objectClass: top > > idnsUniqueID: F6A1245-bar > > idnsName: bar > > aRecord: 5.6.7.8 > > dNSTTL: 3600 > > idnsView: DMZ > > idnsView: VPN > > > > > > NOTE: here I added also a idnsUniqueID as a way to have unique names so > > we can have multiple entries for the same record. This is so that you > > can have 3 different entries for the same record belonging to 3 > > different views. The reason why I added the actual name after a random > > id is that this way it is simpler to recognize what it is when looking > > at an ldap browser w/o having to read the actual object attributes, it > > also make collisions a lot less likely and so it allows to keep the > > random part smaller (and thus more readable). > > Also note that I've put 2 values in idnsView, meaning that this record > > belongs to 2 separate views. This allows compact representation when > > multiple views want to redefine some records in the same way (an dothers > > in a different way, thus why 2 separate views)/ > I also prefer "way 2", as I said above. I actually prefer 3 :) Simo. -- Simo Sorce * Red Hat, Inc * New York From william.e.brown at adelaide.edu.au Tue May 29 15:58:10 2012 From: william.e.brown at adelaide.edu.au (William Brown) Date: Wed, 30 May 2012 01:28:10 +0930 Subject: [Freeipa-devel] routing requests to local servers - DNS SRV [discussion needed] In-Reply-To: <1337955053.16840.286.camel@willson.li.ssimo.org> References: <4FBE6AC9.4020703@redhat.com> <1337955053.16840.286.camel@willson.li.ssimo.org> Message-ID: <4FC4F212.7070706@adelaide.edu.au> On 25/05/12 11:40 PM, Simo Sorce wrote: >> It do not require any change in bind-dyndb-ldap code. All merges/overrides >> > will be done on Directory server. > Given we do persistent searches and we also do some caching in > bind-dyndb-ldap we almost certainly do not want to 'fool' it by > returning different values from DS w/o bind-dyndb-ldap knowledge. > >> > Only /etc/named.conf has to be adjusted with >> > right search base and "view" clause. >> > >> > I asked on 389 mailing list and I'm waiting for reply: >> > http://lists.fedoraproject.org/pipermail/389-users/2012-May/014653.html >> > >> > >> > - Another approach is to add support directly to bind-dyndb-ldap, but it is >> > not so nice solution. > Why ? > >> > In that case both LDAP search bases have to be defined >> > in /etc/named.conf. For each DNS query is necessary to search "override" base >> > first. If required record is not present then is necessary to search through >> > "base" subtree. > No, you do not need to do multiple searches, you just need to filter the > results by view. > > It would make it very easy to manage if we added a 'dnsView' attribute > to overriding entries in the subtree. This would allow us to define a > new overriding entry and even assign it to multiple views if the > 'dnsView' attribute is multivalued. > >> > With persistent search it should be better, because all records are in memory, >> > but basic principle remains same. > psearch is one of the reasons we might want a dnsView attribute base > approach instead of playing games with DS. > I would also prefer to avoid adding more code to DS where the bind > plugin can easily handle this data itself. Another reason is that I > really want this plugin code to be generic and usable with other LDAP > servers and not be tied so strictly to 389ds. I think Simo is on the money here. Adding this to bind-dyndb-ldap makes the most sense. Views become a multivalued attribute, and configurable via an idnsView object of some kind (It may even be some kind of "group" membership even?). This would only need to be in the idnsZone information - All records in a zone are bounded by the policies of that zone. There is no concept of a per-record view. It would also means that records can be correctly updated for DynDNS updates correctly. If memory serves correctly, Named impelements views in such a way that you update the record of the view you are updating with dyndns. So, lets say you have a view public and private. Lets also say you were sharing a zone between these azone.example (You were using the view to apply attributes like recursive query permissions). When the DynDNS update comes in, from a host matched in the "private" view, the code will go to update the zone that the the private view references. In this case, the update would be referenceable by both views. In the case where you have azone.example in public and private, but each has their own unique zone file, the dyndns update would only update the zone that the private zone has access to - The public view is still referencing the other data. This functionality could easily be achieved by the plugin. When the Dns update comes in, and that data is referenced by 2 views you have to choose - Do you duplicate the record, with each record now representing the data in each view, or do you update the record that is accesible by both views? From the FreeIPA standpoint of consistency and simplicity, I think that updating the record, provided the update comes from a view that is able to make updates, would be a sensible solution. Spliting the record is much hassle for no real benefit. This keeps the DNS data intact, and consistent across all views. Views would really only affect access policies to zones, recursive responders etc. The best benefit of this, would be that policies of "views" could be edited with the CLI tool or the web interface, rather than having to edit the named.conf file. This would again simplify administration of DNS services. -- Sincerely, William Brown Research & Teaching, Technology Services The University of Adelaide, AUSTRALIA 5005 CRICOS Provider Number 00123M ----------------------------------------------------------------------------- IMPORTANT: This message may contain confidential or legally privileged information. If you think it was sent to you by mistake, please delete all copies and advise the sender. For the purposes of the SPAM Act 2003, this email is authorised by The University of Adelaide. pgp.mit.edu http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 945 bytes Desc: OpenPGP digital signature URL: From simo at redhat.com Tue May 29 17:48:08 2012 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 May 2012 13:48:08 -0400 Subject: [Freeipa-devel] routing requests to local servers - DNS SRV [discussion needed] In-Reply-To: <4FC4F212.7070706@adelaide.edu.au> References: <4FBE6AC9.4020703@redhat.com> <1337955053.16840.286.camel@willson.li.ssimo.org> <4FC4F212.7070706@adelaide.edu.au> Message-ID: <1338313688.21387.68.camel@willson.li.ssimo.org> On Wed, 2012-05-30 at 01:28 +0930, William Brown wrote: > > The best benefit of this, would be that policies of "views" could be > edited with the CLI tool or the web interface, rather than having to > edit the named.conf file. This would again simplify administration of > DNS services. > Well said, I think this is one of the other key reasons to avoid multiple plugins, and handle everything from one. Otherwise we go back to having issues with updating these configurations dynamically via LDAP (always think replication). Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Tue May 29 20:32:25 2012 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 May 2012 16:32:25 -0400 Subject: [Freeipa-devel] [PATCH] 492 Add options to reduce writes from KDC In-Reply-To: <1337985380.16840.643.camel@willson.li.ssimo.org> References: <1337985380.16840.643.camel@willson.li.ssimo.org> Message-ID: <1338323545.21387.70.camel@willson.li.ssimo.org> On Fri, 2012-05-25 at 18:36 -0400, Simo Sorce wrote: > The original ldap driver we used up to 2.2 had 2 options admins could > set to limit the amount of writes to the database on certain auditing > related operations. > In particular disable_last_success is really important to reduce the > load on database servers. > > I have implemented ticket #2734 with a little twist. Instead of adding > local options in krb5.conf I create global options in the LDAP tree, so > that all KDCs in the domain have the same configuration. > > The 2 new options can be set in ipaConfigString attribute of the > cn=ipaConfig object under cn=etc,$SUFFIX > > These are: > KDC:Disable Last Success > KDC:Disable Lockout > > The first string if set will disable updating the krbLastSuccessfulAuth > field in the service/user entry. > The second one will prevent changing any of the Lockout related fields > and will effectively disable lockout policies. > > I think we may want to set the first one by default in future. > The last successful auth field is not very interesting in general and is > cause for a lot of writes that pressure a lot the LDAP server and get > replicated everywhere with a storm multiplier effect we'd like to avoid. > > The lockout one instead happen only when there are failed authentication > attempt, this means it never happens when keytabs are used for example. > And even with users it should happen rarely enough that traking lockouts > by default make leaving these writes on by default is a good tradeoff. > > Note that simply setting the lockout policy to never lockout is *not* > equivalent to setting KDC:Disable Lockout, as it does not prevent writes > to the database. > > I've tested setting KDC:Disable Last Success and it effectively prevent > MOD operation from showing up in the server access log. > > Any change to these configuration options requires a reconnection from > the KDC to the LDAP server, the simplest way to cause that is to restart > the KDC service. Attached also rebased patch that cleanly applies on top of 2.2. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-2.2-simo-492-1-Add-support-for-disabling-KDC-writes.patch Type: text/x-patch Size: 6822 bytes Desc: not available URL: From rcritten at redhat.com Tue May 29 20:44:26 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 29 May 2012 16:44:26 -0400 Subject: [Freeipa-devel] [PATCH] 1018 enforce sizelimit when searching for permissions In-Reply-To: <1338212519.5538.5.camel@balmora.brq.redhat.com> References: <4FB55B69.5080506@redhat.com> <1337327599.25787.10.camel@balmora.brq.redhat.com> <4FB65045.5060706@redhat.com> <4FB65A0C.5030306@redhat.com> <1337692317.17468.8.camel@balmora.brq.redhat.com> <4FBBFAF3.2070106@redhat.com> <4FBD2483.7090101@redhat.com> <1338212519.5538.5.camel@balmora.brq.redhat.com> Message-ID: <4FC5352A.8020900@redhat.com> Martin Kosek wrote: > On Wed, 2012-05-23 at 13:55 -0400, Rob Crittenden wrote: >> Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> On Fri, 2012-05-18 at 10:17 -0400, Rob Crittenden wrote: >>>>> Rob Crittenden wrote: >>>>>> Martin Kosek wrote: >>>>>>> On Thu, 2012-05-17 at 16:11 -0400, Rob Crittenden wrote: >>>>>>>> We do two searches when looking for permissions. One within the >>>>>>>> permission object itself and a second in the ACIs. We weren't >>>>>>>> enforcing >>>>>>>> a sizelimit on either search. >>>>>>>> >>>>>>>> rob >>>>>>> >>>>>>> This returns the right result, but I don't think it is right with >>>>>>> respect to "truncated" flag because of several reasons: >>>>>>> >>>>>>> 1) You manipulate and set "truncated" flag in post_callback but this >>>>>>> won't affect the flag in the returned result because the new value is >>>>>>> not propagated outside of the post_callback function. I.e. truncated >>>>>>> flag will be set correctly only when it was raised during original >>>>>>> permission_find. >>>>>> >>>>>> Truncated is still honored as expected. I even added a test case for >>>>>> this. >>>> >>>> Yes, but it only works when the truncated flag is raised by the base >>>> LDAP search, i.e. the search for permission objects (which is a case of >>>> your unit test). If the search does not raise the flag and it is set >>>> later in post callback, it is never propagated to the user. Please check >>>> the attached (crappy) test that shows this issue: >>>> >>>> ====================================================================== >>>> FAIL: test_permission[20]: permission_find: Search for permissions by >>>> attr with a limit of 1 (truncated) >>>> ---------------------------------------------------------------------- >>>> ... >>>> AssertionError: assert_deepequal: expected != got. >>>> test_permission[20]: permission_find: Search for permissions by attr >>>> with a limit of 1 (truncated) >>>> expected = True >>>> got = False >>>> path = ('truncated',) >>>> >>>> I am not sure how to solve this right, we may need to add a new return >>>> attribute (truncated) to all LDAPSearch post callbacks so that the post >>>> callback can really modify it - something similar we already do with pre >>>> callbacks which are able to change LDAP search filter, scope etc. >>>> >>>>>> >>>>>>> 2) The part with "ind" is strange: >>>>>>> >>>>>>> + # enforce --sizelimit >>>>>>> + if len(entries) == max_entries: >>>>>>> + if ind + 1< len(results): >>>>>>> + truncated = True >>>>>>> + break >>>>>>> >>>>>>> I think it would be much easier to just do >>>>>>> >>>>>>> ... >>>>>>> if (dn, permission) not in entries: >>>>>>> if len(entries)< max_entries: >>>>>>> entries.append((dn, permission)) >>>>>>> else: >>>>>>> truncated = True >>>>>>> break >>>>>>> >>>>>>> Otherwise you would rise "truncated" even when the rest of "results" >>>>>>> does not contain relevant entries that would have not been added >>>>>>> anyway. >>>>>> >>>>>> Yes, that makes sense. >>>>> >>>>> And now updated patch. >>>> >>>> We can now also remove the "enumerate" part, "ind" is no longer needed. >>>> >>>> Martin >>> >>> You're right, I'd have sworn I tested that... >>> >>> The only solution is going to be to have the post_callback return >>> truncated. This is going to be a rather intrusive change. >>> >>> rob >> >> Updated patch against master that adds a return value to post_callback. >> >> I also included Martin's test. It relies on the ordering of data in LDAP >> but it is better than nothing right now. >> >> rob > > Yeah, returning a value in LDAPSearch post_callback is OK. I just see > you missed post_callback in dnsrecord_find function, this makes > "dnsrecord-find" command and also other DNS command in the unit tests > crash. Yup, guess I didn't have DNS set up when I ran the tests. Fixed now. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1018-4-sizelimit.patch Type: text/x-diff Size: 13493 bytes Desc: not available URL: From rcritten at redhat.com Tue May 29 20:50:04 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 29 May 2012 16:50:04 -0400 Subject: [Freeipa-devel] [PATCH] 1019 require policycoreutils if SELinux is enabled In-Reply-To: <1338279596.30643.15.camel@balmora.brq.redhat.com> References: <4FB67069.1040901@redhat.com> <1338279596.30643.15.camel@balmora.brq.redhat.com> Message-ID: <4FC5367C.7010509@redhat.com> Martin Kosek wrote: > On Fri, 2012-05-18 at 11:53 -0400, Rob Crittenden wrote: >> We don't have an explicit requires on the policycoreutils package in the >> client because SELinux is not required (just recommended). >> >> SELinux can be enabled without this package so check for that condition >> and don't allow installation if it is the case. The resulting install >> will be rather broken. >> >> Also check on the server when installing. This should never happen but >> in theory it could do the server install then fail in the client because >> of this. >> >> rob > > This works fine. I am just thinking if we should not rather use paths > in /usr/ for the check if a binary exists, i.e. check > for /usr/sbin/restorecon instead of /sbin/restorecon on Fedora. > > If we don't do this we need to be sure that the /sbin -> /usr/sbin > symlink created during UsrMove will stay on the system. > > Martin > Ok, that makes sense. Updated patch. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1019-4-selinux.patch Type: text/x-diff Size: 11028 bytes Desc: not available URL: From rcritten at redhat.com Tue May 29 21:29:10 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 29 May 2012 17:29:10 -0400 Subject: [Freeipa-devel] [PATCH] 147 Set network.http.sendRefererHeader to 2 on browser config In-Reply-To: <4FBF99FB.7090206@redhat.com> References: <4FBF99FB.7090206@redhat.com> Message-ID: <4FC53FA6.5030906@redhat.com> Petr Vobornik wrote: > IPA web UI isn't functional when browser doesn't send http headers. > > This patch adds a functionality which sets Firefox > network.http.sendRefererHeader configuration option to value '2' which > enables it. > > Possible values: http://kb.mozillazine.org/Network.http.sendRefererHeader > > https://fedorahosted.org/freeipa/ticket/2778 Should we also add a message when referer is missing to check this setting in about:config? > > I was also thinking about upgrading the configure.jar. We had a ticket > for it, which ended by documenting the steps. > > https://fedorahosted.org/freeipa/ticket/2311 > http://docs.fedoraproject.org/en-US/Fedora/16/html/FreeIPA_Guide/upgrading.html#ticket-delegation > > > I think the documentation is wrong. In it we are rebuilding the .jar > from /usr/share/ipa/html/preferences.html, this file is created on > server install and it is never updated therefore the .jar won't be > updated. The updated file is its template (the one changed in this > patch). The template output is created in > httpinstance.__setup_autoconfig() call. > > For my development purposes I took this code and created a script which > rebuilds the .jar file (attached). Do we want to use it? Yes, I think it is worth having this somewhere, even if just on the wiki. rob From rcritten at redhat.com Tue May 29 21:46:20 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 29 May 2012 17:46:20 -0400 Subject: [Freeipa-devel] [PATCH] 0054 Provide a better error message when deleting nonexistent attributes In-Reply-To: <4FBFA2A6.7050804@redhat.com> References: <4FBFA2A6.7050804@redhat.com> Message-ID: <4FC543AC.6010803@redhat.com> Petr Viktorin wrote: > This fixes "misleading/invalid" error messages given when using > --delattr to delete values from an attribute that doesn't exist on the > entry. > Please see the trac comment for details. > > https://fedorahosted.org/freeipa/ticket/2699 ACK, pushed to master rob From simo at redhat.com Tue May 29 22:03:15 2012 From: simo at redhat.com (Simo Sorce) Date: Tue, 29 May 2012 18:03:15 -0400 Subject: [Freeipa-devel] [PATCH] Fix mspac code Message-ID: <1338328995.21387.71.camel@willson.li.ssimo.org> Hey, I pushed the attached oneliner. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-493-1-Fix-setting-domain_sid.patch Type: text/x-patch Size: 1171 bytes Desc: not available URL: From mkosek at redhat.com Wed May 30 05:45:43 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 30 May 2012 07:45:43 +0200 Subject: [Freeipa-devel] [PATCH] 269 permission-find missed some results with --pkey-only option Message-ID: <1338356743.3112.3.camel@priserak> When permission-find post callback detected a --pkey-only option, it just terminated. However, this way the results that could have been added from aci_find matches were not included. Fix the post callback to go through the entire matching process. Also make sure that DNS permissions have a correct objectclass (ipapermission), otherwise such objects are not matched by the permission LDAP search. https://fedorahosted.org/freeipa/ticket/2658 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-269-permission-find-pkey-only.patch Type: text/x-patch Size: 7457 bytes Desc: not available URL: From mkosek at redhat.com Wed May 30 05:49:29 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 30 May 2012 07:49:29 +0200 Subject: [Freeipa-devel] [PATCH] 79 SSH configuration fixes In-Reply-To: <4FC4E95E.4030504@redhat.com> References: <4FBCAAEB.9000100@redhat.com> <1337962194.5899.37.camel@balmora.brq.redhat.com> <4FC4E95E.4030504@redhat.com> Message-ID: <1338356969.3112.4.camel@priserak> On Tue, 2012-05-29 at 17:21 +0200, Jan Cholasta wrote: > On 25.5.2012 18:09, Martin Kosek wrote: > > On Wed, 2012-05-23 at 11:16 +0200, Jan Cholasta wrote: > >> Hi, > >> > >> this fixes https://fedorahosted.org/freeipa/ticket/2769 as well as some > >> other issues with SSH configuration in ipa-client-install. > >> > >> Honza > >> > > > > This fixed the basic functionality, but I discovered another issue > > (quite serious one). > > > > With /usr/bin/sss_ssh_knownhostsproxy as ssh ProxyCommand, I cannot > > connect to remove client which does not have valid reverse record. > > Without the proxy command, it works fine. I logged a Bugzilla for this > > issue: > > https://bugzilla.redhat.com/show_bug.cgi?id=825316 > > > > Martin > > > > I have sent a fix for this issue to sssd-devel. > > Honza > Ok, thanks. Since there is no change to be done on IPA side when the SSSD fix is released, we can push your change. So ACK, pushed to master, ipa-2-2. Martin From mkosek at redhat.com Wed May 30 06:52:45 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 30 May 2012 08:52:45 +0200 Subject: [Freeipa-devel] [PATCH] 1018 enforce sizelimit when searching for permissions In-Reply-To: <4FC5352A.8020900@redhat.com> References: <4FB55B69.5080506@redhat.com> <1337327599.25787.10.camel@balmora.brq.redhat.com> <4FB65045.5060706@redhat.com> <4FB65A0C.5030306@redhat.com> <1337692317.17468.8.camel@balmora.brq.redhat.com> <4FBBFAF3.2070106@redhat.com> <4FBD2483.7090101@redhat.com> <1338212519.5538.5.camel@balmora.brq.redhat.com> <4FC5352A.8020900@redhat.com> Message-ID: <1338360765.3112.11.camel@priserak> On Tue, 2012-05-29 at 16:44 -0400, Rob Crittenden wrote: > Martin Kosek wrote: > > On Wed, 2012-05-23 at 13:55 -0400, Rob Crittenden wrote: > >> Rob Crittenden wrote: > >>> Martin Kosek wrote: > >>>> On Fri, 2012-05-18 at 10:17 -0400, Rob Crittenden wrote: > >>>>> Rob Crittenden wrote: > >>>>>> Martin Kosek wrote: > >>>>>>> On Thu, 2012-05-17 at 16:11 -0400, Rob Crittenden wrote: > >>>>>>>> We do two searches when looking for permissions. One within the > >>>>>>>> permission object itself and a second in the ACIs. We weren't > >>>>>>>> enforcing > >>>>>>>> a sizelimit on either search. > >>>>>>>> > >>>>>>>> rob > >>>>>>> > >>>>>>> This returns the right result, but I don't think it is right with > >>>>>>> respect to "truncated" flag because of several reasons: > >>>>>>> > >>>>>>> 1) You manipulate and set "truncated" flag in post_callback but this > >>>>>>> won't affect the flag in the returned result because the new value is > >>>>>>> not propagated outside of the post_callback function. I.e. truncated > >>>>>>> flag will be set correctly only when it was raised during original > >>>>>>> permission_find. > >>>>>> > >>>>>> Truncated is still honored as expected. I even added a test case for > >>>>>> this. > >>>> > >>>> Yes, but it only works when the truncated flag is raised by the base > >>>> LDAP search, i.e. the search for permission objects (which is a case of > >>>> your unit test). If the search does not raise the flag and it is set > >>>> later in post callback, it is never propagated to the user. Please check > >>>> the attached (crappy) test that shows this issue: > >>>> > >>>> ====================================================================== > >>>> FAIL: test_permission[20]: permission_find: Search for permissions by > >>>> attr with a limit of 1 (truncated) > >>>> ---------------------------------------------------------------------- > >>>> ... > >>>> AssertionError: assert_deepequal: expected != got. > >>>> test_permission[20]: permission_find: Search for permissions by attr > >>>> with a limit of 1 (truncated) > >>>> expected = True > >>>> got = False > >>>> path = ('truncated',) > >>>> > >>>> I am not sure how to solve this right, we may need to add a new return > >>>> attribute (truncated) to all LDAPSearch post callbacks so that the post > >>>> callback can really modify it - something similar we already do with pre > >>>> callbacks which are able to change LDAP search filter, scope etc. > >>>> > >>>>>> > >>>>>>> 2) The part with "ind" is strange: > >>>>>>> > >>>>>>> + # enforce --sizelimit > >>>>>>> + if len(entries) == max_entries: > >>>>>>> + if ind + 1< len(results): > >>>>>>> + truncated = True > >>>>>>> + break > >>>>>>> > >>>>>>> I think it would be much easier to just do > >>>>>>> > >>>>>>> ... > >>>>>>> if (dn, permission) not in entries: > >>>>>>> if len(entries)< max_entries: > >>>>>>> entries.append((dn, permission)) > >>>>>>> else: > >>>>>>> truncated = True > >>>>>>> break > >>>>>>> > >>>>>>> Otherwise you would rise "truncated" even when the rest of "results" > >>>>>>> does not contain relevant entries that would have not been added > >>>>>>> anyway. > >>>>>> > >>>>>> Yes, that makes sense. > >>>>> > >>>>> And now updated patch. > >>>> > >>>> We can now also remove the "enumerate" part, "ind" is no longer needed. > >>>> > >>>> Martin > >>> > >>> You're right, I'd have sworn I tested that... > >>> > >>> The only solution is going to be to have the post_callback return > >>> truncated. This is going to be a rather intrusive change. > >>> > >>> rob > >> > >> Updated patch against master that adds a return value to post_callback. > >> > >> I also included Martin's test. It relies on the ordering of data in LDAP > >> but it is better than nothing right now. > >> > >> rob > > > > Yeah, returning a value in LDAPSearch post_callback is OK. I just see > > you missed post_callback in dnsrecord_find function, this makes > > "dnsrecord-find" command and also other DNS command in the unit tests > > crash. > > Yup, guess I didn't have DNS set up when I ran the tests. Fixed now. > > rob ACK. Pushed to master. Martin From mkosek at redhat.com Wed May 30 10:07:14 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 30 May 2012 12:07:14 +0200 Subject: [Freeipa-devel] [PATCH] 1019 require policycoreutils if SELinux is enabled In-Reply-To: <4FC5367C.7010509@redhat.com> References: <4FB67069.1040901@redhat.com> <1338279596.30643.15.camel@balmora.brq.redhat.com> <4FC5367C.7010509@redhat.com> Message-ID: <1338372434.3112.17.camel@priserak> On Tue, 2012-05-29 at 16:50 -0400, Rob Crittenden wrote: > Martin Kosek wrote: > > On Fri, 2012-05-18 at 11:53 -0400, Rob Crittenden wrote: > >> We don't have an explicit requires on the policycoreutils package in the > >> client because SELinux is not required (just recommended). > >> > >> SELinux can be enabled without this package so check for that condition > >> and don't allow installation if it is the case. The resulting install > >> will be rather broken. > >> > >> Also check on the server when installing. This should never happen but > >> in theory it could do the server install then fail in the client because > >> of this. > >> > >> rob > > > > This works fine. I am just thinking if we should not rather use paths > > in /usr/ for the check if a binary exists, i.e. check > > for /usr/sbin/restorecon instead of /sbin/restorecon on Fedora. > > > > If we don't do this we need to be sure that the /sbin -> /usr/sbin > > symlink created during UsrMove will stay on the system. > > > > Martin > > > > Ok, that makes sense. Updated patch. > > rob I think I was not entirely clear - the path /usr/sbin/restorecon shall be used for redhat platform only. UsrMove was done only in Fedora, IIRC, in RHEL 6.x /usr/sbin/restorecon is not a valid path to restorecon (I don't have my RHEL 6.x VM ready ATM) and the check would always fail on RHEL 6.x systems. Bottomline is that we may want to use a different path to the binary on redhat and fedora16 platform. I also think it would be useful to put the path to the binary to global constant, so that it is not repeated so many items over the platform files, i.e. something like that: ipapython/platform/redhat.py: RESTORECON_PATH='/sbin/restorecon' ... ipapython/platform/fedora16.py: RESTORECON_PATH='/usr/sbin/restorecon' ... Martin From pviktori at redhat.com Wed May 30 11:04:44 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 30 May 2012 13:04:44 +0200 Subject: [Freeipa-devel] [PATCH] 0040 Move install script error handling to a common function In-Reply-To: <1338288398.30643.35.camel@balmora.brq.redhat.com> References: <4F951EB7.9050908@redhat.com> <4F956FB7.3040000@redhat.com> <4FBB9896.7020908@redhat.com> <1338288398.30643.35.camel@balmora.brq.redhat.com> Message-ID: <4FC5FECC.9040207@redhat.com> On 05/29/2012 12:46 PM, Martin Kosek wrote: > On Tue, 2012-05-22 at 15:45 +0200, Petr Viktorin wrote: >> On 2012-04-23 17:05, John Dennis wrote: >>> On 04/23/2012 05:19 AM, Petr Viktorin wrote: >>>> This fixes https://fedorahosted.org/freeipa/ticket/2071 (Add final debug >>>> message in installers). >>>> >>>> I submitted an earlier version of this patch before (0014), but it was >>>> too much to include in 2.2. Hopefully now there's more space for >>>> restructuring. I think it's better to start a new thread with this >>>> approach. >>>> >>>> The try/except blocks at the end of installers/management scripts are >>>> replaced by a call to a common function, which includes the final >>>> message. >>>> For each specific error, the error handlers in all scripts was almost >>>> the same, but each script handled a different selection of errors. >>>> Instead of having this copy/pasted code (with subtle differences >>>> creeping in over time), this patch consolidates it all in one place. >>> >>> I like this approach much better than the earlier patch, great, thanks. >>> I'm a big fan of calling into common code instead of copying code to my >>> mind the refactoring to utilize common code is great approach. I also >>> like the fact the logging configuration is not modified after it's >>> established. >>> >>> At some point we may want to revist how the log messages are generated. >>> For example should all communication to the console pass through the >>> console handler? Is there a logger established for the script? Should >>> the format of messages emitted to the console be altered? Should all >>> command line utilities accept the both the verbose and debug flag? Etc. >>> But for now this is fantastic start in the right direction. >>> >>> I have not installed and exercised the patch so I can't comment on any >>> runtime time issues that might be present, but from code inspection only >>> it has my ACK. >>> >>> >> Thanks John! >> Yes, this is just a start. >> >> >> Patch rebased to curent master >> > > The patch needs another rebase. > > Besides that, the approach is fine (and removes a lot of redundant code) > but I found several issues with the patches: Thanks for the review! I was just looking into the issue. It missed that when logging is set up via api.bootstrap, by default the same logging level is set in both the log and console handlers. This makes it impossible to log only to the file -- INFO messages go to both file and console, and lower-level ones don't appear at al. Fixing this will require changes to the logging setup of individual commands, and possibly the logging mechanism itself, which is more than I want to include in this patch. Most scripts where this happens are not top-level installers (they're ipa-compliance, ipa-replica-conncheck, ipa-replica-manage, ipa-replica-prepare, ipa-ldap-updater). Since the ticket only asks for installers, I reverted the changes in these. Fixing them is left to ticket #2652. So, issues 2-3 & 5-10 that you found are avoided. I'll keep the cases around to test further work. I changed ipa-server-certinstall to set up logging explicitly. > 1) ipa-server-install: Fails when option parser error is raised: > > # ipa-server-install -p foo > Usage: ipa-server-install [options] > > ipa-server-install: error: DS admin password: Password must be at least > 8 characters long > Traceback (most recent call last): > File "/sbin/ipa-server-install", line 1131, in > if not success and installation_cleanup: > NameError: name 'success' is not defined Fixed > 4) ipa-ca-install emits failed_message when option error is detected: > > # ipa-ca-install > Usage: ipa-ca-install [options] REPLICA_FILE > > ipa-ca-install: error: you must provide a file generated by > ipa-replica-prepare > >>>> Your system may be partly configured. >>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. The attached patch skips printing the message on SystemExit, as it's done without the patch. > > [ipa-csreplica-manage] also has a wrong operation_name. This is what happens when a > script name is hard-coded and it is not detected automatically, e.g. > from __file__. I'll keep that in mind for #2652. Perhaps I'm trying too hard to avoid magic. > Martin > -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0040-03-Move-install-script-error-handling-to-a-common-funct.patch Type: text/x-patch Size: 28899 bytes Desc: not available URL: From ohamada at redhat.com Wed May 30 12:43:55 2012 From: ohamada at redhat.com (Ondrej Hamada) Date: Wed, 30 May 2012 14:43:55 +0200 Subject: [Freeipa-devel] [PATCH] 269 permission-find missed some results with --pkey-only option In-Reply-To: <1338356743.3112.3.camel@priserak> References: <1338356743.3112.3.camel@priserak> Message-ID: <4FC6160B.3030405@redhat.com> On 05/30/2012 07:45 AM, Martin Kosek wrote: > When permission-find post callback detected a --pkey-only option, > it just terminated. However, this way the results that could have > been added from aci_find matches were not included. > > Fix the post callback to go through the entire matching process. > Also make sure that DNS permissions have a correct objectclass > (ipapermission), otherwise such objects are not matched by the > permission LDAP search. > > https://fedorahosted.org/freeipa/ticket/2658 > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel Patch needs rebase It does not apply because of changes made to ipalib/plugins/permission.py (by Rob's patch #1018) -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada -------------- next part -------------- An HTML attachment was scrubbed... URL: From pviktori at redhat.com Wed May 30 13:39:59 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 30 May 2012 15:39:59 +0200 Subject: [Freeipa-devel] [PATCH] 0048 Rework the CallbackInterface In-Reply-To: <4FB21314.6040205@redhat.com> References: <4FABB28C.90208@redhat.com> <4FB21314.6040205@redhat.com> Message-ID: <4FC6232F.7070402@redhat.com> On 05/15/2012 10:25 AM, Petr Viktorin wrote: > On 05/10/2012 02:20 PM, Petr Viktorin wrote: >> While investigating ticket 2674, I found several problems with our >> implementation of the CallbackInterface ?? it required complicated >> calling code, and would subtly break if command classes were >> instantiated in different ways than they are currently. >> >> Here's my fix. See commit message for details. >> > > Rebased to current master > Rebased again. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0048-03-Rework-the-CallbackInterface.patch Type: text/x-patch Size: 28758 bytes Desc: not available URL: From jcholast at redhat.com Wed May 30 16:01:45 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 30 May 2012 18:01:45 +0200 Subject: [Freeipa-devel] [PATCH] 268 Add rename option for DNS records In-Reply-To: <1338303561.10122.2.camel@balmora.brq.redhat.com> References: <1338300097.30643.56.camel@balmora.brq.redhat.com> <4FC4DFE0.5010508@redhat.com> <1338303561.10122.2.camel@balmora.brq.redhat.com> Message-ID: <4FC64469.1020608@redhat.com> On 29.5.2012 16:59, Martin Kosek wrote: > On Tue, 2012-05-29 at 16:40 +0200, Jan Cholasta wrote: >> On 29.5.2012 16:01, Martin Kosek wrote: >>> This option will make renaming DNS records much easier. >>> Add a unit test for this new functionality. >>> >>> https://fedorahosted.org/freeipa/ticket/2600 >>> >> >> I wonder, how hard would it be to modify the patch to allow --rename on >> all objects, as requested in? >> >> Honza >> > > Not sure, but it would require a lot of testing. As you can see in a > case of dnsrecord object, it may need more changes than just adding > rdn_is_primary_key=True to LDAPObjects. I assume there is a reason why > we deferred this effort for now. > > Martin > OK. Regarding the patch: 1) When I do: $ ipa dnsrecord-mod example.com @ --rename test it returns an error: ipa: ERROR: @: DNS resource record not found and the zone "example.com" is renamed to "test". 2) The patch needs to be rebased. Honza -- Jan Cholasta From rcritten at redhat.com Wed May 30 21:47:35 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 30 May 2012 17:47:35 -0400 Subject: [Freeipa-devel] [PATCH] 1019 require policycoreutils if SELinux is enabled In-Reply-To: <1338372434.3112.17.camel@priserak> References: <4FB67069.1040901@redhat.com> <1338279596.30643.15.camel@balmora.brq.redhat.com> <4FC5367C.7010509@redhat.com> <1338372434.3112.17.camel@priserak> Message-ID: <4FC69577.2040307@redhat.com> Martin Kosek wrote: > On Tue, 2012-05-29 at 16:50 -0400, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On Fri, 2012-05-18 at 11:53 -0400, Rob Crittenden wrote: >>>> We don't have an explicit requires on the policycoreutils package in the >>>> client because SELinux is not required (just recommended). >>>> >>>> SELinux can be enabled without this package so check for that condition >>>> and don't allow installation if it is the case. The resulting install >>>> will be rather broken. >>>> >>>> Also check on the server when installing. This should never happen but >>>> in theory it could do the server install then fail in the client because >>>> of this. >>>> >>>> rob >>> >>> This works fine. I am just thinking if we should not rather use paths >>> in /usr/ for the check if a binary exists, i.e. check >>> for /usr/sbin/restorecon instead of /sbin/restorecon on Fedora. >>> >>> If we don't do this we need to be sure that the /sbin -> /usr/sbin >>> symlink created during UsrMove will stay on the system. >>> >>> Martin >>> >> >> Ok, that makes sense. Updated patch. >> >> rob > > I think I was not entirely clear - the path /usr/sbin/restorecon shall > be used for redhat platform only. UsrMove was done only in Fedora, IIRC, > in RHEL 6.x /usr/sbin/restorecon is not a valid path to restorecon (I > don't have my RHEL 6.x VM ready ATM) and the check would always fail on > RHEL 6.x systems. Bottomline is that we may want to use a different path > to the binary on redhat and fedora16 platform. > > I also think it would be useful to put the path to the binary to global > constant, so that it is not repeated so many items over the platform > files, i.e. something like that: > > ipapython/platform/redhat.py: > RESTORECON_PATH='/sbin/restorecon' > ... > > ipapython/platform/fedora16.py: > RESTORECON_PATH='/usr/sbin/restorecon' > ... > > Martin Ok, now I see what you were getting at. This should achieve it. Can't do per-file variables like this as the one in redhat.py will always win. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1019-5-selinux.patch Type: text/x-diff Size: 17113 bytes Desc: not available URL: From mkosek at redhat.com Thu May 31 09:10:24 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 31 May 2012 11:10:24 +0200 Subject: [Freeipa-devel] [PATCH] 268 Add rename option for DNS records In-Reply-To: <4FC64469.1020608@redhat.com> References: <1338300097.30643.56.camel@balmora.brq.redhat.com> <4FC4DFE0.5010508@redhat.com> <1338303561.10122.2.camel@balmora.brq.redhat.com> <4FC64469.1020608@redhat.com> Message-ID: <1338455424.2956.6.camel@balmora.brq.redhat.com> On Wed, 2012-05-30 at 18:01 +0200, Jan Cholasta wrote: > On 29.5.2012 16:59, Martin Kosek wrote: > > On Tue, 2012-05-29 at 16:40 +0200, Jan Cholasta wrote: > >> On 29.5.2012 16:01, Martin Kosek wrote: > >>> This option will make renaming DNS records much easier. > >>> Add a unit test for this new functionality. > >>> > >>> https://fedorahosted.org/freeipa/ticket/2600 > >>> > >> > >> I wonder, how hard would it be to modify the patch to allow --rename on > >> all objects, as requested in? > >> > >> Honza > >> > > > > Not sure, but it would require a lot of testing. As you can see in a > > case of dnsrecord object, it may need more changes than just adding > > rdn_is_primary_key=True to LDAPObjects. I assume there is a reason why > > we deferred this effort for now. > > > > Martin > > > > OK. > > Regarding the patch: > > 1) When I do: > > $ ipa dnsrecord-mod example.com @ --rename test > > it returns an error: > > ipa: ERROR: @: DNS resource record not found > > and the zone "example.com" is renamed to "test". > > 2) The patch needs to be rebased. > > Honza > Patch is rebased. I also fixed a case when user tries to rename DNS zone root record (good catch btw.) and added a test case for that. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-268-2-add-rename-option-for-dns-records.patch Type: text/x-patch Size: 7526 bytes Desc: not available URL: From pspacek at redhat.com Thu May 31 10:36:33 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 31 May 2012 12:36:33 +0200 Subject: [Freeipa-devel] DNS zone serial number updates [#2554]: local SOA approach In-Reply-To: <20120525124105.GA11168@redhat.com> References: <4F8D9110.3030703@redhat.com> <1334679211.16658.200.camel@willson.li.ssimo.org> <4F8EC1C8.4090204@redhat.com> <1334757864.16658.258.camel@willson.li.ssimo.org> <4F8EDBE0.7090108@redhat.com> <1334764404.16658.277.camel@willson.li.ssimo.org> <4FA7EEB1.1040909@redhat.com> <4FA83098.4000800@redhat.com> <20120525124105.GA11168@redhat.com> Message-ID: <4FC749B1.8050802@redhat.com> On 05/25/2012 02:41 PM, Adam Tkac wrote: > On Mon, May 07, 2012 at 10:29:12PM +0200, Petr Spacek wrote: >> Hello, >> >> on the last meeting there was another approach to $SUBJ$ discussed: >> >> Each DNS server will maintain its own serial number value >> independently from other servers. >> >> Pros: >> Should be simpler to implement; no DS plugin required. >> >> Cons: >> Slave DNS servers cannot fall-back to other masters, because of SOA >> serial inconsistency. >> >> >> Very basic implementation: >> 1) Do not replicate idnsSoaSerial attribute >> 2) Use persistent search to watch for incoming changes >> 3) After each change increment "local" SOA serial number (and write >> it to LDAP - to survive DNS server restart) >> >> There is a problem with database updates if bind-dyndb-ldap plugin >> is not running, but "its" DS runs. In that case DS can replicate >> changes from other servers but there is no bind-dyndb-ldap plugin to >> modify serial number. >> >> I think it can happen during startup/shutdown phase or after BIND >> crash. In this case zone content can be changed without appropriate >> SOA serial update. >> >> Dumb solution: >> Always increment stored SOA serial number after DNS server start. It >> causes unnecessary zone transfer, but we are safe. >> >> Another solution (IPA+389 specific): >> Remember max(entryUSN) value computed from all records together with >> SOA serial. (Principle is the same as with modifyTimestamp described >> earlier in this thread.) It requires new LDAP object with two >> attributes: >> - assigned value of idnsSOASerial >> - highest entryUSN found >> DNS server after start can check if max(entryUSN) == recorded max >> value. If values do not match plugin bumps idnsSOASerial. >> >> If entryUSN is not supported by server, we can fall-back to bumping >> idnsSOASerial on every start-up. >> >> Did I miss anything? :-) >> >> It requires persistent search, but I resigned already. > > After further discussion this seems like the best approach for me as well. > > Regards, Adam The original RFE reporter is not comfortable with the proposed solution (local SOA serials). Can somebody judge seriousness of these objections? The latest discussion from BZ https://bugzilla.redhat.com/show_bug.cgi?id=766233 follows: Comment 18 Brian J. Atkisson 2012-05-25 11:51:26 EDT wrote: > > Hrm I replied on 5/9 and don't see the answer posted to bugzilla. In any case: > > >> Is it acceptable to have SOA serial numbers only locally significant? (I.e. each DNS master (= IPA server with DNS) will have different SOA serial number for given zone.) > > This would cause problems for non-IPA slaves performing zone transfer. > All the serial numbers should be in sync across the IPA masters. > > For example, in a corporate environment, where they maybe one department > using plain bind to host lab zones from corporate IT running IPA > Different labs will have different zone versions based upon those labs > pointing to regional IPA servers. > > Also, it would be problematic in environments where a plain old Bind server does zone transfers from IPA servers in a fail-over/load-balanced config. Comment 19 Petr Spacek 2012-05-28 06:45:42 EDT wrote: > Comment 18 Brian J. Atkisson 2012-05-25 11:51:26 EDT wrote: >> Hrm I replied on 5/9 and don't see the answer posted to bugzilla. In any >> case: >> >> >> > Is it acceptable to have SOA serial numbers only locally significant? (I.e. each DNS master (= IPA server with DNS) will have different SOA serial number for given zone.) >> >> This would cause problems for non-IPA slaves performing zone transfer. >> All the serial numbers should be in sync across the IPA masters. > Of course, generally you are right. But IPA violates classical single-master DNS model and we search for some trade-offs. (For example RFC 5936 http://tools.ietf.org/html/rfc5936#section-3.1 says "don't do it".) > Question should be: "Is there real configuration where it matters?" > > >> For example, in a corporate environment, where they maybe one department >> using plain bind to host lab zones from corporate IT running IPA >> Different labs will have different zone versions based upon those labs >> pointing to regional IPA servers. > I'm probably missing something, I don't see the problem. Different IPA servers will serve same data (when LDAP replication settles down), only SOA serial number will be different. Each plain BIND will see and serve exactly same zone except serial number. > > >> Also, it would be problematic in environments where a plain old Bind server >> does zone transfers from IPA servers in a fail-over/load-balanced config. > Yes, I agree. It's known problem of "local SOA serial" approach. Is described configuration deployed anywhere? Probably this problem can be partially alleviated with "tree" of DNS slaves or something similar. Petr^2 Spacek From mkosek at redhat.com Thu May 31 10:42:04 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 31 May 2012 12:42:04 +0200 Subject: [Freeipa-devel] [PATCH] 269 permission-find missed some results with --pkey-only option In-Reply-To: <4FC6160B.3030405@redhat.com> References: <1338356743.3112.3.camel@priserak> <4FC6160B.3030405@redhat.com> Message-ID: <1338460924.2956.7.camel@balmora.brq.redhat.com> On Wed, 2012-05-30 at 14:43 +0200, Ondrej Hamada wrote: > On 05/30/2012 07:45 AM, Martin Kosek wrote: > > When permission-find post callback detected a --pkey-only option, > > it just terminated. However, this way the results that could have > > been added from aci_find matches were not included. > > > > Fix the post callback to go through the entire matching process. > > Also make sure that DNS permissions have a correct objectclass > > (ipapermission), otherwise such objects are not matched by the > > permission LDAP search. > > > > https://fedorahosted.org/freeipa/ticket/2658 > > > > > > > > _______________________________________________ > > Freeipa-devel mailing list > > Freeipa-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/freeipa-devel > Patch needs rebase > > It does not apply because of changes made to > ipalib/plugins/permission.py (by Rob's patch #1018) > Rebased version attached. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-269-2-permission-find-pkey-only.patch Type: text/x-patch Size: 7771 bytes Desc: not available URL: From jcholast at redhat.com Thu May 31 10:44:58 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 31 May 2012 12:44:58 +0200 Subject: [Freeipa-devel] [PATCH] 268 Add rename option for DNS records In-Reply-To: <1338455424.2956.6.camel@balmora.brq.redhat.com> References: <1338300097.30643.56.camel@balmora.brq.redhat.com> <4FC4DFE0.5010508@redhat.com> <1338303561.10122.2.camel@balmora.brq.redhat.com> <4FC64469.1020608@redhat.com> <1338455424.2956.6.camel@balmora.brq.redhat.com> Message-ID: <4FC74BAA.50402@redhat.com> On 31.5.2012 11:10, Martin Kosek wrote: > On Wed, 2012-05-30 at 18:01 +0200, Jan Cholasta wrote: >> On 29.5.2012 16:59, Martin Kosek wrote: >>> On Tue, 2012-05-29 at 16:40 +0200, Jan Cholasta wrote: >>>> On 29.5.2012 16:01, Martin Kosek wrote: >>>>> This option will make renaming DNS records much easier. >>>>> Add a unit test for this new functionality. >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/2600 >>>>> >>>> >>>> I wonder, how hard would it be to modify the patch to allow --rename on >>>> all objects, as requested in? >>>> >>>> Honza >>>> >>> >>> Not sure, but it would require a lot of testing. As you can see in a >>> case of dnsrecord object, it may need more changes than just adding >>> rdn_is_primary_key=True to LDAPObjects. I assume there is a reason why >>> we deferred this effort for now. >>> >>> Martin >>> >> >> OK. >> >> Regarding the patch: >> >> 1) When I do: >> >> $ ipa dnsrecord-mod example.com @ --rename test >> >> it returns an error: >> >> ipa: ERROR: @: DNS resource record not found >> >> and the zone "example.com" is renamed to "test". >> >> 2) The patch needs to be rebased. >> >> Honza >> > > Patch is rebased. I also fixed a case when user tries to rename DNS zone > root record (good catch btw.) and added a test case for that. > > Martin ACK. Honza -- Jan Cholasta From mkosek at redhat.com Thu May 31 10:47:52 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 31 May 2012 12:47:52 +0200 Subject: [Freeipa-devel] [PATCH] 268 Add rename option for DNS records In-Reply-To: <4FC74BAA.50402@redhat.com> References: <1338300097.30643.56.camel@balmora.brq.redhat.com> <4FC4DFE0.5010508@redhat.com> <1338303561.10122.2.camel@balmora.brq.redhat.com> <4FC64469.1020608@redhat.com> <1338455424.2956.6.camel@balmora.brq.redhat.com> <4FC74BAA.50402@redhat.com> Message-ID: <1338461272.2956.8.camel@balmora.brq.redhat.com> On Thu, 2012-05-31 at 12:44 +0200, Jan Cholasta wrote: > On 31.5.2012 11:10, Martin Kosek wrote: > > On Wed, 2012-05-30 at 18:01 +0200, Jan Cholasta wrote: > >> On 29.5.2012 16:59, Martin Kosek wrote: > >>> On Tue, 2012-05-29 at 16:40 +0200, Jan Cholasta wrote: > >>>> On 29.5.2012 16:01, Martin Kosek wrote: > >>>>> This option will make renaming DNS records much easier. > >>>>> Add a unit test for this new functionality. > >>>>> > >>>>> https://fedorahosted.org/freeipa/ticket/2600 > >>>>> > >>>> > >>>> I wonder, how hard would it be to modify the patch to allow --rename on > >>>> all objects, as requested in? > >>>> > >>>> Honza > >>>> > >>> > >>> Not sure, but it would require a lot of testing. As you can see in a > >>> case of dnsrecord object, it may need more changes than just adding > >>> rdn_is_primary_key=True to LDAPObjects. I assume there is a reason why > >>> we deferred this effort for now. > >>> > >>> Martin > >>> > >> > >> OK. > >> > >> Regarding the patch: > >> > >> 1) When I do: > >> > >> $ ipa dnsrecord-mod example.com @ --rename test > >> > >> it returns an error: > >> > >> ipa: ERROR: @: DNS resource record not found > >> > >> and the zone "example.com" is renamed to "test". > >> > >> 2) The patch needs to be rebased. > >> > >> Honza > >> > > > > Patch is rebased. I also fixed a case when user tries to rename DNS zone > > root record (good catch btw.) and added a test case for that. > > > > Martin > > ACK. > > Honza > Pushed to master. Martin From pviktori at redhat.com Thu May 31 11:42:43 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 31 May 2012 13:42:43 +0200 Subject: [Freeipa-devel] [PATCH] 267 Allow relative DNS name in NS validator In-Reply-To: <1338297105.30643.54.camel@balmora.brq.redhat.com> References: <1338297105.30643.54.camel@balmora.brq.redhat.com> Message-ID: <4FC75933.9040008@redhat.com> On 05/29/2012 03:11 PM, Martin Kosek wrote: > Precallback validator was failing when a zone-relative name was > used as a NS record (for example record "ns" in a zone "example.com"). > However, this is valid in BIND and we should allow it as well. > > Imports in dns module had to be switched to absolute imports > (available from Python 2.5) to deal with a conflict of IPA dns > module and dnspython module. > > https://fedorahosted.org/freeipa/ticket/2630 > This works fine, but it breaks a test: ====================================================================== FAIL: test_dns[48]: dnsrecord_add: Try to add unresolvable NS record to u'testdnsres' using dnsrecord_add ---------------------------------------------------------------------- [...] expected = u"Nameserver 'does.not.exist' does not have a corresponding A/AAAA record" got = u"Nameserver 'does.not.exist.dnszone.test.' does not have a corresponding A/AAAA record" path = () -- Petr? From mkosek at redhat.com Thu May 31 12:18:00 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 31 May 2012 14:18:00 +0200 Subject: [Freeipa-devel] [PATCH] 1019 require policycoreutils if SELinux is enabled In-Reply-To: <4FC69577.2040307@redhat.com> References: <4FB67069.1040901@redhat.com> <1338279596.30643.15.camel@balmora.brq.redhat.com> <4FC5367C.7010509@redhat.com> <1338372434.3112.17.camel@priserak> <4FC69577.2040307@redhat.com> Message-ID: <1338466680.2956.10.camel@balmora.brq.redhat.com> On Wed, 2012-05-30 at 17:47 -0400, Rob Crittenden wrote: > Martin Kosek wrote: > > On Tue, 2012-05-29 at 16:50 -0400, Rob Crittenden wrote: > >> Martin Kosek wrote: > >>> On Fri, 2012-05-18 at 11:53 -0400, Rob Crittenden wrote: > >>>> We don't have an explicit requires on the policycoreutils package in the > >>>> client because SELinux is not required (just recommended). > >>>> > >>>> SELinux can be enabled without this package so check for that condition > >>>> and don't allow installation if it is the case. The resulting install > >>>> will be rather broken. > >>>> > >>>> Also check on the server when installing. This should never happen but > >>>> in theory it could do the server install then fail in the client because > >>>> of this. > >>>> > >>>> rob > >>> > >>> This works fine. I am just thinking if we should not rather use paths > >>> in /usr/ for the check if a binary exists, i.e. check > >>> for /usr/sbin/restorecon instead of /sbin/restorecon on Fedora. > >>> > >>> If we don't do this we need to be sure that the /sbin -> /usr/sbin > >>> symlink created during UsrMove will stay on the system. > >>> > >>> Martin > >>> > >> > >> Ok, that makes sense. Updated patch. > >> > >> rob > > > > I think I was not entirely clear - the path /usr/sbin/restorecon shall > > be used for redhat platform only. UsrMove was done only in Fedora, IIRC, > > in RHEL 6.x /usr/sbin/restorecon is not a valid path to restorecon (I > > don't have my RHEL 6.x VM ready ATM) and the check would always fail on > > RHEL 6.x systems. Bottomline is that we may want to use a different path > > to the binary on redhat and fedora16 platform. > > > > I also think it would be useful to put the path to the binary to global > > constant, so that it is not repeated so many items over the platform > > files, i.e. something like that: > > > > ipapython/platform/redhat.py: > > RESTORECON_PATH='/sbin/restorecon' > > ... > > > > ipapython/platform/fedora16.py: > > RESTORECON_PATH='/usr/sbin/restorecon' > > ... > > > > Martin > > Ok, now I see what you were getting at. This should achieve it. > > Can't do per-file variables like this as the one in redhat.py will > always win. > > rob That's OK, ACK. Rebased and pushed to master. Martin From mkosek at redhat.com Thu May 31 12:44:20 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 31 May 2012 14:44:20 +0200 Subject: [Freeipa-devel] [PATCH] 0040 Move install script error handling to a common function In-Reply-To: <4FC5FECC.9040207@redhat.com> References: <4F951EB7.9050908@redhat.com> <4F956FB7.3040000@redhat.com> <4FBB9896.7020908@redhat.com> <1338288398.30643.35.camel@balmora.brq.redhat.com> <4FC5FECC.9040207@redhat.com> Message-ID: <1338468260.2956.12.camel@balmora.brq.redhat.com> On Wed, 2012-05-30 at 13:04 +0200, Petr Viktorin wrote: > On 05/29/2012 12:46 PM, Martin Kosek wrote: > > On Tue, 2012-05-22 at 15:45 +0200, Petr Viktorin wrote: > >> On 2012-04-23 17:05, John Dennis wrote: > >>> On 04/23/2012 05:19 AM, Petr Viktorin wrote: > >>>> This fixes https://fedorahosted.org/freeipa/ticket/2071 (Add final debug > >>>> message in installers). > >>>> > >>>> I submitted an earlier version of this patch before (0014), but it was > >>>> too much to include in 2.2. Hopefully now there's more space for > >>>> restructuring. I think it's better to start a new thread with this > >>>> approach. > >>>> > >>>> The try/except blocks at the end of installers/management scripts are > >>>> replaced by a call to a common function, which includes the final > >>>> message. > >>>> For each specific error, the error handlers in all scripts was almost > >>>> the same, but each script handled a different selection of errors. > >>>> Instead of having this copy/pasted code (with subtle differences > >>>> creeping in over time), this patch consolidates it all in one place. > >>> > >>> I like this approach much better than the earlier patch, great, thanks. > >>> I'm a big fan of calling into common code instead of copying code to my > >>> mind the refactoring to utilize common code is great approach. I also > >>> like the fact the logging configuration is not modified after it's > >>> established. > >>> > >>> At some point we may want to revist how the log messages are generated. > >>> For example should all communication to the console pass through the > >>> console handler? Is there a logger established for the script? Should > >>> the format of messages emitted to the console be altered? Should all > >>> command line utilities accept the both the verbose and debug flag? Etc. > >>> But for now this is fantastic start in the right direction. > >>> > >>> I have not installed and exercised the patch so I can't comment on any > >>> runtime time issues that might be present, but from code inspection only > >>> it has my ACK. > >>> > >>> > >> Thanks John! > >> Yes, this is just a start. > >> > >> > >> Patch rebased to curent master > >> > > > > The patch needs another rebase. > > > > Besides that, the approach is fine (and removes a lot of redundant code) > > but I found several issues with the patches: > > Thanks for the review! > > I was just looking into the issue. It missed that when logging is set up > via api.bootstrap, by default the same logging level is set in both the > log and console handlers. This makes it impossible to log only to the > file -- INFO messages go to both file and console, and lower-level ones > don't appear at al. > Fixing this will require changes to the logging setup of individual > commands, and possibly the logging mechanism itself, which is more than > I want to include in this patch. > > Most scripts where this happens are not top-level installers (they're > ipa-compliance, ipa-replica-conncheck, ipa-replica-manage, > ipa-replica-prepare, ipa-ldap-updater). Since the ticket only asks for > installers, I reverted the changes in these. Fixing them is left to > ticket #2652. > So, issues 2-3 & 5-10 that you found are avoided. > I'll keep the cases around to test further work. > > > I changed ipa-server-certinstall to set up logging explicitly. > > > > 1) ipa-server-install: Fails when option parser error is raised: > > > > # ipa-server-install -p foo > > Usage: ipa-server-install [options] > > > > ipa-server-install: error: DS admin password: Password must be at least > > 8 characters long > > Traceback (most recent call last): > > File "/sbin/ipa-server-install", line 1131, in > > if not success and installation_cleanup: > > NameError: name 'success' is not defined > > Fixed > > > 4) ipa-ca-install emits failed_message when option error is detected: > > > > # ipa-ca-install > > Usage: ipa-ca-install [options] REPLICA_FILE > > > > ipa-ca-install: error: you must provide a file generated by > > ipa-replica-prepare > > > >>>> Your system may be partly configured. > >>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. > > The attached patch skips printing the message on SystemExit, as it's > done without the patch. > > > > > [ipa-csreplica-manage] also has a wrong operation_name. This is what > happens when a > > script name is hard-coded and it is not detected automatically, e.g. > > from __file__. > > I'll keep that in mind for #2652. Perhaps I'm trying too hard to avoid > magic. > > > Martin > > > > Ok, lets handle the rest in #2652. The updated patch looks good, I did not found any other blocking issue - ACK. I rebased the patch and pushed to master. Martin From simo at redhat.com Thu May 31 14:05:36 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 31 May 2012 10:05:36 -0400 Subject: [Freeipa-devel] DNS zone serial number updates [#2554]: local SOA approach In-Reply-To: <4FC749B1.8050802@redhat.com> References: <4F8D9110.3030703@redhat.com> <1334679211.16658.200.camel@willson.li.ssimo.org> <4F8EC1C8.4090204@redhat.com> <1334757864.16658.258.camel@willson.li.ssimo.org> <4F8EDBE0.7090108@redhat.com> <1334764404.16658.277.camel@willson.li.ssimo.org> <4FA7EEB1.1040909@redhat.com> <4FA83098.4000800@redhat.com> <20120525124105.GA11168@redhat.com> <4FC749B1.8050802@redhat.com> Message-ID: <1338473136.8230.65.camel@willson.li.ssimo.org> On Thu, 2012-05-31 at 12:36 +0200, Petr Spacek wrote: > On 05/25/2012 02:41 PM, Adam Tkac wrote: > > On Mon, May 07, 2012 at 10:29:12PM +0200, Petr Spacek wrote: > >> Hello, > >> > >> on the last meeting there was another approach to $SUBJ$ discussed: > >> > >> Each DNS server will maintain its own serial number value > >> independently from other servers. > >> > >> Pros: > >> Should be simpler to implement; no DS plugin required. > >> > >> Cons: > >> Slave DNS servers cannot fall-back to other masters, because of SOA > >> serial inconsistency. > >> > >> > >> Very basic implementation: > >> 1) Do not replicate idnsSoaSerial attribute > >> 2) Use persistent search to watch for incoming changes > >> 3) After each change increment "local" SOA serial number (and write > >> it to LDAP - to survive DNS server restart) > >> > >> There is a problem with database updates if bind-dyndb-ldap plugin > >> is not running, but "its" DS runs. In that case DS can replicate > >> changes from other servers but there is no bind-dyndb-ldap plugin to > >> modify serial number. > >> > >> I think it can happen during startup/shutdown phase or after BIND > >> crash. In this case zone content can be changed without appropriate > >> SOA serial update. > >> > >> Dumb solution: > >> Always increment stored SOA serial number after DNS server start. It > >> causes unnecessary zone transfer, but we are safe. > >> > >> Another solution (IPA+389 specific): > >> Remember max(entryUSN) value computed from all records together with > >> SOA serial. (Principle is the same as with modifyTimestamp described > >> earlier in this thread.) It requires new LDAP object with two > >> attributes: > >> - assigned value of idnsSOASerial > >> - highest entryUSN found > >> DNS server after start can check if max(entryUSN) == recorded max > >> value. If values do not match plugin bumps idnsSOASerial. > >> > >> If entryUSN is not supported by server, we can fall-back to bumping > >> idnsSOASerial on every start-up. > >> > >> Did I miss anything? :-) > >> > >> It requires persistent search, but I resigned already. > > > > After further discussion this seems like the best approach for me as well. > > > > Regards, Adam > > The original RFE reporter is not comfortable with the proposed solution (local > SOA serials). Can somebody judge seriousness of these objections? > > The latest discussion from BZ > https://bugzilla.redhat.com/show_bug.cgi?id=766233 follows: Those objections are serious, but I do not think they necessarily have the weight to turn down the proposed solution. As you noted in the bugzilla it is a tradeoff, and we are adding functionality (multi-master DNS servers) not normally available. Bind servers cannot attach to multiple master servers, bind understand only a single master so normally slaves will be connected to a specific IPA server which acts as their master. For these server the serial does not change arbitrarily, it is always the same. The load balancer objection is bogus imo, the reason you add slaves is to spread the load, and I've never seen load balanced DNS masters, for the simple reason that Bind does not understand having multiple masters. So all in all I still think that having a local SOA is the most effective solution. If there are major concerns what we can do is to keep using the date format (we loose a bit of granularity I guess, but that shouldn't be a big deal), so that on average all masters will be pretty close and at most you'll have to wait a couple of seconds to have a SOA that is greater than what a previous master had. > Comment 18 Brian J. Atkisson 2012-05-25 11:51:26 EDT wrote: > > > > Hrm I replied on 5/9 and don't see the answer posted to bugzilla. In any case: > > > > > >> Is it acceptable to have SOA serial numbers only locally significant? (I.e. each DNS master (= IPA server with DNS) will have different SOA serial number for given zone.) > > > > This would cause problems for non-IPA slaves performing zone transfer. > > All the serial numbers should be in sync across the IPA masters. > > > > For example, in a corporate environment, where they maybe one department > > using plain bind to host lab zones from corporate IT running IPA > > Different labs will have different zone versions based upon those labs > > pointing to regional IPA servers. > > > > Also, it would be problematic in environments where a plain old Bind server does zone transfers from IPA servers in a fail-over/load-balanced config. > > Comment 19 Petr Spacek 2012-05-28 06:45:42 EDT wrote: > > Comment 18 Brian J. Atkisson 2012-05-25 11:51:26 EDT wrote: > >> Hrm I replied on 5/9 and don't see the answer posted to bugzilla. In any > >> case: > >> > >> > >> > Is it acceptable to have SOA serial numbers only locally significant? (I.e. each DNS master (= IPA server with DNS) will have different SOA serial number for given zone.) > >> > >> This would cause problems for non-IPA slaves performing zone transfer. > >> All the serial numbers should be in sync across the IPA masters. > > Of course, generally you are right. But IPA violates classical single-master DNS model and we search for some trade-offs. (For example RFC 5936 http://tools.ietf.org/html/rfc5936#section-3.1 says "don't do it".) > > Question should be: "Is there real configuration where it matters?" > > > > > >> For example, in a corporate environment, where they maybe one department > >> using plain bind to host lab zones from corporate IT running IPA > >> Different labs will have different zone versions based upon those labs > >> pointing to regional IPA servers. > > I'm probably missing something, I don't see the problem. Different IPA servers will serve same data (when LDAP replication settles down), only SOA serial number will be different. Each plain BIND will see and serve exactly same zone except serial number. > > > > > >> Also, it would be problematic in environments where a plain old Bind server > >> does zone transfers from IPA servers in a fail-over/load-balanced config. > > Yes, I agree. It's known problem of "local SOA serial" approach. Is described configuration deployed anywhere? Probably this problem can be partially alleviated with "tree" of DNS slaves or something similar. > > Petr^2 Spacek > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Thu May 31 14:58:35 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 31 May 2012 16:58:35 +0200 Subject: [Freeipa-devel] [PATCH] 267 Allow relative DNS name in NS validator In-Reply-To: <4FC75933.9040008@redhat.com> References: <1338297105.30643.54.camel@balmora.brq.redhat.com> <4FC75933.9040008@redhat.com> Message-ID: <1338476315.15462.1.camel@balmora.brq.redhat.com> On Thu, 2012-05-31 at 13:42 +0200, Petr Viktorin wrote: > On 05/29/2012 03:11 PM, Martin Kosek wrote: > > Precallback validator was failing when a zone-relative name was > > used as a NS record (for example record "ns" in a zone "example.com"). > > However, this is valid in BIND and we should allow it as well. > > > > Imports in dns module had to be switched to absolute imports > > (available from Python 2.5) to deal with a conflict of IPA dns > > module and dnspython module. > > > > https://fedorahosted.org/freeipa/ticket/2630 > > > > This works fine, but it breaks a test: > > ====================================================================== > FAIL: test_dns[48]: dnsrecord_add: Try to add unresolvable NS record to > u'testdnsres' using dnsrecord_add > ---------------------------------------------------------------------- > [...] > > expected = u"Nameserver 'does.not.exist' does not have a > corresponding A/AAAA record" > got = u"Nameserver 'does.not.exist.dnszone.test.' does not have a > corresponding A/AAAA record" > path = () > I updated the tests to use an absolute DNS record. All DNS tests should now succeed. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-267-2-allow-relative-dns-name-in-ns-validator.patch Type: text/x-patch Size: 4913 bytes Desc: not available URL: From pspacek at redhat.com Thu May 31 15:16:15 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 31 May 2012 17:16:15 +0200 Subject: [Freeipa-devel] full BIND view support [was: routing requests to local servers - DNS SRV] In-Reply-To: <1338305762.21387.56.camel@willson.li.ssimo.org> References: <4FBE6AC9.4020703@redhat.com> <1337955053.16840.286.camel@willson.li.ssimo.org> <4FC4E836.6080804@redhat.com> <1338305762.21387.56.camel@willson.li.ssimo.org> Message-ID: <4FC78B3F.4020507@redhat.com> Hello, I found RFE for this feature: https://bugzilla.redhat.com/show_bug.cgi?id=815621 It was filled against IPA component in BZ so I didn't find it up to now. On 05/29/2012 05:36 PM, Simo Sorce wrote: > On Tue, 2012-05-29 at 17:16 +0200, Petr Spacek wrote: >> Discussion about major changes should be read as "design for far future". >> >> On 05/25/2012 04:10 PM, Simo Sorce wrote: >>> On Thu, 2012-05-24 at 19:07 +0200, Petr Spacek wrote: ...snip ... >>>> This request actually means "differentiate answer to DNS query on client's IP >>>> address basics". >>>> Relevant thread on ipa-users-list: >>>> https://www.redhat.com/archives/freeipa-users/2012-April/msg00070.html >>>> >>>> First question is: Do we want to implement it? >>>> (IMHO it is very important for large-scale deployments.) >>> >>> I am not really sure I like it, as it makes things quite difficult to >>> handle. >>> >>> Please think how we are going to operate if you ahve a view dfined and a >>> client does a dyn DNS update. ...snip ... Merged with proposal from another thread: On 05/29/2012 05:58 PM, William Brown wrote: > I think Simo is on the money here. Adding this to bind-dyndb-ldap makes > the most sense. Views become a multivalued attribute, and configurable > via an idnsView object of some kind (It may even be some kind of "group" > membership even?). This would only need to be in the idnsZone > information - All records in a zone are bounded by the policies of that > zone. There is no concept of a per-record view. It would also means that > records can be correctly updated for DynDNS updates correctly. > > If memory serves correctly, Named impelements views in such a way that > you update the record of the view you are updating with dyndns. So, lets > say you have a view public and private. Lets also say you were sharing a > zone between these azone.example (You were using the view to apply > attributes like recursive query permissions). When the DynDNS update > comes in, from a host matched in the "private" view, the code will go to > update the zone that the the private view references. In this case, the > update would be referenceable by both views. > > In the case where you have azone.example in public and private, but each > has their own unique zone file, the dyndns update would only update the > zone that the private zone has access to - The public view is still > referencing the other data. If I understood correctly, you propose to share all records or nothing. If I didn't miss something, it equals to current behaviour. You can define two views with same base DN or different DN - and share all or nothing. Main problem is how to create and maintain "slightly" different zones, e.g. SRV records with different priorities. With plain BIND you can create two zone files and use separate file and $INCLUDE for shared records. I will explore BIND's behaviour with two zones containing shared part. Proposed approaches should allow sharing at record level in our LDAP scheme (see my reply below). Or - missed I something in you text? > This functionality could easily be achieved by the plugin. When the Dns > update comes in, and that data is referenced by 2 views you have to > choose - Do you duplicate the record, with each record now representing > the data in each view, or do you update the record that is accesible by > both views? From the FreeIPA standpoint of consistency and simplicity, I > think that updating the record, provided the update comes from a view > that is able to make updates, would be a sensible solution. Spliting the > record is much hassle for no real benefit. This keeps the DNS data > intact, and consistent across all views. Views would really only affect > access policies to zones, recursive responders etc. > > The best benefit of this, would be that policies of "views" could be > edited with the CLI tool or the web interface, rather than having to > edit the named.conf file. This would again simplify administration of > DNS services. > -- Sincerely, William Brown Yes, it is a big benefit of LDAP configuration. >> ...snip ... >> >>>> Adam and I discussed possible approaches. We agreed on "stackable" approach: >>>> - Store whole original DNS tree in one subtree, let say "base". >>>> - Create "override" subtrees for each "locality" = set of customized records. >>>> Shared records are only in "base". During DNS query processing "override" >>>> subtree is searched first. If record does not exist in "override" subtree, >>>> search will continue in "base" subtree. >>> >>> Yes, this is my first thought too. >>> >>>> Second question is how to realize these "overrides": ...snip ... >>> The simple way is to do subtree searches, if you get back more than one >>> result for a specific name then you know you have views and proceed to >>> filter out the one you want. The bonus here is that you can cache all >>> replies if you keep different caches per view. >>> >>> The alternative is to add a 'viewname' or something in the filter, but >>> then you need to do 2 searches and fallbacks. This sounds more >>> expensive. >>> >>>> It do not require any change in bind-dyndb-ldap code. All merges/overrides >>>> will be done on Directory server. >>> >>> Given we do persistent searches and we also do some caching in >>> bind-dyndb-ldap we almost certainly do not want to 'fool' it by >>> returning different values from DS w/o bind-dyndb-ldap knowledge. >> I probably missed something, I don't see the problem. One bind-dyndb-ldap >> instance is run for each "view" separately. Each instance has own caches and >> other data structures. They don't know about each other. > > This is a problem, 389DS uses a lot fo resources for persistent > searches. It means if you have views we'd have to disable persistent > searches or enable them only in some views. I don't understand why 389DS requires a lot of resources in this case. My (naive) imagination says: "Do normal search during initial query and then send any updates to client as soon as they comes in." What I missed? (I really don't know about 389 internals, so sorry for naive questions.) > Why do you need to run multiple instances ? Do each of them have its own > in memory cache ? How much memory are we going to sue with many views ? Each "instance" is completely independent. I can't explain this decision in it's full deep, this design was done a long time ago (before I started). > I think we really need to discuss also these architectural issues. I checked the code: Full view support will require bigger changes, but it is doable. (All necessary BIND-plugin APIs are in place.) Re-implementing all configuration options from named.conf can be a bit difficult. http://www.zytrax.com/books/dns/ch7/view.html contains ~ 60 different view options, ~ 33 of them are not present in zones. General question is: What we want to do with named.conf? Is our target to re-implement all named.conf options in plugin/LDAP? Configuration is quite complex and some general rule how to do it is necessary for interface consistency. ... snip ... >>>> - Another approach is to add support directly to bind-dyndb-ldap, but it is >>>> not so nice solution. >>> >>> Why ? >>> >>>> In that case both LDAP search bases have to be defined >>>> in /etc/named.conf. For each DNS query is necessary to search "override" base >>>> first. If required record is not present then is necessary to search through >>>> "base" subtree. >>> >>> No, you do not need to do multiple searches, you just need to filter the >>> results by view. >> You are right, if we change the way how DNS data are stored in LDAP (a bit). >> Mentioned idea was to create new subtree ("cn=dns-viewname") and re-use >> existing driver and all tools for managing it with minimal changes. > > I see too many pitfalls, seem like not a good tradeoff in this case. No way is really simple and straight-forward. I'm not against plugin side implementation generally, it allows better configuration management and simpler record-sharing. >>> It would make it very easy to manage if we added a 'dnsView' attribute >>> to overriding entries in the subtree. This would allow us to define a >>> new overriding entry and even assign it to multiple views if the >>> 'dnsView' attribute is multivalued. >> Merging with another thread: >> On 05/25/2012 09:20 PM, Simo Sorce wrote: >>> On Fri, 2012-05-25 at 15:52 +0200, Petr Spacek wrote: >>>>> On 05/24/2012 08:00 PM, Dmitri Pal wrote: ... snip ... >>>>> Current DNS "name" (name can potentially contain multiple records) >>>>> structure >>>>> is following: >>>>> >>>>> dn: idnsname=_kerberos._tcp,idnsname=e.localnet,cn=dns,dc=e,dc=org >>>>> objectClass: idnsrecord >>>>> objectClass: top >>>>> idnsName: _kerberos._tcp >>>>> sRVRecord: 0 100 88 unused-4-107 >>>>> >>>>> DNS name is part of DN. It is not possible to have more objects with >>>>> same DNS >>>>> name and different attributes. This problem lead me to "stackable" >>>>> approach. >>> Yes, and we can also use multiple attributes in the same tree, although >>> for clarity I probably prefer the subtree approach. >>> >>> So a few options: >>> >>> 1. all in the same subtree: >>> # Normal object >>> dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org >>> objectClass: idnsrecord >>> objectClass: top >>> idnsName: bar >>> aRecord: 192.168.12.34 >>> dNSTTL: 1200 >>> >>> # Object belongin to the 'DMZ' view >>> dn:cn=DMZ-bar,idnsname=foo.org,cn=dns,dc=foo,dc=org >>> objectClass: idnsrecord >>> objectClass: top >>> objectClass: nsContainer >>> cn: DMZ-bar >>> idnsName: bar >>> aRecord: 5.6.7.8 >>> dNSTTL: 3600 >>> idnsView: DMZ >>> >>> >>> NOTE: I had to add nsContainer here in order to give the object a way to >>> have a unique name by using the CN attribute. I am not very fond of this >>> arrangement though. It is also ugly to parse out using a LDAP browser. >>> It make one thing simpler in that using multiple values for dnsView you >>> can assign the same entry to multiple views. Advantage can be speed: You can do subtree search with base idnsname=foo.org,cn=dns,dc=foo,dc=org, so only necessary part of whole LDAP tree is walked through. Filter can contain only idnsView condition without a idnsName. (Current code does LDAP search with base idnsname=foo.org,cn=dns,dc=foo,dc=org and scope "base". Is it good approach?) We can create idnsRecordOverride class with single attribute idnsViewGroupID or similar to avoid necessity of nsContainer. It allows to use "(|(idnsView=view1)(!(idnsView=*)))" filter also. It can replace extendedMatch filter mentioned below, it is not necessary to walk through objects not related to "target" view or default. >>> 2. using per view subtrees >> Generally - I like this idea. >> >>> # Normal object >>> dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org >>> objectClass: idnsrecord >>> objectClass: top >>> idnsName: bar >>> aRecord: 192.168.12.34 >>> dNSTTL: 1200 >> I prefer to create "_default" view for normal records (as BIND does). >> e.g. dn: idnsname=bar,cn=_default,idnsname=foo.org,cn=dns,dc=foo,dc=org >> >> It allows to treat "normal" and override records in the same way. Also it >> allows to optimize query with extensibleMatch filter. >> E.g. plugin is configured to serve records for "view1": Filter can be >> (|(cn:dn:=view1)(cn:dn:=_default)) so LDAP server will not return unnecessary >> records for view2, view3, view4 ... > > I don;t think that filter is standard LDAP, I do not want to rely on > something like that. http://tools.ietf.org/html/rfc4511#section-4.5.1.7.7 > It is preferrable to have a idnsView attribute in the entry, so you can > use a normal filter and index on it. > >> Another problem to solve is how to modify SOA record/hide zone/add new zone to >> view. With this view-zone-dnssubtree ordering you cannot modify SOA record or >> hide the zone from certain views. > > Please provide more info, it is not clear to me why you can't do it. I probably misunderstood original text. I can nest idnsZone object to cn=view subtree ... Most basic question is how to merge "view" data with "base" data: What about simplest approach - make override at "name" (= LDAP object) level? E.g. "base" object: dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org objectClass: idnsrecord objectClass: top idnsName: bar aRecord: 192.168.12.34 tXTRecord: "blah blah" dNSTTL: 1200 "view" object: dn: idnsname=bar,cn=view,idnsname=foo.org,cn=dns,dc=foo,dc=org objectClass: idnsrecord objectClass: top idnsName: bar aRecord: 11.22.33.44 Query for bar.foo.org will be resolved (through view) to: bar IN A 11.22.33.44 (There is default TTL and no TXT record.) If I think about override at record (= LDAP attribute) level, I don't see a way how to delete a record. >> With ordering zone-view-dnssubtree situation is better. We still can run >> subtree query in 'cn=dns' and as bonus it is possible to define zone only in >> some views or create modified version of zone's SOA record in another views. >> >> Problem is how to *not* override zone SOA record. One (not very nice) approach: >> We can create extensibleObject and define container only with idnsName > > extensibleObject is not ok to use except for experimentation. > >> attribute (necessary for RDN construction). "Real" zones from "fake" ones can >> be distinguished by objectClass. Records are leaves under this zone record as >> usual. > > Create a idnsViewZone objectclass or something like that. It is better approach, definitely. >>> # Object belongin to the 'DMZ' view >>> dn:idnsname=bar,cn=DMZ,idnsname=foo.org,cn=dns,dc=foo,dc=org >>> objectClass: idnsrecord >>> objectClass: top >>> idnsName: bar >>> aRecord: 5.6.7.8 >>> dNSTTL: 3600 >>> >>> >>> NOTE: I prefer this method as it makes things a lot easier to manage and >>> view through an LDAP broiwser, however it makes sharing entries between >>> multiple views a bit awkward. >> I like this idea. What about LDAP aliases? (I never used them, so I don't >> know.) It is not very nice, but for limited amount of "override" records it >> can be sufficient. > > Obviously this feature will be abused, so if you start with assuming > "limited" you are already on the wrong path IMO :-) Ok, now we are talking about full view support. Not the original (limited) "routing requests to local servers". >>> 3. using only one 'views' subtree pr zone and dnsView to discrimnate >>> >>> # Normal object >>> dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org >>> objectClass: idnsrecord >>> objectClass: top >>> idnsName: bar >>> aRecord: 192.168.12.34 >>> dNSTTL: 1200 >>> >>> # Object belongin to the 'DMZ' view >>> dn:idnsUniqueID=F6A1245-bar,cn=views,idnsname=foo.org,cn=dns,dc=foo,dc=org >>> objectClass: idnsrecord >>> objectClass: top >>> idnsUniqueID: F6A1245-bar >>> idnsName: bar >>> aRecord: 5.6.7.8 >>> dNSTTL: 3600 >>> idnsView: DMZ >>> idnsView: VPN >>> >>> >>> NOTE: here I added also a idnsUniqueID as a way to have unique names so >>> we can have multiple entries for the same record. This is so that you >>> can have 3 different entries for the same record belonging to 3 >>> different views. The reason why I added the actual name after a random >>> id is that this way it is simpler to recognize what it is when looking >>> at an ldap browser w/o having to read the actual object attributes, it >>> also make collisions a lot less likely and so it allows to keep the >>> random part smaller (and thus more readable). >>> Also note that I've put 2 values in idnsView, meaning that this record >>> belongs to 2 separate views. This allows compact representation when >>> multiple views want to redefine some records in the same way (an dothers >>> in a different way, thus why 2 separate views)/ >> I also prefer "way 2", as I said above. > I actually prefer 3 :) > Simo. After some time I started to prefer variant 1. Only disadvantage is a bit worse reading (for humans :-) in LDAP browser. I tried it in my LDAP browser (Apache LDAP studio): With search filter (|(cn=view1)(cn=view3)) and return attribute list "tXTRecord, cn" it draws nice table: "DN" "tXTRecord" "cn" "cn=view1,idnsname=test2,idnsname=ee.localnet,cn=dns,dc=e,dc=org" "text for view 1+3" "view1|view3" "cn=view1,idnsname=test,idnsname=ee.localnet,cn=dns,dc=e,dc=org" "text view 1" "view1" Oh, text wrapping is not perfect for ASCII art ... See the attachment. With click on DN I can jump directly to found record. Yes, you has to do search operation. I think you has to do search even with "way three" on real databases. Multi-valued idnsView attribute can make eye-browsing hard, because override record<->view pairing is not obvious from DN. For debugging purposes you probably use search to limit whole tree to problematic parts only and so on ... (I have testing DB with 60k records and it is really not good idea to use LDAP browser without limits to walk through it ...) For mentioned reasons I prefer "way 1". Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: view_search.png Type: image/png Size: 16798 bytes Desc: not available URL: From ohamada at redhat.com Thu May 31 15:43:03 2012 From: ohamada at redhat.com (Ondrej Hamada) Date: Thu, 31 May 2012 17:43:03 +0200 Subject: [Freeipa-devel] [PATCH] 269 permission-find missed some results with --pkey-only option In-Reply-To: <1338460924.2956.7.camel@balmora.brq.redhat.com> References: <1338356743.3112.3.camel@priserak> <4FC6160B.3030405@redhat.com> <1338460924.2956.7.camel@balmora.brq.redhat.com> Message-ID: <4FC79187.2000607@redhat.com> On 05/31/2012 12:42 PM, Martin Kosek wrote: > On Wed, 2012-05-30 at 14:43 +0200, Ondrej Hamada wrote: >> On 05/30/2012 07:45 AM, Martin Kosek wrote: >>> When permission-find post callback detected a --pkey-only option, >>> it just terminated. However, this way the results that could have >>> been added from aci_find matches were not included. >>> >>> Fix the post callback to go through the entire matching process. >>> Also make sure that DNS permissions have a correct objectclass >>> (ipapermission), otherwise such objects are not matched by the >>> permission LDAP search. >>> >>> https://fedorahosted.org/freeipa/ticket/2658 >>> >>> >>> >>> _______________________________________________ >>> Freeipa-devel mailing list >>> Freeipa-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/freeipa-devel >> Patch needs rebase >> >> It does not apply because of changes made to >> ipalib/plugins/permission.py (by Rob's patch #1018) >> > Rebased version attached. > > Martin ACK -- Regards, Ondrej Hamada FreeIPA team jabber: ohama at jabbim.cz IRC: ohamada From JR.Aquino at citrix.com Thu May 31 21:52:05 2012 From: JR.Aquino at citrix.com (JR Aquino) Date: Thu, 31 May 2012 21:52:05 +0000 Subject: [Freeipa-devel] [PATCH] 492 Add options to reduce writes from KDC In-Reply-To: <1338323545.21387.70.camel@willson.li.ssimo.org> References: <1337985380.16840.643.camel@willson.li.ssimo.org> <1338323545.21387.70.camel@willson.li.ssimo.org> Message-ID: <34736B61-9560-49A5-8F5F-DE769EF90F00@citrixonline.com> On May 29, 2012, at 1:32 PM, Simo Sorce wrote: > On Fri, 2012-05-25 at 18:36 -0400, Simo Sorce wrote: >> The original ldap driver we used up to 2.2 had 2 options admins could >> set to limit the amount of writes to the database on certain auditing >> related operations. >> In particular disable_last_success is really important to reduce the >> load on database servers. >> >> I have implemented ticket #2734 with a little twist. Instead of adding >> local options in krb5.conf I create global options in the LDAP tree, so >> that all KDCs in the domain have the same configuration. >> >> The 2 new options can be set in ipaConfigString attribute of the >> cn=ipaConfig object under cn=etc,$SUFFIX >> >> These are: >> KDC:Disable Last Success >> KDC:Disable Lockout >> >> The first string if set will disable updating the krbLastSuccessfulAuth >> field in the service/user entry. >> The second one will prevent changing any of the Lockout related fields >> and will effectively disable lockout policies. >> >> I think we may want to set the first one by default in future. >> The last successful auth field is not very interesting in general and is >> cause for a lot of writes that pressure a lot the LDAP server and get >> replicated everywhere with a storm multiplier effect we'd like to avoid. >> >> The lockout one instead happen only when there are failed authentication >> attempt, this means it never happens when keytabs are used for example. >> And even with users it should happen rarely enough that traking lockouts >> by default make leaving these writes on by default is a good tradeoff. >> >> Note that simply setting the lockout policy to never lockout is *not* >> equivalent to setting KDC:Disable Lockout, as it does not prevent writes >> to the database. >> >> I've tested setting KDC:Disable Last Success and it effectively prevent >> MOD operation from showing up in the server access log. >> >> Any change to these configuration options requires a reconnection from >> the KDC to the LDAP server, the simplest way to cause that is to restart >> the KDC service. > > Attached also rebased patch that cleanly applies on top of 2.2. > > Simo. > > -- > Simo Sorce * Red Hat, Inc * New York > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel ACK This patch and " KDC:Disable Last Success" brought the writes and replications down by an order of a magnitude!