From pzuna at redhat.com Mon Nov 2 12:03:20 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Mon, 02 Nov 2009 13:03:20 +0100 Subject: [Freeipa-devel] [PATCH] Handle ipaEnabledFlag as bool (TRUE/FALSE) instead of string (enabled/disabled). Message-ID: <4AEECA88.7050207@redhat.com> Syntax in schema is 1.3.6.1.4.1.1466.115.121.1.7 == Boolean syntax. Thanks to Sumit for pointing this out. Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Handle-ipaEnabledFlag-as-bool-TRUE-FALSE-instead-o.patch Type: application/mbox Size: 1826 bytes Desc: not available URL: From pzuna at redhat.com Mon Nov 2 15:19:08 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Mon, 02 Nov 2009 16:19:08 +0100 Subject: [Freeipa-devel] [PATCH] Add DS to IPA migration plugin and password migration page. In-Reply-To: <4AE9CEC3.5060905@redhat.com> References: <4AE9CEC3.5060905@redhat.com> Message-ID: <4AEEF86C.9030508@redhat.com> Everyone wrote: ... A LOT and Thunderbird isn't able to display a thread on a mailing list properly. I did some testing on how much time does it take to migrate "a few" users. I started with 10000, but unfortunately my VM can't handle that much (always running out of space and I already deleted /usr/share/doc :D). Anyway, I successfully migrated about ~4200 users in 27 minutes using the current method. I didn't test it using the IPA commands yet, because I ran into the problem of making LDAP data valid for IPA commands - it's actually not that easy. We can't pass user passwords to them and we also cannot set attributes the command don't support, so we have to manually set them using ldap2.update_entry anyway. I know that the numbers at the beginning of this paragraph mean nothing if I have nothing to compare them to, but I thought you might be interested anyway. I'll keep you updated. Another thing: with user friendliness/experience. I think users will actually suffer a little after being migrated, because they will have to take all of these steps: 1) login to the migration page 2) use kinit 3) if their password doesn't meet IPA password policy, change their password 4) go to ipa page, probably won't work 5) configure their browsers 6) go to ipa page again, this time it will work :) Just saying. Pavel From rcritten at redhat.com Mon Nov 2 15:31:31 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 02 Nov 2009 10:31:31 -0500 Subject: [Freeipa-devel] [PATCH] Add DS to IPA migration plugin and password migration page. In-Reply-To: <4AEEF86C.9030508@redhat.com> References: <4AE9CEC3.5060905@redhat.com> <4AEEF86C.9030508@redhat.com> Message-ID: <4AEEFB53.4080808@redhat.com> Pavel Zuna wrote: > Everyone wrote: > ... > A LOT and Thunderbird isn't able to display a thread on a mailing list > properly. > > I did some testing on how much time does it take to migrate "a few" > users. I started with 10000, but unfortunately my VM can't handle that > much (always running out of space and I already deleted /usr/share/doc :D). > > Anyway, I successfully migrated about ~4200 users in 27 minutes using > the current method. I didn't test it using the IPA commands yet, because > I ran into the problem of making LDAP data valid for IPA commands - it's > actually not that easy. We can't pass user passwords to them and we also > cannot set attributes the command don't support, so we have to manually > set them using ldap2.update_entry anyway. I know that the numbers at the > beginning of this paragraph mean nothing if I have nothing to compare > them to, but I thought you might be interested anyway. I'll keep you > updated. Yes, something we need in baseldap.py is a way to pass in arbitrary attributes to Add and Modify. There are several modes we need: Add a new value to an attribute (this attr may or may not be in the entry) Set an attribute to a value (a replace operation) Remove a value from an attribute. Removing the last value should remove the attribute from the entry. We had the first two options in v1, delete was there but a bit flaky IIRC. > Another thing: with user friendliness/experience. I think users will > actually suffer a little after being migrated, because they will have to > take all of these steps: > > 1) login to the migration page > 2) use kinit > 3) if their password doesn't meet IPA password policy, change their > password > 4) go to ipa page, probably won't work > 5) configure their browsers > 6) go to ipa page again, this time it will work :) > > Just saying. > > Pavel Yes, though perhaps in the migration page we should add the "configure Firefox" button so they set their password, configure their browser, quit, kinit, restart and done. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From dpal at redhat.com Mon Nov 2 15:52:56 2009 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 02 Nov 2009 10:52:56 -0500 Subject: [Freeipa-devel] [PATCH] Add DS to IPA migration plugin and password migration page. In-Reply-To: <4AEEFB53.4080808@redhat.com> References: <4AE9CEC3.5060905@redhat.com> <4AEEF86C.9030508@redhat.com> <4AEEFB53.4080808@redhat.com> Message-ID: <4AEF0058.40607@redhat.com> Rob Crittenden wrote: > Pavel Zuna wrote: >> Everyone wrote: >> ... >> A LOT and Thunderbird isn't able to display a thread on a mailing >> list properly. >> >> I did some testing on how much time does it take to migrate "a few" >> users. I started with 10000, but unfortunately my VM can't handle >> that much (always running out of space and I already deleted >> /usr/share/doc :D). >> >> Anyway, I successfully migrated about ~4200 users in 27 minutes using >> the current method. I didn't test it using the IPA commands yet, >> because I ran into the problem of making LDAP data valid for IPA >> commands - it's actually not that easy. We can't pass user passwords >> to them and we also cannot set attributes the command don't support, >> so we have to manually set them using ldap2.update_entry anyway. I >> know that the numbers at the beginning of this paragraph mean nothing >> if I have nothing to compare them to, but I thought you might be >> interested anyway. I'll keep you updated. > > Yes, something we need in baseldap.py is a way to pass in arbitrary > attributes to Add and Modify. There are several modes we need: > > Add a new value to an attribute (this attr may or may not be in the > entry) > Set an attribute to a value (a replace operation) > Remove a value from an attribute. Removing the last value should > remove the attribute from the entry. > > We had the first two options in v1, delete was there but a bit flaky > IIRC. > >> Another thing: with user friendliness/experience. I think users will >> actually suffer a little after being migrated, because they will have >> to take all of these steps: >> >> 1) login to the migration page >> 2) use kinit >> 3) if their password doesn't meet IPA password policy, change their >> password >> 4) go to ipa page, probably won't work >> 5) configure their browsers >> 6) go to ipa page again, this time it will work :) >> >> Just saying. >> >> Pavel > > Yes, though perhaps in the migration page we should add the "configure > Firefox" button so they set their password, configure their browser, > quit, kinit, restart and done. > And please, please, please document it for David. > rob > ------------------------------------------------------------------------ > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jderose at redhat.com Mon Nov 2 21:28:26 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Mon, 02 Nov 2009 21:28:26 +0000 Subject: [Freeipa-devel] [PATCH] jderose 026 ipa-server-install now renders UI assets Message-ID: <1257197306.2491.10.camel@jgd-dsk> This patch changes ipa-server-install so that it renders the UI assets (CSS and JavaScript) and puts them in /var/cache/ipa/assets. It also adds the appropriate Apache config and serves the assets from /ipa-assets/. The assets are accessed using SSL, but there's no reason to protect them with Kerberos. This patch will require some SELinux config changes, but I'll let someone more experienced make these changes (I just saw Rob raise his hand!). I've just been testing with `setenforce permissive`. I've been beating up on these changes on and off for the last week and feel pretty confident they wont break anything. For more info on the approach I'm taking for asset management and use of the "Expires" header, see: http://jderose.fedorapeople.org/assets/current/apidoc/ -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jderose-026-ipa-server-install-now-renders-UI-assets.patch Type: text/x-patch Size: 6177 bytes Desc: not available URL: From jderose at redhat.com Tue Nov 3 08:20:13 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Tue, 03 Nov 2009 01:20:13 -0700 Subject: [Freeipa-devel] [PATCH] 304 hosts requesting certificates In-Reply-To: <4AE8BA7A.6070700@redhat.com> References: <4AE8BA7A.6070700@redhat.com> Message-ID: <1257236413.16763.3.camel@jgd-dsk> On Wed, 2009-10-28 at 17:41 -0400, Rob Crittenden wrote: > I had originally implemented allowing a host to request certificates for > other hosts using the requesting IP address. That was a pretty lousy way > to do it. > > This patch uses the DS ACI system instead. We came up with a clever ACI > that lets hosts listed in the managedBy attribute in the service modify > the userCertificate attribute. So you can use this to delegate which > hosts can request certificates for which services, even for other machines. > > I also re-ordered the request_certificate() method a bit. We want all > the service work done before we do the certificate request. It was > previously adding the service after the cert request was done. This > could mean a failed request if the requestor isn't allowed to add > services. But it is also too late because the cert had already been issued. > > I documented how this works a bit at > http://www.freeipa.org/page/Certificate_Authority > > rob I'm having problems applying this patch: error: install/share/60basev2.ldif: patch does not apply From rcritten at redhat.com Tue Nov 3 14:37:10 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 03 Nov 2009 09:37:10 -0500 Subject: [Freeipa-devel] [PATCH] 304 hosts requesting certificates In-Reply-To: <1257236413.16763.3.camel@jgd-dsk> References: <4AE8BA7A.6070700@redhat.com> <1257236413.16763.3.camel@jgd-dsk> Message-ID: <4AF04016.9020008@redhat.com> Jason Gerard DeRose wrote: > On Wed, 2009-10-28 at 17:41 -0400, Rob Crittenden wrote: >> I had originally implemented allowing a host to request certificates for >> other hosts using the requesting IP address. That was a pretty lousy way >> to do it. >> >> This patch uses the DS ACI system instead. We came up with a clever ACI >> that lets hosts listed in the managedBy attribute in the service modify >> the userCertificate attribute. So you can use this to delegate which >> hosts can request certificates for which services, even for other machines. >> >> I also re-ordered the request_certificate() method a bit. We want all >> the service work done before we do the certificate request. It was >> previously adding the service after the cert request was done. This >> could mean a failed request if the requestor isn't allowed to add >> services. But it is also too late because the cert had already been issued. >> >> I documented how this works a bit at >> http://www.freeipa.org/page/Certificate_Authority >> >> rob > > I'm having problems applying this patch: > > error: install/share/60basev2.ldif: patch does not apply > It was because the syntax of the fqdn attribute in 60basev2.ldif changed and it was in the context of this patch. New patch attached. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-304-2-cert.patch Type: application/mbox Size: 12393 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From jderose at redhat.com Tue Nov 3 17:00:05 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Tue, 03 Nov 2009 10:00:05 -0700 Subject: [Freeipa-devel] [PATCH] 304 hosts requesting certificates In-Reply-To: <4AF04016.9020008@redhat.com> References: <4AE8BA7A.6070700@redhat.com> <1257236413.16763.3.camel@jgd-dsk> <4AF04016.9020008@redhat.com> Message-ID: <1257267605.16763.436.camel@jgd-dsk> On Tue, 2009-11-03 at 09:37 -0500, Rob Crittenden wrote: > Jason Gerard DeRose wrote: > > On Wed, 2009-10-28 at 17:41 -0400, Rob Crittenden wrote: > >> I had originally implemented allowing a host to request certificates for > >> other hosts using the requesting IP address. That was a pretty lousy way > >> to do it. > >> > >> This patch uses the DS ACI system instead. We came up with a clever ACI > >> that lets hosts listed in the managedBy attribute in the service modify > >> the userCertificate attribute. So you can use this to delegate which > >> hosts can request certificates for which services, even for other machines. > >> > >> I also re-ordered the request_certificate() method a bit. We want all > >> the service work done before we do the certificate request. It was > >> previously adding the service after the cert request was done. This > >> could mean a failed request if the requestor isn't allowed to add > >> services. But it is also too late because the cert had already been issued. > >> > >> I documented how this works a bit at > >> http://www.freeipa.org/page/Certificate_Authority > >> > >> rob > > > > I'm having problems applying this patch: > > > > error: install/share/60basev2.ldif: patch does not apply > > > > It was because the syntax of the fqdn attribute in 60basev2.ldif changed > and it was in the context of this patch. New patch attached. > > rob ack. pushed to master. From jderose at redhat.com Tue Nov 3 18:32:37 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Tue, 03 Nov 2009 11:32:37 -0700 Subject: [Freeipa-devel] [PATCH] 305 remove a principal from a keytab In-Reply-To: <4AEB4CD1.9010707@redhat.com> References: <4AEB4CD1.9010707@redhat.com> Message-ID: <1257273157.16763.440.camel@jgd-dsk> On Fri, 2009-10-30 at 16:30 -0400, Rob Crittenden wrote: > I wasn't able to find a command-line program to remove principals from a > keytab so I wrote my own. ktutil can do it but it doesn't take > command-line arguments. Java ships a utility named ktab but adding a > huge dependency for one app seem a bit much :-) > > In any case, this program has 2 modes: > > 1. Given a keytab and a principal, remove all entries of that principal > from the keytab. This removes all versions and encryption types. > > 2. Given a realm remove all principals in that realm. I cheat a little > and insert an @ before the principal name because all this really does > is a strstr() to see if the principal in the keytab is in the realm > provided. > > This utility will be added to the ipa-client-uninstall script at some > point to clean up /etc/krb5.keytab. > > rob ack. Rob walked me through its use on #freeipa, and it works as advertised. From rcritten at redhat.com Tue Nov 3 20:29:13 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 03 Nov 2009 15:29:13 -0500 Subject: [Freeipa-devel] [PATCH] 306 selinux policy for assets Message-ID: <4AF09299.4060304@redhat.com> This adds some SELinux policy for /var/cache/ipa/assets and /var/cache/ipa/sessions. I've also disabled Indexing on /ipa-assets and removed the deprecated IPADebug option. This effectively removes ipa_webgui too. I've left the directory there for now (mostly for reference). rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-306-selinux.patch Type: application/mbox Size: 5735 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Tue Nov 3 20:29:31 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 03 Nov 2009 15:29:31 -0500 Subject: [Freeipa-devel] [PATCH] jderose 026 ipa-server-install now renders UI assets In-Reply-To: <1257197306.2491.10.camel@jgd-dsk> References: <1257197306.2491.10.camel@jgd-dsk> Message-ID: <4AF092AB.9090106@redhat.com> Jason Gerard DeRose wrote: > This patch changes ipa-server-install so that it renders the UI assets > (CSS and JavaScript) and puts them in /var/cache/ipa/assets. It also > adds the appropriate Apache config and serves the assets > from /ipa-assets/. The assets are accessed using SSL, but there's no > reason to protect them with Kerberos. > > This patch will require some SELinux config changes, but I'll let > someone more experienced make these changes (I just saw Rob raise his > hand!). I've just been testing with `setenforce permissive`. > > I've been beating up on these changes on and off for the last week and > feel pretty confident they wont break anything. For more info on the > approach I'm taking for asset management and use of the "Expires" > header, see: > > http://jderose.fedorapeople.org/assets/current/apidoc/ Ack. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From nalin at redhat.com Tue Nov 3 23:37:11 2009 From: nalin at redhat.com (Nalin Dahyabhai) Date: Tue, 3 Nov 2009 18:37:11 -0500 Subject: [Freeipa-devel] Certificate enrollment, principal names Message-ID: <20091103233711.GC9409@redhat.com> I think I'm getting closer to having certmonger (the provider of the ipa-getcert command) be useful enough to throw certificate enrollment requests at the IPA server, and I've got a couple of questions about how the server decides what it will issue and what it puts in the certificates that it issues. First, how we are we going to be expected to pass, to the server, information about the certificate we'd like it to issue? Until now, I've been storing the principal name in a subjectAltName value in an extensionRequest attribute in the signing request. I can actually put quite a bit of information in extensionRequests. It's not a lot of trouble to also provide that information along with the signing request (as 1.9.0 expects, at least for the Kerberos principal name), but if the server's going to be taking direction from the client on any of these things, it might be more future-proof if it could parse the request and validate its contents directly. This would make adding a requested dnsName subjectAltName possible without breaking any of the existing interfaces -- the client could request it, or not, or more than one value, and the server would pick and choose from everything that the client requested when deciding what to put into a certificate. The other question is about client authorization: have we set down the rules about which client identities are allowed to request what, and what they get? I ask because I think that we'll have to use the client host's identity (via creds obtained using its keytab) to handle the case where the connection to the CA doesn't become available until long after the admin's logged out, but when I try that now, requests submitted using the host's identity are being denied by the access control mechanisms. Anyone have some insight to share here? Thanks, Nalin From jderose at redhat.com Wed Nov 4 11:53:20 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Wed, 04 Nov 2009 04:53:20 -0700 Subject: [Freeipa-devel] [PATCH] 306 selinux policy for assets In-Reply-To: <4AF09299.4060304@redhat.com> References: <4AF09299.4060304@redhat.com> Message-ID: <1257335600.20441.0.camel@jgd-dsk> On Tue, 2009-11-03 at 15:29 -0500, Rob Crittenden wrote: > This adds some SELinux policy for /var/cache/ipa/assets and > /var/cache/ipa/sessions. > > I've also disabled Indexing on /ipa-assets and removed the deprecated > IPADebug option. > > This effectively removes ipa_webgui too. I've left the directory there > for now (mostly for reference). > > rob ack. I pushed this and my 026 patch to master. From rcritten at redhat.com Wed Nov 4 14:46:24 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 04 Nov 2009 09:46:24 -0500 Subject: [Freeipa-devel] [PATCH] 307 enforce scalar Message-ID: <4AF193C0.8030606@redhat.com> _convert_scalar() should not handle tuples/lists (by definition). A parameter may be mutivalued but even then _convert_scalar() gets the values one at a time. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-307-scalar.patch Type: text/x-patch Size: 3449 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From jdennis at redhat.com Wed Nov 4 17:39:10 2009 From: jdennis at redhat.com (John Dennis) Date: Wed, 04 Nov 2009 12:39:10 -0500 Subject: [Freeipa-devel] GeneralizedTime v.s datetime.datetime in XMLRPC Message-ID: <4AF1BC3E.6090403@redhat.com> In parameters.py we define a GeneralizedTime object to be used as an XMLRPC parameter. Why? * XMLRPC defines the dateTime.iso8601 parameter value type for passing date/time information * Python has good support for date/time processing in it's datetime module * Python's xmlrpclib supports both xmlrpclib.DateTime and datetime.datetime objects. * Python's xmlrpclib can be configured to use datetime.datetime objects intead of xmlrpclib.DateTime objects if you pass use_datetime=True when invoking xmlrpclib.loads(), however we don't do that. Why? * ISO 8601 is an internet standard for passing date time information between cooperating network entities. However GeneralizedTime is only valid in a subset of binary protocols (primarily LDAP and PKI) Given that ISO 8601 is the preferred standard, that's it is directly supported by XMLRPC, is compatible with datetime.datetime and the fact datetime.datetime has excellent support in Python shouldn't we be using datetime.datetime for all our date/time information and only convert to and from GeneralizedTime for the subset of interfaces which require GeneralizedTime? -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Wed Nov 4 20:52:40 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 04 Nov 2009 15:52:40 -0500 Subject: [Freeipa-devel] GeneralizedTime v.s datetime.datetime in XMLRPC In-Reply-To: <4AF1BC3E.6090403@redhat.com> References: <4AF1BC3E.6090403@redhat.com> Message-ID: <4AF1E998.8020800@redhat.com> John Dennis wrote: > In parameters.py we define a GeneralizedTime object to be used as an > XMLRPC parameter. Why? GeneralizedTime isn't defined as an XML-RPC paramter, just an IPA one and XML-RPC just comes along for the ride. We only needed support for RFC 4517. > * XMLRPC defines the dateTime.iso8601 parameter value type for passing > date/time information > > * Python has good support for date/time processing in it's datetime module > > * Python's xmlrpclib supports both xmlrpclib.DateTime and > datetime.datetime objects. > > * Python's xmlrpclib can be configured to use datetime.datetime objects > intead of xmlrpclib.DateTime objects if you pass use_datetime=True when > invoking xmlrpclib.loads(), however we don't do that. Why? Never needed dates. > * ISO 8601 is an internet standard for passing date time information > between cooperating network entities. However GeneralizedTime is only > valid in a subset of binary protocols (primarily LDAP and PKI) And it is LDAP we end up speaking. > Given that ISO 8601 is the preferred standard, that's it is directly > supported by XMLRPC, is compatible with datetime.datetime and the fact > datetime.datetime has excellent support in Python shouldn't we be using > datetime.datetime for all our date/time information and only convert to > and from GeneralizedTime for the subset of interfaces which require > GeneralizedTime? > This could always be revisited but at the time we didn't need full-blown support and in fact don't want timezone information. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Wed Nov 4 21:04:02 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 04 Nov 2009 16:04:02 -0500 Subject: [Freeipa-devel] [PATCH] 308 manage arbitrary attributes Message-ID: <4AF1EC42.5000603@redhat.com> This adds 2 new parameters, --setattr and --addattr and lets you manage whatever attribute you want (within the given set of objectclasses). Both take a name/value pair. --setattr sets the attribute to the given value --addattr adds the value to an attribute. Can be used to manage multi-valued attributes For example: ipa user-mod --addattr=postalcode=90210 jsmith If the attribute to be modified is an another param then the value is silently dropped. You can include multiples of these on a single command-line: ipa user-mod --addattr=postalcode=20601 --addattr=postalcode=30330 jsmith Setting an attribute to "" deletes it: ipa user-mod --setattr=postalcode= jsmith rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Wed Nov 4 21:39:40 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 04 Nov 2009 16:39:40 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <20091103233711.GC9409@redhat.com> References: <20091103233711.GC9409@redhat.com> Message-ID: <4AF1F49C.3080900@redhat.com> Nalin Dahyabhai wrote: > I think I'm getting closer to having certmonger (the provider of the > ipa-getcert command) be useful enough to throw certificate enrollment > requests at the IPA server, and I've got a couple of questions about how > the server decides what it will issue and what it puts in the > certificates that it issues. > > First, how we are we going to be expected to pass, to the server, > information about the certificate we'd like it to issue? > > Until now, I've been storing the principal name in a subjectAltName > value in an extensionRequest attribute in the signing request. I can > actually put quite a bit of information in extensionRequests. > > It's not a lot of trouble to also provide that information along with > the signing request (as 1.9.0 expects, at least for the Kerberos > principal name), but if the server's going to be taking direction from > the client on any of these things, it might be more future-proof if it > could parse the request and validate its contents directly. > > This would make adding a requested dnsName subjectAltName possible > without breaking any of the existing interfaces -- the client could > request it, or not, or more than one value, and the server would pick > and choose from everything that the client requested when deciding what > to put into a certificate. > > The other question is about client authorization: have we set down the > rules about which client identities are allowed to request what, and > what they get? There are 2 options. There is a rolegroup called certadmin. Members of this role are allowed to call cert-request, others will be rejected. Alternatively you can specify which host(s) can request a certificate for a given service. Use the service-add-member command to add hosts that can request certs for it. > I ask because I think that we'll have to use the client host's identity > (via creds obtained using its keytab) to handle the case where the > connection to the CA doesn't become available until long after the > admin's logged out, but when I try that now, requests submitted using > the host's identity are being denied by the access control mechanisms. Yes, the first access method is really designed for users/administrators. The second if you are binding as a host. > Anyone have some insight to share here? A couple of tidbits: - In 1.9.0 we'll issue a certificate for any subject requested. dogtag has a fix that we will be able to use once it's released that will let us pull the CN from the request and use just that with the subject and use a fixed value for the rest. - The management framework doesn't do anything to the CSR right now, it literally just passes it onto the CA for processing. - The whole ugly client IP thing has been ripped out post 1.9.0. - I still compare the hostname in the subject with the hostname of the service. This is unfortunately currently broken in python 2.4-based systems. - I'm not opposed to including more "stuff" into the CSR itself we just need to be sure the average admin who doesn't want to use certmonger can still make a request too. Right now the bar is pretty low to understanding what is required IMHO with the exception of pasting in the ugly one-line CSR :-( rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From jdennis at redhat.com Wed Nov 4 21:40:24 2009 From: jdennis at redhat.com (John Dennis) Date: Wed, 04 Nov 2009 16:40:24 -0500 Subject: [Freeipa-devel] GeneralizedTime v.s datetime.datetime in XMLRPC In-Reply-To: <4AF1E998.8020800@redhat.com> References: <4AF1BC3E.6090403@redhat.com> <4AF1E998.8020800@redhat.com> Message-ID: <4AF1F4C8.4040701@redhat.com> On 11/04/2009 03:52 PM, Rob Crittenden wrote: > John Dennis wrote: >> In parameters.py we define a GeneralizedTime object to be used as an >> XMLRPC parameter. Why? > > GeneralizedTime isn't defined as an XML-RPC paramter, just an IPA one > and XML-RPC just comes along for the ride. We only needed support for > RFC 4517. Exactly, that's the problem. GeneralizedTime is not known to anybody who speaks XMLRPC, but iso8601 is known to anybody who does speak XMLRPC, and since GeneralizedTime is a subset of iso8601 anybody requiring GeneralizeTime can convert to GeneralizedTime from iso8601. Whenever possible we should stay within the definitions of the specifications, since XMLRPC already has a type for iso8601 there is no need to introduce a private type into XMLRPC which would be known only to select parties. > >> * XMLRPC defines the dateTime.iso8601 parameter value type for passing >> date/time information >> >> * Python has good support for date/time processing in it's datetime >> module >> >> * Python's xmlrpclib supports both xmlrpclib.DateTime and >> datetime.datetime objects. >> >> * Python's xmlrpclib can be configured to use datetime.datetime >> objects intead of xmlrpclib.DateTime objects if you pass >> use_datetime=True when invoking xmlrpclib.loads(), however we don't do >> that. Why? > > Never needed dates. This has nothing to do with needing dates, rather it's an issue of which date/time object xmlrpclib will use. xmlrpclib apparently was written prior to the introduction of datetime.datetime so it created its own date/time type called DateTime. The introduction of datetime.datetime should supersede xmlrpclib.DateTime but it was left as the default for backwards compatibility. We have no need for that backward compatibility, we should be representing date/time information in Python's native datetime.datetime object. > >> * ISO 8601 is an internet standard for passing date time information >> between cooperating network entities. However GeneralizedTime is only >> valid in a subset of binary protocols (primarily LDAP and PKI) > > And it is LDAP we end up speaking. No, our API is not speaking native LDAP, we're providing an abstraction layer over LDAP. > >> Given that ISO 8601 is the preferred standard, that's it is directly >> supported by XMLRPC, is compatible with datetime.datetime and the fact >> datetime.datetime has excellent support in Python shouldn't we be >> using datetime.datetime for all our date/time information and only >> convert to and from GeneralizedTime for the subset of interfaces which >> require GeneralizedTime? >> > > This could always be revisited but at the time we didn't need full-blown > support and in fact don't want timezone information. datetime.datetime can be use with and without timezone information. We probably want to establish a convention that all date/time information is exchanged in UTC (effectively the same thing as omitting timezone information, if that's what you meant). datetime.datetime handles UTC trivially. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From nalin at redhat.com Thu Nov 5 00:16:58 2009 From: nalin at redhat.com (Nalin Dahyabhai) Date: Wed, 4 Nov 2009 19:16:58 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF1F49C.3080900@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> Message-ID: <20091105001658.GA7176@redhat.com> On Wed, Nov 04, 2009 at 04:39:40PM -0500, Rob Crittenden wrote: > Alternatively you can specify which host(s) can request a > certificate for a given service. Use the service-add-member command > to add hosts that can request certs for it. That sounds reasonable. Is this new post-1.9.0? I can add members to various groups, but there's no service-add-member command yet. > A couple of tidbits: > > - In 1.9.0 we'll issue a certificate for any subject requested. > dogtag has a fix that we will be able to use once it's released that > will let us pull the CN from the request and use just that with the > subject and use a fixed value for the rest. That sounds good -- the default request subject is just 'CN=hostname' unless it's told different. > - The management framework doesn't do anything to the CSR right now, > it literally just passes it onto the CA for processing. > - The whole ugly client IP thing has been ripped out post 1.9.0. > - I still compare the hostname in the subject with the hostname of > the service. This is unfortunately currently broken in python > 2.4-based systems. If we're requiring that every certificate has an associated principal name, then ensuring it agrees with the hostname in the subject field makes a lot of sense. I'd kind of like to see both a dnsName and a Kerberos principal name added to the subjectAltName fields in the issued certificate, but that's as much because we can as anything else. > - I'm not opposed to including more "stuff" into the CSR itself we > just need to be sure the average admin who doesn't want to use > certmonger can still make a request too. NSS's certutil can trivially add dns and email subjectAltName (SAN) values and extendedKeyUsage (EKU) values. I don't see a flag for adding a Kerberos principal name. OpenSSL's req command doesn't do most of that by default, but the configuration file can be used to tell it to do any of that. It could be scripted, for sure. > Right now the bar is pretty > low to understanding what is required IMHO with the exception of > pasting in the ugly one-line CSR :-( Yeah, it took me a while to figure out that that was how we were supposed to pass it in. Thanks, Nalin From awnuk at redhat.com Thu Nov 5 01:25:47 2009 From: awnuk at redhat.com (Andrew Wnuk) Date: Wed, 04 Nov 2009 17:25:47 -0800 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <20091105001658.GA7176@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> Message-ID: <4AF2299B.7010007@redhat.com> On 11/04/09 16:16, Nalin Dahyabhai wrote: > On Wed, Nov 04, 2009 at 04:39:40PM -0500, Rob Crittenden wrote: > >> Alternatively you can specify which host(s) can request a >> certificate for a given service. Use the service-add-member command >> to add hosts that can request certs for it. >> > That sounds reasonable. Is this new post-1.9.0? I can add members to > various groups, but there's no service-add-member command yet. > > >> A couple of tidbits: >> >> - In 1.9.0 we'll issue a certificate for any subject requested. >> dogtag has a fix that we will be able to use once it's released that >> will let us pull the CN from the request and use just that with the >> subject and use a fixed value for the rest. >> > That sounds good -- the default request subject is just 'CN=hostname' > unless it's told different. > > >> - The management framework doesn't do anything to the CSR right now, >> it literally just passes it onto the CA for processing. >> - The whole ugly client IP thing has been ripped out post 1.9.0. >> - I still compare the hostname in the subject with the hostname of >> the service. This is unfortunately currently broken in python >> 2.4-based systems. >> > If we're requiring that every certificate has an associated principal > name, then ensuring it agrees with the hostname in the subject field > makes a lot of sense. I'd kind of like to see both a dnsName and a > Kerberos principal name added to the subjectAltName fields in the issued > certificate, but that's as much because we can as anything else. > > >> - I'm not opposed to including more "stuff" into the CSR itself we >> just need to be sure the average admin who doesn't want to use >> certmonger can still make a request too. >> > NSS's certutil can trivially add dns and email subjectAltName (SAN) > values and extendedKeyUsage (EKU) values. I don't see a flag for adding > a Kerberos principal name. OpenSSL's req command doesn't do most of > that by default, but the configuration file can be used to tell it to do > any of that. It could be scripted, for sure. > > >> Right now the bar is pretty >> low to understanding what is required IMHO with the exception of >> pasting in the ugly one-line CSR :-( >> > Yeah, it took me a while to figure out that that was how we were > supposed to pass it in. > Passing entire CSR as a parameter to ipa command could avoided if XML-RPC framework would provide pre and post processing callbacks on the client side. Parameters could be used to describe CSR (instead of passing entire CSR), pre-processing callback could generate CSR based on provided description, then XML-RPC call could submit generated CSR and finally post-processing callback could properly place obtained certificate. Regards, Andrew From dpal at redhat.com Thu Nov 5 01:29:39 2009 From: dpal at redhat.com (Dmitri Pal) Date: Wed, 04 Nov 2009 20:29:39 -0500 Subject: [Freeipa-devel] GeneralizedTime v.s datetime.datetime in XMLRPC In-Reply-To: <4AF1F4C8.4040701@redhat.com> References: <4AF1BC3E.6090403@redhat.com> <4AF1E998.8020800@redhat.com> <4AF1F4C8.4040701@redhat.com> Message-ID: <4AF22A83.4010907@redhat.com> Rob, is it a big problem to do it right? It seems like we are cutting corners a bit and I understand why but my general experience tells me that these things are just time bombs waiting to explode. Do we really want to leave them there or we should clean it up before we release? I know it is more work but these things just do not smell right... Is there any argument against following XMLRPC standards? How much would it take to make the changes and who would be the right person to do them? John Dennis wrote: > On 11/04/2009 03:52 PM, Rob Crittenden wrote: >> John Dennis wrote: >>> In parameters.py we define a GeneralizedTime object to be used as an >>> XMLRPC parameter. Why? >> >> GeneralizedTime isn't defined as an XML-RPC paramter, just an IPA one >> and XML-RPC just comes along for the ride. We only needed support for >> RFC 4517. > > Exactly, that's the problem. GeneralizedTime is not known to anybody > who speaks XMLRPC, but iso8601 is known to anybody who does speak > XMLRPC, and since GeneralizedTime is a subset of iso8601 anybody > requiring GeneralizeTime can convert to GeneralizedTime from iso8601. > Whenever possible we should stay within the definitions of the > specifications, since XMLRPC already has a type for iso8601 there is > no need to introduce a private type into XMLRPC which would be known > only to select parties. > >> >>> * XMLRPC defines the dateTime.iso8601 parameter value type for passing >>> date/time information >>> >>> * Python has good support for date/time processing in it's datetime >>> module >>> >>> * Python's xmlrpclib supports both xmlrpclib.DateTime and >>> datetime.datetime objects. >>> >>> * Python's xmlrpclib can be configured to use datetime.datetime >>> objects intead of xmlrpclib.DateTime objects if you pass >>> use_datetime=True when invoking xmlrpclib.loads(), however we don't do >>> that. Why? >> >> Never needed dates. > > This has nothing to do with needing dates, rather it's an issue of > which date/time object xmlrpclib will use. xmlrpclib apparently was > written prior to the introduction of datetime.datetime so it created > its own date/time type called DateTime. The introduction of > datetime.datetime should supersede xmlrpclib.DateTime but it was left > as the default for backwards compatibility. We have no need for that > backward compatibility, we should be representing date/time > information in Python's native datetime.datetime object. > >> >>> * ISO 8601 is an internet standard for passing date time information >>> between cooperating network entities. However GeneralizedTime is only >>> valid in a subset of binary protocols (primarily LDAP and PKI) >> >> And it is LDAP we end up speaking. > > No, our API is not speaking native LDAP, we're providing an > abstraction layer over LDAP. > >> >>> Given that ISO 8601 is the preferred standard, that's it is directly >>> supported by XMLRPC, is compatible with datetime.datetime and the fact >>> datetime.datetime has excellent support in Python shouldn't we be >>> using datetime.datetime for all our date/time information and only >>> convert to and from GeneralizedTime for the subset of interfaces which >>> require GeneralizedTime? >>> >> >> This could always be revisited but at the time we didn't need full-blown >> support and in fact don't want timezone information. > > datetime.datetime can be use with and without timezone information. We > probably want to establish a convention that all date/time information > is exchanged in UTC (effectively the same thing as omitting timezone > information, if that's what you meant). datetime.datetime handles UTC > trivially. > -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From dpal at redhat.com Thu Nov 5 01:31:02 2009 From: dpal at redhat.com (Dmitri Pal) Date: Wed, 04 Nov 2009 20:31:02 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF2299B.7010007@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> Message-ID: <4AF22AD6.7090004@redhat.com> Andrew Wnuk wrote: > On 11/04/09 16:16, Nalin Dahyabhai wrote: >> On Wed, Nov 04, 2009 at 04:39:40PM -0500, Rob Crittenden wrote: >> >>> Alternatively you can specify which host(s) can request a >>> certificate for a given service. Use the service-add-member command >>> to add hosts that can request certs for it. >>> >> That sounds reasonable. Is this new post-1.9.0? I can add members to >> various groups, but there's no service-add-member command yet. >> >> >>> A couple of tidbits: >>> >>> - In 1.9.0 we'll issue a certificate for any subject requested. >>> dogtag has a fix that we will be able to use once it's released that >>> will let us pull the CN from the request and use just that with the >>> subject and use a fixed value for the rest. >>> >> That sounds good -- the default request subject is just 'CN=hostname' >> unless it's told different. >> >> >>> - The management framework doesn't do anything to the CSR right now, >>> it literally just passes it onto the CA for processing. >>> - The whole ugly client IP thing has been ripped out post 1.9.0. >>> - I still compare the hostname in the subject with the hostname of >>> the service. This is unfortunately currently broken in python >>> 2.4-based systems. >>> >> If we're requiring that every certificate has an associated principal >> name, then ensuring it agrees with the hostname in the subject field >> makes a lot of sense. I'd kind of like to see both a dnsName and a >> Kerberos principal name added to the subjectAltName fields in the issued >> certificate, but that's as much because we can as anything else. >> >> >>> - I'm not opposed to including more "stuff" into the CSR itself we >>> just need to be sure the average admin who doesn't want to use >>> certmonger can still make a request too. >>> >> NSS's certutil can trivially add dns and email subjectAltName (SAN) >> values and extendedKeyUsage (EKU) values. I don't see a flag for adding >> a Kerberos principal name. OpenSSL's req command doesn't do most of >> that by default, but the configuration file can be used to tell it to do >> any of that. It could be scripted, for sure. >> >> >>> Right now the bar is pretty >>> low to understanding what is required IMHO with the exception of >>> pasting in the ugly one-line CSR :-( >>> >> Yeah, it took me a while to figure out that that was how we were >> supposed to pass it in. >> > Passing entire CSR as a parameter to ipa command could avoided if > XML-RPC framework would provide pre and post processing callbacks on > the client side. Parameters could be used to describe CSR (instead of > passing entire CSR), pre-processing callback could generate CSR based > on provided description, then XML-RPC call could submit generated CSR > and finally post-processing callback could properly place obtained > certificate. > I though we talked about these callbacks a year ago and planned to do them. Was this work ever finished? > Regards, > Andrew > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jderose at redhat.com Thu Nov 5 05:19:38 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Wed, 04 Nov 2009 22:19:38 -0700 Subject: [Freeipa-devel] [PATCH] 308 manage arbitrary attributes In-Reply-To: <4AF1EC42.5000603@redhat.com> References: <4AF1EC42.5000603@redhat.com> Message-ID: <1257398378.20441.1.camel@jgd-dsk> Oops, was this missing the attachment? ;) On Wed, 2009-11-04 at 16:04 -0500, Rob Crittenden wrote: > This adds 2 new parameters, --setattr and --addattr and lets you manage > whatever attribute you want (within the given set of objectclasses). > > Both take a name/value pair. > > --setattr sets the attribute to the given value > --addattr adds the value to an attribute. Can be used to manage > multi-valued attributes > > For example: > > ipa user-mod --addattr=postalcode=90210 jsmith > > If the attribute to be modified is an another param then the value is > silently dropped. > > You can include multiples of these on a single command-line: > > ipa user-mod --addattr=postalcode=20601 --addattr=postalcode=30330 jsmith > > Setting an attribute to "" deletes it: > > ipa user-mod --setattr=postalcode= jsmith > > rob > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From awnuk at redhat.com Thu Nov 5 05:44:09 2009 From: awnuk at redhat.com (Andrew Wnuk) Date: Wed, 04 Nov 2009 21:44:09 -0800 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF22AD6.7090004@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> Message-ID: <4AF26629.8040906@redhat.com> On 11/04/2009 05:31 PM, Dmitri Pal wrote: > Andrew Wnuk wrote: > >> On 11/04/09 16:16, Nalin Dahyabhai wrote: >> >>> On Wed, Nov 04, 2009 at 04:39:40PM -0500, Rob Crittenden wrote: >>> >>> >>>> Alternatively you can specify which host(s) can request a >>>> certificate for a given service. Use the service-add-member command >>>> to add hosts that can request certs for it. >>>> >>>> >>> That sounds reasonable. Is this new post-1.9.0? I can add members to >>> various groups, but there's no service-add-member command yet. >>> >>> >>> >>>> A couple of tidbits: >>>> >>>> - In 1.9.0 we'll issue a certificate for any subject requested. >>>> dogtag has a fix that we will be able to use once it's released that >>>> will let us pull the CN from the request and use just that with the >>>> subject and use a fixed value for the rest. >>>> >>>> >>> That sounds good -- the default request subject is just 'CN=hostname' >>> unless it's told different. >>> >>> >>> >>>> - The management framework doesn't do anything to the CSR right now, >>>> it literally just passes it onto the CA for processing. >>>> - The whole ugly client IP thing has been ripped out post 1.9.0. >>>> - I still compare the hostname in the subject with the hostname of >>>> the service. This is unfortunately currently broken in python >>>> 2.4-based systems. >>>> >>>> >>> If we're requiring that every certificate has an associated principal >>> name, then ensuring it agrees with the hostname in the subject field >>> makes a lot of sense. I'd kind of like to see both a dnsName and a >>> Kerberos principal name added to the subjectAltName fields in the issued >>> certificate, but that's as much because we can as anything else. >>> >>> >>> >>>> - I'm not opposed to including more "stuff" into the CSR itself we >>>> just need to be sure the average admin who doesn't want to use >>>> certmonger can still make a request too. >>>> >>>> >>> NSS's certutil can trivially add dns and email subjectAltName (SAN) >>> values and extendedKeyUsage (EKU) values. I don't see a flag for adding >>> a Kerberos principal name. OpenSSL's req command doesn't do most of >>> that by default, but the configuration file can be used to tell it to do >>> any of that. It could be scripted, for sure. >>> >>> >>> >>>> Right now the bar is pretty >>>> low to understanding what is required IMHO with the exception of >>>> pasting in the ugly one-line CSR :-( >>>> >>>> >>> Yeah, it took me a while to figure out that that was how we were >>> supposed to pass it in. >>> >>> >> Passing entire CSR as a parameter to ipa command could avoided if >> XML-RPC framework would provide pre and post processing callbacks on >> the client side. Parameters could be used to describe CSR (instead of >> passing entire CSR), pre-processing callback could generate CSR based >> on provided description, then XML-RPC call could submit generated CSR >> and finally post-processing callback could properly place obtained >> certificate. >> >> > I though we talked about these callbacks a year ago and planned to do them. > Was this work ever finished? > > Jason, are there any dedicated pre and post processing callbacks available on the client side without using get_default for pre-processing and and display callback for post-processing? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jderose at redhat.com Thu Nov 5 07:05:59 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Thu, 05 Nov 2009 00:05:59 -0700 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF2299B.7010007@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> Message-ID: <1257404759.7352.14.camel@jgd-dsk> On Wed, 2009-11-04 at 17:25 -0800, Andrew Wnuk wrote: > On 11/04/09 16:16, Nalin Dahyabhai wrote: > > On Wed, Nov 04, 2009 at 04:39:40PM -0500, Rob Crittenden wrote: > > > >> Alternatively you can specify which host(s) can request a > >> certificate for a given service. Use the service-add-member command > >> to add hosts that can request certs for it. > >> > > That sounds reasonable. Is this new post-1.9.0? I can add members to > > various groups, but there's no service-add-member command yet. > > > > > >> A couple of tidbits: > >> > >> - In 1.9.0 we'll issue a certificate for any subject requested. > >> dogtag has a fix that we will be able to use once it's released that > >> will let us pull the CN from the request and use just that with the > >> subject and use a fixed value for the rest. > >> > > That sounds good -- the default request subject is just 'CN=hostname' > > unless it's told different. > > > > > >> - The management framework doesn't do anything to the CSR right now, > >> it literally just passes it onto the CA for processing. > >> - The whole ugly client IP thing has been ripped out post 1.9.0. > >> - I still compare the hostname in the subject with the hostname of > >> the service. This is unfortunately currently broken in python > >> 2.4-based systems. > >> > > If we're requiring that every certificate has an associated principal > > name, then ensuring it agrees with the hostname in the subject field > > makes a lot of sense. I'd kind of like to see both a dnsName and a > > Kerberos principal name added to the subjectAltName fields in the issued > > certificate, but that's as much because we can as anything else. > > > > > >> - I'm not opposed to including more "stuff" into the CSR itself we > >> just need to be sure the average admin who doesn't want to use > >> certmonger can still make a request too. > >> > > NSS's certutil can trivially add dns and email subjectAltName (SAN) > > values and extendedKeyUsage (EKU) values. I don't see a flag for adding > > a Kerberos principal name. OpenSSL's req command doesn't do most of > > that by default, but the configuration file can be used to tell it to do > > any of that. It could be scripted, for sure. > > > > > >> Right now the bar is pretty > >> low to understanding what is required IMHO with the exception of > >> pasting in the ugly one-line CSR :-( > >> > > Yeah, it took me a while to figure out that that was how we were > > supposed to pass it in. > > > Passing entire CSR as a parameter to ipa command could avoided if > XML-RPC framework would provide pre and post processing callbacks on the > client side. Parameters could be used to describe CSR (instead of > passing entire CSR), pre-processing callback could generate CSR based on > provided description, then XML-RPC call could submit generated CSR and > finally post-processing callback could properly place obtained certificate. > > Regards, > Andrew We have had pre-processing available for a long time... the Param system provides plenty of flexibility to do what you are talking about by filling in any of the per-param callback slots (normalizer, default_from, create_default). For more on this, see: http://freeipa.org/developer-docs/ipalib.parameters.Param-class.html#get_default Plus you can always override Command.run() to add pre/post steps around a call to super(Foo, self).run() And Pavel's encoder/decoder stuff allows pre/post hooks as well. Not all of this is documented nicely (or at all), so feel free to ask question or pester us for documentation. But, just to set the record strait, ipalib does have these features. ;) From jderose at redhat.com Thu Nov 5 07:18:32 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Thu, 05 Nov 2009 00:18:32 -0700 Subject: [Freeipa-devel] GeneralizedTime v.s datetime.datetime in XMLRPC In-Reply-To: <4AF22A83.4010907@redhat.com> References: <4AF1BC3E.6090403@redhat.com> <4AF1E998.8020800@redhat.com> <4AF1F4C8.4040701@redhat.com> <4AF22A83.4010907@redhat.com> Message-ID: <1257405512.7352.26.camel@jgd-dsk> On Wed, 2009-11-04 at 20:29 -0500, Dmitri Pal wrote: > Rob, is it a big problem to do it right? > It seems like we are cutting corners a bit and I understand why but my > general experience tells me that these things are just time bombs > waiting to explode. > Do we really want to leave them there or we should clean it up before we > release? > I know it is more work but these things just do not smell right... > Is there any argument against following XMLRPC standards? > How much would it take to make the changes and who would be the right > person to do them? I've been toying a bit with a replacement for the xmlrpclib.dumps() loads() functions, which leave quite a bit to be desired, especially with regard to their magic guessing as to whether an str is character data or binary data. It wouldn't be much work to replace these two functions, especially as ipalib strictly uses `str` for binary data, `unicode` for decoded character data (I did this because it's the "right thing to do" and it allows us to gracefully transition to Python 3 when the times comes). As I've already daydreamed through the design issues, I bet I could do it in 3 days. > John Dennis wrote: > > On 11/04/2009 03:52 PM, Rob Crittenden wrote: > >> John Dennis wrote: > >>> In parameters.py we define a GeneralizedTime object to be used as an > >>> XMLRPC parameter. Why? > >> > >> GeneralizedTime isn't defined as an XML-RPC paramter, just an IPA one > >> and XML-RPC just comes along for the ride. We only needed support for > >> RFC 4517. > > > > Exactly, that's the problem. GeneralizedTime is not known to anybody > > who speaks XMLRPC, but iso8601 is known to anybody who does speak > > XMLRPC, and since GeneralizedTime is a subset of iso8601 anybody > > requiring GeneralizeTime can convert to GeneralizedTime from iso8601. > > Whenever possible we should stay within the definitions of the > > specifications, since XMLRPC already has a type for iso8601 there is > > no need to introduce a private type into XMLRPC which would be known > > only to select parties. > > > >> > >>> * XMLRPC defines the dateTime.iso8601 parameter value type for passing > >>> date/time information > >>> > >>> * Python has good support for date/time processing in it's datetime > >>> module > >>> > >>> * Python's xmlrpclib supports both xmlrpclib.DateTime and > >>> datetime.datetime objects. > >>> > >>> * Python's xmlrpclib can be configured to use datetime.datetime > >>> objects intead of xmlrpclib.DateTime objects if you pass > >>> use_datetime=True when invoking xmlrpclib.loads(), however we don't do > >>> that. Why? > >> > >> Never needed dates. > > > > This has nothing to do with needing dates, rather it's an issue of > > which date/time object xmlrpclib will use. xmlrpclib apparently was > > written prior to the introduction of datetime.datetime so it created > > its own date/time type called DateTime. The introduction of > > datetime.datetime should supersede xmlrpclib.DateTime but it was left > > as the default for backwards compatibility. We have no need for that > > backward compatibility, we should be representing date/time > > information in Python's native datetime.datetime object. > > > >> > >>> * ISO 8601 is an internet standard for passing date time information > >>> between cooperating network entities. However GeneralizedTime is only > >>> valid in a subset of binary protocols (primarily LDAP and PKI) > >> > >> And it is LDAP we end up speaking. > > > > No, our API is not speaking native LDAP, we're providing an > > abstraction layer over LDAP. > > > >> > >>> Given that ISO 8601 is the preferred standard, that's it is directly > >>> supported by XMLRPC, is compatible with datetime.datetime and the fact > >>> datetime.datetime has excellent support in Python shouldn't we be > >>> using datetime.datetime for all our date/time information and only > >>> convert to and from GeneralizedTime for the subset of interfaces which > >>> require GeneralizedTime? > >>> > >> > >> This could always be revisited but at the time we didn't need full-blown > >> support and in fact don't want timezone information. > > > > datetime.datetime can be use with and without timezone information. We > > probably want to establish a convention that all date/time information > > is exchanged in UTC (effectively the same thing as omitting timezone > > information, if that's what you meant). datetime.datetime handles UTC > > trivially. > > > > From jderose at redhat.com Thu Nov 5 07:48:38 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Thu, 05 Nov 2009 00:48:38 -0700 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF26629.8040906@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> Message-ID: <1257407318.7352.51.camel@jgd-dsk> On Wed, 2009-11-04 at 21:44 -0800, Andrew Wnuk wrote: > On 11/04/2009 05:31 PM, Dmitri Pal wrote: > > Andrew Wnuk wrote: > > > > > On 11/04/09 16:16, Nalin Dahyabhai wrote: > > > > > > > On Wed, Nov 04, 2009 at 04:39:40PM -0500, Rob Crittenden wrote: > > > > > > > > > > > > > Alternatively you can specify which host(s) can request a > > > > > certificate for a given service. Use the service-add-member command > > > > > to add hosts that can request certs for it. > > > > > > > > > > > > > > That sounds reasonable. Is this new post-1.9.0? I can add members to > > > > various groups, but there's no service-add-member command yet. > > > > > > > > > > > > > > > > > A couple of tidbits: > > > > > > > > > > - In 1.9.0 we'll issue a certificate for any subject requested. > > > > > dogtag has a fix that we will be able to use once it's released that > > > > > will let us pull the CN from the request and use just that with the > > > > > subject and use a fixed value for the rest. > > > > > > > > > > > > > > That sounds good -- the default request subject is just 'CN=hostname' > > > > unless it's told different. > > > > > > > > > > > > > > > > > - The management framework doesn't do anything to the CSR right now, > > > > > it literally just passes it onto the CA for processing. > > > > > - The whole ugly client IP thing has been ripped out post 1.9.0. > > > > > - I still compare the hostname in the subject with the hostname of > > > > > the service. This is unfortunately currently broken in python > > > > > 2.4-based systems. > > > > > > > > > > > > > > If we're requiring that every certificate has an associated principal > > > > name, then ensuring it agrees with the hostname in the subject field > > > > makes a lot of sense. I'd kind of like to see both a dnsName and a > > > > Kerberos principal name added to the subjectAltName fields in the issued > > > > certificate, but that's as much because we can as anything else. > > > > > > > > > > > > > > > > > - I'm not opposed to including more "stuff" into the CSR itself we > > > > > just need to be sure the average admin who doesn't want to use > > > > > certmonger can still make a request too. > > > > > > > > > > > > > > NSS's certutil can trivially add dns and email subjectAltName (SAN) > > > > values and extendedKeyUsage (EKU) values. I don't see a flag for adding > > > > a Kerberos principal name. OpenSSL's req command doesn't do most of > > > > that by default, but the configuration file can be used to tell it to do > > > > any of that. It could be scripted, for sure. > > > > > > > > > > > > > > > > > Right now the bar is pretty > > > > > low to understanding what is required IMHO with the exception of > > > > > pasting in the ugly one-line CSR :-( > > > > > > > > > > > > > > Yeah, it took me a while to figure out that that was how we were > > > > supposed to pass it in. > > > > > > > > > > > Passing entire CSR as a parameter to ipa command could avoided if > > > XML-RPC framework would provide pre and post processing callbacks on > > > the client side. Parameters could be used to describe CSR (instead of > > > passing entire CSR), pre-processing callback could generate CSR based > > > on provided description, then XML-RPC call could submit generated CSR > > > and finally post-processing callback could properly place obtained > > > certificate. > > > > > > > > I though we talked about these callbacks a year ago and planned to do them. > > Was this work ever finished? > > > > > Jason, are there any dedicated pre and post processing callbacks > available on the client side without using get_default for > pre-processing and and display callback for post-processing? Sorry, I didn't read the thread in order... I swore we added a post-process hook that took the same signature as output_for_cli(), but I don't see it now. Anyway, there are several ways you can approach this depending on your exact needs, and we can add explicit hooks once the use case is well defined. First of all, why will using a create_default callback for the csr not work? I vaguely remember you explaining this before, but I've forgotten the details. As far as generalized pre/post hooks, there are 4 places they could make sense: at the start and end of __call__(), and at the start and end of run(), like this sort of thing: def __call__(self, *args, **options): self.pre_call() self.run(*args, **options) self.post_call() def run(self, *args, **options): self.pre_run() super(Foo, self).run(*args, **options) self.post_run() You have to decide whether your pre hook should happen before or after the normalization/conversion/validation round or after it. We are adding return value validation also, so the same thing needs to be decided for the post hook. Also, does the post hook need to change the return value, or just take some action based on it? From rcritten at redhat.com Thu Nov 5 13:59:35 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 05 Nov 2009 08:59:35 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF2299B.7010007@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> Message-ID: <4AF2DA47.4060206@redhat.com> Andrew Wnuk wrote: > On 11/04/09 16:16, Nalin Dahyabhai wrote: >> On Wed, Nov 04, 2009 at 04:39:40PM -0500, Rob Crittenden wrote: >> >>> Alternatively you can specify which host(s) can request a >>> certificate for a given service. Use the service-add-member command >>> to add hosts that can request certs for it. >>> >> That sounds reasonable. Is this new post-1.9.0? I can add members to >> various groups, but there's no service-add-member command yet. >> >> >>> A couple of tidbits: >>> >>> - In 1.9.0 we'll issue a certificate for any subject requested. >>> dogtag has a fix that we will be able to use once it's released that >>> will let us pull the CN from the request and use just that with the >>> subject and use a fixed value for the rest. >>> >> That sounds good -- the default request subject is just 'CN=hostname' >> unless it's told different. >> >> >>> - The management framework doesn't do anything to the CSR right now, >>> it literally just passes it onto the CA for processing. >>> - The whole ugly client IP thing has been ripped out post 1.9.0. >>> - I still compare the hostname in the subject with the hostname of >>> the service. This is unfortunately currently broken in python >>> 2.4-based systems. >>> >> If we're requiring that every certificate has an associated principal >> name, then ensuring it agrees with the hostname in the subject field >> makes a lot of sense. I'd kind of like to see both a dnsName and a >> Kerberos principal name added to the subjectAltName fields in the issued >> certificate, but that's as much because we can as anything else. >> >> >>> - I'm not opposed to including more "stuff" into the CSR itself we >>> just need to be sure the average admin who doesn't want to use >>> certmonger can still make a request too. >>> >> NSS's certutil can trivially add dns and email subjectAltName (SAN) >> values and extendedKeyUsage (EKU) values. I don't see a flag for adding >> a Kerberos principal name. OpenSSL's req command doesn't do most of >> that by default, but the configuration file can be used to tell it to do >> any of that. It could be scripted, for sure. >> >> >>> Right now the bar is pretty >>> low to understanding what is required IMHO with the exception of >>> pasting in the ugly one-line CSR :-( >>> >> Yeah, it took me a while to figure out that that was how we were >> supposed to pass it in. >> > Passing entire CSR as a parameter to ipa command could avoided if > XML-RPC framework would provide pre and post processing callbacks on the > client side. Parameters could be used to describe CSR (instead of > passing entire CSR), pre-processing callback could generate CSR based on > provided description, then XML-RPC call could submit generated CSR and > finally post-processing callback could properly place obtained certificate. > > Regards, > Andrew As I've said before, it is possible to do this right now by overriding execute(). Use the variable self.env.in_server to determine if you are on the client or server side. For the client you do the CSR generation, call forward(), then load the cert when forward() returns. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Thu Nov 5 14:22:16 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 05 Nov 2009 09:22:16 -0500 Subject: [Freeipa-devel] GeneralizedTime v.s datetime.datetime in XMLRPC In-Reply-To: <4AF22A83.4010907@redhat.com> References: <4AF1BC3E.6090403@redhat.com> <4AF1E998.8020800@redhat.com> <4AF1F4C8.4040701@redhat.com> <4AF22A83.4010907@redhat.com> Message-ID: <4AF2DF98.2090709@redhat.com> Dmitri Pal wrote: > Rob, is it a big problem to do it right? I don't consider what Pavel did to be wrong. > It seems like we are cutting corners a bit and I understand why but my > general experience tells me that these things are just time bombs > waiting to explode. > Do we really want to leave them there or we should clean it up before we > release? > I know it is more work but these things just do not smell right... > Is there any argument against following XMLRPC standards? This has nothing to do with XML-RPC. I think the name GeneralizedTime is a misnomer. This isn't used as a pure date string. Instead it can hold absolute times and periodic information. For example, here is a perfectly valid value: periodic daily 1200 absolute 200912161032 ~ 200912161033 Those date values are enforced using the GeneralizedTime format but that's as close as things get. > How much would it take to make the changes and who would be the right > person to do them? I don't think there is anything to do other than perhaps to rename the type (and perhaps add some additional error handling and documentation). We need a set of unit tests for this too. rob > > John Dennis wrote: >> On 11/04/2009 03:52 PM, Rob Crittenden wrote: >>> John Dennis wrote: >>>> In parameters.py we define a GeneralizedTime object to be used as an >>>> XMLRPC parameter. Why? >>> GeneralizedTime isn't defined as an XML-RPC paramter, just an IPA one >>> and XML-RPC just comes along for the ride. We only needed support for >>> RFC 4517. >> Exactly, that's the problem. GeneralizedTime is not known to anybody >> who speaks XMLRPC, but iso8601 is known to anybody who does speak >> XMLRPC, and since GeneralizedTime is a subset of iso8601 anybody >> requiring GeneralizeTime can convert to GeneralizedTime from iso8601. >> Whenever possible we should stay within the definitions of the >> specifications, since XMLRPC already has a type for iso8601 there is >> no need to introduce a private type into XMLRPC which would be known >> only to select parties. >> >>>> * XMLRPC defines the dateTime.iso8601 parameter value type for passing >>>> date/time information >>>> >>>> * Python has good support for date/time processing in it's datetime >>>> module >>>> >>>> * Python's xmlrpclib supports both xmlrpclib.DateTime and >>>> datetime.datetime objects. >>>> >>>> * Python's xmlrpclib can be configured to use datetime.datetime >>>> objects intead of xmlrpclib.DateTime objects if you pass >>>> use_datetime=True when invoking xmlrpclib.loads(), however we don't do >>>> that. Why? >>> Never needed dates. >> This has nothing to do with needing dates, rather it's an issue of >> which date/time object xmlrpclib will use. xmlrpclib apparently was >> written prior to the introduction of datetime.datetime so it created >> its own date/time type called DateTime. The introduction of >> datetime.datetime should supersede xmlrpclib.DateTime but it was left >> as the default for backwards compatibility. We have no need for that >> backward compatibility, we should be representing date/time >> information in Python's native datetime.datetime object. >> >>>> * ISO 8601 is an internet standard for passing date time information >>>> between cooperating network entities. However GeneralizedTime is only >>>> valid in a subset of binary protocols (primarily LDAP and PKI) >>> And it is LDAP we end up speaking. >> No, our API is not speaking native LDAP, we're providing an >> abstraction layer over LDAP. >> >>>> Given that ISO 8601 is the preferred standard, that's it is directly >>>> supported by XMLRPC, is compatible with datetime.datetime and the fact >>>> datetime.datetime has excellent support in Python shouldn't we be >>>> using datetime.datetime for all our date/time information and only >>>> convert to and from GeneralizedTime for the subset of interfaces which >>>> require GeneralizedTime? >>>> >>> This could always be revisited but at the time we didn't need full-blown >>> support and in fact don't want timezone information. >> datetime.datetime can be use with and without timezone information. We >> probably want to establish a convention that all date/time information >> is exchanged in UTC (effectively the same thing as omitting timezone >> information, if that's what you meant). datetime.datetime handles UTC >> trivially. >> > > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From dpal at redhat.com Thu Nov 5 15:21:57 2009 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 05 Nov 2009 10:21:57 -0500 Subject: [Freeipa-devel] GeneralizedTime v.s datetime.datetime in XMLRPC In-Reply-To: <4AF2DF98.2090709@redhat.com> References: <4AF1BC3E.6090403@redhat.com> <4AF1E998.8020800@redhat.com> <4AF1F4C8.4040701@redhat.com> <4AF22A83.4010907@redhat.com> <4AF2DF98.2090709@redhat.com> Message-ID: <4AF2ED95.3040207@redhat.com> Rob Crittenden wrote: > Dmitri Pal wrote: >> Rob, is it a big problem to do it right? > > I don't consider what Pavel did to be wrong. > >> It seems like we are cutting corners a bit and I understand why but my >> general experience tells me that these things are just time bombs >> waiting to explode. >> Do we really want to leave them there or we should clean it up before we >> release? >> I know it is more work but these things just do not smell right... >> Is there any argument against following XMLRPC standards? > > This has nothing to do with XML-RPC. I think the name GeneralizedTime > is a misnomer. This isn't used as a pure date string. Instead it can > hold absolute times and periodic information. For example, here is a > perfectly valid value: > > periodic daily 1200 > > absolute 200912161032 ~ 200912161033 > > Those date values are enforced using the GeneralizedTime format but > that's as close as things get. > Well this makes much more sense now. So we are passing a generalized "time rule" that contains grammar that we defined? Then sure it should be renamed to avoid confusion and treated as a unicode string. >> How much would it take to make the changes and who would be the right >> person to do them? > > I don't think there is anything to do other than perhaps to rename the > type (and perhaps add some additional error handling and > documentation). We need a set of unit tests for this too. > > rob > >> >> John Dennis wrote: >>> On 11/04/2009 03:52 PM, Rob Crittenden wrote: >>>> John Dennis wrote: >>>>> In parameters.py we define a GeneralizedTime object to be used as an >>>>> XMLRPC parameter. Why? >>>> GeneralizedTime isn't defined as an XML-RPC paramter, just an IPA one >>>> and XML-RPC just comes along for the ride. We only needed support for >>>> RFC 4517. >>> Exactly, that's the problem. GeneralizedTime is not known to anybody >>> who speaks XMLRPC, but iso8601 is known to anybody who does speak >>> XMLRPC, and since GeneralizedTime is a subset of iso8601 anybody >>> requiring GeneralizeTime can convert to GeneralizedTime from iso8601. >>> Whenever possible we should stay within the definitions of the >>> specifications, since XMLRPC already has a type for iso8601 there is >>> no need to introduce a private type into XMLRPC which would be known >>> only to select parties. >>> >>>>> * XMLRPC defines the dateTime.iso8601 parameter value type for >>>>> passing >>>>> date/time information >>>>> >>>>> * Python has good support for date/time processing in it's datetime >>>>> module >>>>> >>>>> * Python's xmlrpclib supports both xmlrpclib.DateTime and >>>>> datetime.datetime objects. >>>>> >>>>> * Python's xmlrpclib can be configured to use datetime.datetime >>>>> objects intead of xmlrpclib.DateTime objects if you pass >>>>> use_datetime=True when invoking xmlrpclib.loads(), however we >>>>> don't do >>>>> that. Why? >>>> Never needed dates. >>> This has nothing to do with needing dates, rather it's an issue of >>> which date/time object xmlrpclib will use. xmlrpclib apparently was >>> written prior to the introduction of datetime.datetime so it created >>> its own date/time type called DateTime. The introduction of >>> datetime.datetime should supersede xmlrpclib.DateTime but it was left >>> as the default for backwards compatibility. We have no need for that >>> backward compatibility, we should be representing date/time >>> information in Python's native datetime.datetime object. >>> >>>>> * ISO 8601 is an internet standard for passing date time information >>>>> between cooperating network entities. However GeneralizedTime is only >>>>> valid in a subset of binary protocols (primarily LDAP and PKI) >>>> And it is LDAP we end up speaking. >>> No, our API is not speaking native LDAP, we're providing an >>> abstraction layer over LDAP. >>> >>>>> Given that ISO 8601 is the preferred standard, that's it is directly >>>>> supported by XMLRPC, is compatible with datetime.datetime and the >>>>> fact >>>>> datetime.datetime has excellent support in Python shouldn't we be >>>>> using datetime.datetime for all our date/time information and only >>>>> convert to and from GeneralizedTime for the subset of interfaces >>>>> which >>>>> require GeneralizedTime? >>>>> >>>> This could always be revisited but at the time we didn't need >>>> full-blown >>>> support and in fact don't want timezone information. >>> datetime.datetime can be use with and without timezone information. We >>> probably want to establish a convention that all date/time information >>> is exchanged in UTC (effectively the same thing as omitting timezone >>> information, if that's what you meant). datetime.datetime handles UTC >>> trivially. >>> >> >> > -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From dpal at redhat.com Thu Nov 5 15:24:32 2009 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 05 Nov 2009 10:24:32 -0500 Subject: [Freeipa-devel] GeneralizedTime v.s datetime.datetime in XMLRPC In-Reply-To: <1257405512.7352.26.camel@jgd-dsk> References: <4AF1BC3E.6090403@redhat.com> <4AF1E998.8020800@redhat.com> <4AF1F4C8.4040701@redhat.com> <4AF22A83.4010907@redhat.com> <1257405512.7352.26.camel@jgd-dsk> Message-ID: <4AF2EE30.9070201@redhat.com> Jason Gerard DeRose wrote: > On Wed, 2009-11-04 at 20:29 -0500, Dmitri Pal wrote: > >> Rob, is it a big problem to do it right? >> It seems like we are cutting corners a bit and I understand why but my >> general experience tells me that these things are just time bombs >> waiting to explode. >> Do we really want to leave them there or we should clean it up before we >> release? >> I know it is more work but these things just do not smell right... >> Is there any argument against following XMLRPC standards? >> How much would it take to make the changes and who would be the right >> person to do them? >> > > I've been toying a bit with a replacement for the xmlrpclib.dumps() > loads() functions, which leave quite a bit to be desired, especially > with regard to their magic guessing as to whether an str is character > data or binary data. > > It wouldn't be much work to replace these two functions, especially as > ipalib strictly uses `str` for binary data, `unicode` for decoded > character data (I did this because it's the "right thing to do" and it > allows us to gracefully transition to Python 3 when the times comes). > > As I've already daydreamed through the design issues, I bet I could do > it in 3 days. > > It seems to be related but not exactly the same thing that was discussed about GeneralizedTime. So is it something that can be refactored if we have time after we have a UI that people can test? >> John Dennis wrote: >> >>> On 11/04/2009 03:52 PM, Rob Crittenden wrote: >>> >>>> John Dennis wrote: >>>> >>>>> In parameters.py we define a GeneralizedTime object to be used as an >>>>> XMLRPC parameter. Why? >>>>> >>>> GeneralizedTime isn't defined as an XML-RPC paramter, just an IPA one >>>> and XML-RPC just comes along for the ride. We only needed support for >>>> RFC 4517. >>>> >>> Exactly, that's the problem. GeneralizedTime is not known to anybody >>> who speaks XMLRPC, but iso8601 is known to anybody who does speak >>> XMLRPC, and since GeneralizedTime is a subset of iso8601 anybody >>> requiring GeneralizeTime can convert to GeneralizedTime from iso8601. >>> Whenever possible we should stay within the definitions of the >>> specifications, since XMLRPC already has a type for iso8601 there is >>> no need to introduce a private type into XMLRPC which would be known >>> only to select parties. >>> >>> >>>>> * XMLRPC defines the dateTime.iso8601 parameter value type for passing >>>>> date/time information >>>>> >>>>> * Python has good support for date/time processing in it's datetime >>>>> module >>>>> >>>>> * Python's xmlrpclib supports both xmlrpclib.DateTime and >>>>> datetime.datetime objects. >>>>> >>>>> * Python's xmlrpclib can be configured to use datetime.datetime >>>>> objects intead of xmlrpclib.DateTime objects if you pass >>>>> use_datetime=True when invoking xmlrpclib.loads(), however we don't do >>>>> that. Why? >>>>> >>>> Never needed dates. >>>> >>> This has nothing to do with needing dates, rather it's an issue of >>> which date/time object xmlrpclib will use. xmlrpclib apparently was >>> written prior to the introduction of datetime.datetime so it created >>> its own date/time type called DateTime. The introduction of >>> datetime.datetime should supersede xmlrpclib.DateTime but it was left >>> as the default for backwards compatibility. We have no need for that >>> backward compatibility, we should be representing date/time >>> information in Python's native datetime.datetime object. >>> >>> >>>>> * ISO 8601 is an internet standard for passing date time information >>>>> between cooperating network entities. However GeneralizedTime is only >>>>> valid in a subset of binary protocols (primarily LDAP and PKI) >>>>> >>>> And it is LDAP we end up speaking. >>>> >>> No, our API is not speaking native LDAP, we're providing an >>> abstraction layer over LDAP. >>> >>> >>>>> Given that ISO 8601 is the preferred standard, that's it is directly >>>>> supported by XMLRPC, is compatible with datetime.datetime and the fact >>>>> datetime.datetime has excellent support in Python shouldn't we be >>>>> using datetime.datetime for all our date/time information and only >>>>> convert to and from GeneralizedTime for the subset of interfaces which >>>>> require GeneralizedTime? >>>>> >>>>> >>>> This could always be revisited but at the time we didn't need full-blown >>>> support and in fact don't want timezone information. >>>> >>> datetime.datetime can be use with and without timezone information. We >>> probably want to establish a convention that all date/time information >>> is exchanged in UTC (effectively the same thing as omitting timezone >>> information, if that's what you meant). datetime.datetime handles UTC >>> trivially. >>> >>> >> > > -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From nalin at redhat.com Thu Nov 5 15:28:39 2009 From: nalin at redhat.com (Nalin Dahyabhai) Date: Thu, 5 Nov 2009 10:28:39 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF26629.8040906@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> Message-ID: <20091105152839.GA17324@redhat.com> On Wed, Nov 04, 2009 at 09:44:09PM -0800, Andrew Wnuk wrote: > Passing entire CSR as a parameter to ipa command could avoided if > XML-RPC framework would provide pre and post processing callbacks on > the client side. Parameters could be used to describe CSR (instead of > passing entire CSR), pre-processing callback could generate CSR based > on provided description, then XML-RPC call could submit generated CSR > and finally post-processing callback could properly place obtained > certificate. The CSR is usually signed with the client's key, so we can't generate it at the server (unless we're doing server-side key generation, which I don't think we're doing yet). We could pass a public key by itself with other bits of info alongside, but then you lose the signing of it. In the general case, you really want the client-supplied data to be signed if the approval process is going to use any of it. Besides, CSRs are just how this stuff's done, and the reformatting at the client end can be done with an awk script. I don't want to add more work for ourselves by trying to change that part of it. Cheers, Nalin From dpal at redhat.com Thu Nov 5 15:39:00 2009 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 05 Nov 2009 10:39:00 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <20091105152839.GA17324@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> Message-ID: <4AF2F194.1050302@redhat.com> Nalin Dahyabhai wrote: > On Wed, Nov 04, 2009 at 09:44:09PM -0800, Andrew Wnuk wrote: > >> Passing entire CSR as a parameter to ipa command could avoided if >> XML-RPC framework would provide pre and post processing callbacks on >> the client side. Parameters could be used to describe CSR (instead of >> passing entire CSR), pre-processing callback could generate CSR based >> on provided description, then XML-RPC call could submit generated CSR >> and finally post-processing callback could properly place obtained >> certificate. >> > > The CSR is usually signed with the client's key, so we can't generate it > at the server (unless we're doing server-side key generation, which I > don't think we're doing yet). > > This is not what Andrew was saying. As far as I understand the CSR is a blob of signed information that client sends to server to get a certificate. The CSR needs to be created somehow. I do not know how it is created but it seems that there are two steps: * create CSR * pass CSR to the command Andrew suggested to do it in one step (and I agree with him). However we have two different use cases: * Admin wants to create a cert - in this case our standard CLI is called (and the discussion about hooks is related to this part) * Certmonger asks for cert -in this case it does not use our standard XML-RPC framework but rather prepares the request itself and sends it over XML-RPC So how do we do or plan to do it in each of the two use cases? It seems that the most user friendly approach would be to ask user for arguments, using these arguments generate CSR on the client and then pass it to the server. In both cases it should be done in one step by invoking other tools and utils (if any) needed to prepare CSR from within the command. And I correct? > We could pass a public key by itself with other bits of info alongside, > but then you lose the signing of it. In the general case, you really > want the client-supplied data to be signed if the approval process is > going to use any of it. > > Besides, CSRs are just how this stuff's done, and the reformatting at > the client end can be done with an awk script. I don't want to add more > work for ourselves by trying to change that part of it. > > Cheers, > > Nalin > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel > > > -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From nalin at redhat.com Thu Nov 5 17:04:43 2009 From: nalin at redhat.com (Nalin Dahyabhai) Date: Thu, 5 Nov 2009 12:04:43 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF2F194.1050302@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> Message-ID: <20091105170443.GB17324@redhat.com> On Thu, Nov 05, 2009 at 10:39:00AM -0500, Dmitri Pal wrote: > This is not what Andrew was saying. > As far as I understand the CSR is a blob of signed information that > client sends to server > to get a certificate. The CSR needs to be created somehow. I do not know > how it is created > but it seems that there are two steps: > * create CSR > * pass CSR to the command > > Andrew suggested to do it in one step (and I agree with him). > However we have two different use cases: > * Admin wants to create a cert - in this case our standard CLI is called > (and the discussion about hooks > is related to this part) > * Certmonger asks for cert -in this case it does not use our standard > XML-RPC framework but rather > prepares the request itself and sends it over XML-RPC > > So how do we do or plan to do it in each of the two use cases? > It seems that the most user friendly approach would be to ask user for > arguments, > using these arguments generate CSR on the client and then pass it to the > server. > In both cases it should be done in one step by invoking other tools and > utils (if any) > needed to prepare CSR from within the command. > > And I correct? Yes, and that's what we do now -- when you tell certmonger you want a certificate, it handles generating a CSR with the key you provide (or one it generates and saves, if there isn't already one in the location where you told it to look) and the settings you supply, or with default settings if you don't provide any. It then goes about the business of getting a certificate, usually from a CA, using the CSR. For IPA, that's done with the XMLRPC call to the cert_request() method. At the moment, we also have to send the principal name that we've encoded into the CSR as a separate string parameter to the XMLRPC request that we send to IPA. If we find we need more info later, we'll either need to add another argument to the request (and my reading of the XMLRPC spec doesn't tell me if that's considered an incompatible change or not) or teach the server to dig it out of the CSR. Nalin From jdennis at redhat.com Thu Nov 5 17:37:05 2009 From: jdennis at redhat.com (John Dennis) Date: Thu, 05 Nov 2009 12:37:05 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <20091105170443.GB17324@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> Message-ID: <4AF30D41.8060400@redhat.com> On 11/05/2009 12:04 PM, Nalin Dahyabhai wrote: > On Thu, Nov 05, 2009 at 10:39:00AM -0500, Dmitri Pal wrote: >> This is not what Andrew was saying. >> As far as I understand the CSR is a blob of signed information that >> client sends to server >> to get a certificate. The CSR needs to be created somehow. I do not know >> how it is created >> but it seems that there are two steps: >> * create CSR >> * pass CSR to the command >> >> Andrew suggested to do it in one step (and I agree with him). >> However we have two different use cases: >> * Admin wants to create a cert - in this case our standard CLI is called >> (and the discussion about hooks >> is related to this part) >> * Certmonger asks for cert -in this case it does not use our standard >> XML-RPC framework but rather >> prepares the request itself and sends it over XML-RPC >> >> So how do we do or plan to do it in each of the two use cases? >> It seems that the most user friendly approach would be to ask user for >> arguments, >> using these arguments generate CSR on the client and then pass it to the >> server. >> In both cases it should be done in one step by invoking other tools and >> utils (if any) >> needed to prepare CSR from within the command. >> >> And I correct? > > Yes, and that's what we do now -- when you tell certmonger you want a > certificate, it handles generating a CSR with the key you provide (or > one it generates and saves, if there isn't already one in the location > where you told it to look) and the settings you supply, or with default > settings if you don't provide any. It then goes about the business of > getting a certificate, usually from a CA, using the CSR. For IPA, > that's done with the XMLRPC call to the cert_request() method. > > At the moment, we also have to send the principal name that we've > encoded into the CSR as a separate string parameter to the XMLRPC > request that we send to IPA. If we find we need more info later, we'll > either need to add another argument to the request (and my reading of > the XMLRPC spec doesn't tell me if that's considered an incompatible > change or not) or teach the server to dig it out of the CSR. Because of the way we're using XMLRPC in IPA we can add a new parameter at anytime. That's because we don't actually use positional parameters in the traditional sense. Instead our XMLRPC interface takes a one positional parameter which is a struct (e.g. dict) with all the *named* parameters in it as key/value pairs. (O.K. I fudging a bit on the details). Because we're passing *all* arguments as key/value pairs we can add or remove a parameter at will without breaking core XMLRPC. However, both the client and server need to be versioned correctly. It's fine to send a new parameter through XMLRPC but the server has to know to look for it. This is really no different than "teaching the server to extract it from the CSR". In both cases the server has to be aware of the new value. The XMLRPC interface in this instance is a bit of a red herring. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From dpal at redhat.com Thu Nov 5 18:03:07 2009 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 05 Nov 2009 13:03:07 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF30D41.8060400@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> Message-ID: <4AF3135B.2070303@redhat.com> John Dennis wrote: > On 11/05/2009 12:04 PM, Nalin Dahyabhai wrote: >> On Thu, Nov 05, 2009 at 10:39:00AM -0500, Dmitri Pal wrote: >>> This is not what Andrew was saying. >>> As far as I understand the CSR is a blob of signed information that >>> client sends to server >>> to get a certificate. The CSR needs to be created somehow. I do not >>> know >>> how it is created >>> but it seems that there are two steps: >>> * create CSR >>> * pass CSR to the command >>> >>> Andrew suggested to do it in one step (and I agree with him). >>> However we have two different use cases: >>> * Admin wants to create a cert - in this case our standard CLI is >>> called >>> (and the discussion about hooks >>> is related to this part) >>> * Certmonger asks for cert -in this case it does not use our standard >>> XML-RPC framework but rather >>> prepares the request itself and sends it over XML-RPC >>> >>> So how do we do or plan to do it in each of the two use cases? >>> It seems that the most user friendly approach would be to ask user for >>> arguments, >>> using these arguments generate CSR on the client and then pass it to >>> the >>> server. >>> In both cases it should be done in one step by invoking other tools and >>> utils (if any) >>> needed to prepare CSR from within the command. >>> >>> And I correct? >> >> Yes, and that's what we do now -- when you tell certmonger you want a >> certificate, it handles generating a CSR with the key you provide (or >> one it generates and saves, if there isn't already one in the location >> where you told it to look) and the settings you supply, or with default >> settings if you don't provide any. It then goes about the business of >> getting a certificate, usually from a CA, using the CSR. For IPA, >> that's done with the XMLRPC call to the cert_request() method. >> >> At the moment, we also have to send the principal name that we've >> encoded into the CSR as a separate string parameter to the XMLRPC >> request that we send to IPA. If we find we need more info later, we'll >> either need to add another argument to the request (and my reading of >> the XMLRPC spec doesn't tell me if that's considered an incompatible >> change or not) or teach the server to dig it out of the CSR. > > Because of the way we're using XMLRPC in IPA we can add a new > parameter at anytime. That's because we don't actually use positional > parameters in the traditional sense. Instead our XMLRPC interface > takes a one positional parameter which is a struct (e.g. dict) with > all the *named* parameters in it as key/value pairs. (O.K. I fudging a > bit on the details). Because we're passing *all* arguments as > key/value pairs we can add or remove a parameter at will without > breaking core XMLRPC. > > However, both the client and server need to be versioned correctly. > It's fine to send a new parameter through XMLRPC but the server has to > know to look for it. This is really no different than "teaching the > server to extract it from the CSR". In both cases the server has to be > aware of the new value. The XMLRPC interface in this instance is a bit > of a red herring. > Ok so here are the questions I have: We send the CSR from cermonger and the principal name separately. Is this a right approach? Do we send the principal in case admin requests a cert for someone using main CLI. I hope not. Does that mean that server already knows how to dig into the CSR and get the principal out of it? Why we can't reuse same approach then? The more I think about the difference between the admin+CLI use case vs cermonger use case the more I come to conclusion that they should have a lot in common. I.e: a) Server side part of the XML RPC is probably same b) The permissions are checked on the ACI basis so interface and XML-RPC code does not know anything about that part at all. c) Logic of the client is pretty same. The only main difference it seems that certmonger is a C application and admin CLI is python but they should do pretty much same thing: * create CSR * send it to the server * when server responds, process the result (save cert to the file or NSS database or just spit out as based 64 encoded string). Am I oversimplifying things and there is really something different on the server side between processing certmonger's cert request vs processing CLI request initiated by admin? -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Thu Nov 5 18:21:21 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 05 Nov 2009 13:21:21 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF3135B.2070303@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> Message-ID: <4AF317A1.6040406@redhat.com> Dmitri Pal wrote: > John Dennis wrote: >> On 11/05/2009 12:04 PM, Nalin Dahyabhai wrote: >>> On Thu, Nov 05, 2009 at 10:39:00AM -0500, Dmitri Pal wrote: >>>> This is not what Andrew was saying. >>>> As far as I understand the CSR is a blob of signed information that >>>> client sends to server >>>> to get a certificate. The CSR needs to be created somehow. I do not >>>> know >>>> how it is created >>>> but it seems that there are two steps: >>>> * create CSR >>>> * pass CSR to the command >>>> >>>> Andrew suggested to do it in one step (and I agree with him). >>>> However we have two different use cases: >>>> * Admin wants to create a cert - in this case our standard CLI is >>>> called >>>> (and the discussion about hooks >>>> is related to this part) >>>> * Certmonger asks for cert -in this case it does not use our standard >>>> XML-RPC framework but rather >>>> prepares the request itself and sends it over XML-RPC >>>> >>>> So how do we do or plan to do it in each of the two use cases? >>>> It seems that the most user friendly approach would be to ask user for >>>> arguments, >>>> using these arguments generate CSR on the client and then pass it to >>>> the >>>> server. >>>> In both cases it should be done in one step by invoking other tools and >>>> utils (if any) >>>> needed to prepare CSR from within the command. >>>> >>>> And I correct? >>> Yes, and that's what we do now -- when you tell certmonger you want a >>> certificate, it handles generating a CSR with the key you provide (or >>> one it generates and saves, if there isn't already one in the location >>> where you told it to look) and the settings you supply, or with default >>> settings if you don't provide any. It then goes about the business of >>> getting a certificate, usually from a CA, using the CSR. For IPA, >>> that's done with the XMLRPC call to the cert_request() method. >>> >>> At the moment, we also have to send the principal name that we've >>> encoded into the CSR as a separate string parameter to the XMLRPC >>> request that we send to IPA. If we find we need more info later, we'll >>> either need to add another argument to the request (and my reading of >>> the XMLRPC spec doesn't tell me if that's considered an incompatible >>> change or not) or teach the server to dig it out of the CSR. >> Because of the way we're using XMLRPC in IPA we can add a new >> parameter at anytime. That's because we don't actually use positional >> parameters in the traditional sense. Instead our XMLRPC interface >> takes a one positional parameter which is a struct (e.g. dict) with >> all the *named* parameters in it as key/value pairs. (O.K. I fudging a >> bit on the details). Because we're passing *all* arguments as >> key/value pairs we can add or remove a parameter at will without >> breaking core XMLRPC. >> >> However, both the client and server need to be versioned correctly. >> It's fine to send a new parameter through XMLRPC but the server has to >> know to look for it. This is really no different than "teaching the >> server to extract it from the CSR". In both cases the server has to be >> aware of the new value. The XMLRPC interface in this instance is a bit >> of a red herring. >> > Ok so here are the questions I have: > > We send the CSR from cermonger and the principal name separately. Is > this a right approach? > Do we send the principal in case admin requests a cert for someone using > main CLI. I hope not. > Does that mean that server already knows how to dig into the CSR and get > the principal out of it? > Why we can't reuse same approach then? > > The more I think about the difference between the admin+CLI use case vs > cermonger use case > the more I come to conclusion that they should have a lot in common. I.e: > > a) Server side part of the XML RPC is probably same > b) The permissions are checked on the ACI basis so interface and XML-RPC > code does not know anything about that part at all. > c) Logic of the client is pretty same. The only main difference it seems > that certmonger is a C application > and admin CLI is python but they should do pretty much same thing: > * create CSR > * send it to the server > * when server responds, process the result (save cert to the file or NSS > database or just spit out as based 64 encoded string). > > Am I oversimplifying things and there is really something different on > the server side between processing certmonger's cert > request vs processing CLI request initiated by admin? > This is about right. What you're missing is storing the certificate in the service record. To do this we need to know what the target is. Nalin and I simply took two different approaches to sending this. We can easily support either method by making the principal an optional attribute and looking for it in the CSR if not provided (assuming I can get my head around PKCS#10 enough to grab attributes). rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From jderose at redhat.com Thu Nov 5 18:28:14 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Thu, 05 Nov 2009 11:28:14 -0700 Subject: [Freeipa-devel] GeneralizedTime v.s datetime.datetime in XMLRPC In-Reply-To: <4AF2EE30.9070201@redhat.com> References: <4AF1BC3E.6090403@redhat.com> <4AF1E998.8020800@redhat.com> <4AF1F4C8.4040701@redhat.com> <4AF22A83.4010907@redhat.com> <1257405512.7352.26.camel@jgd-dsk> <4AF2EE30.9070201@redhat.com> Message-ID: <1257445695.7352.53.camel@jgd-dsk> On Thu, 2009-11-05 at 10:24 -0500, Dmitri Pal wrote: > Jason Gerard DeRose wrote: > > On Wed, 2009-11-04 at 20:29 -0500, Dmitri Pal wrote: > > > >> Rob, is it a big problem to do it right? > >> It seems like we are cutting corners a bit and I understand why but my > >> general experience tells me that these things are just time bombs > >> waiting to explode. > >> Do we really want to leave them there or we should clean it up before we > >> release? > >> I know it is more work but these things just do not smell right... > >> Is there any argument against following XMLRPC standards? > >> How much would it take to make the changes and who would be the right > >> person to do them? > >> > > > > I've been toying a bit with a replacement for the xmlrpclib.dumps() > > loads() functions, which leave quite a bit to be desired, especially > > with regard to their magic guessing as to whether an str is character > > data or binary data. > > > > It wouldn't be much work to replace these two functions, especially as > > ipalib strictly uses `str` for binary data, `unicode` for decoded > > character data (I did this because it's the "right thing to do" and it > > allows us to gracefully transition to Python 3 when the times comes). > > > > As I've already daydreamed through the design issues, I bet I could do > > it in 3 days. > > > > > > It seems to be related but not exactly the same thing that was discussed > about GeneralizedTime. > So is it something that can be refactored if we have time after we have > a UI that people can test? Yes, this can be deferred till a better time. What we have works okay for now. > >> John Dennis wrote: > >> > >>> On 11/04/2009 03:52 PM, Rob Crittenden wrote: > >>> > >>>> John Dennis wrote: > >>>> > >>>>> In parameters.py we define a GeneralizedTime object to be used as an > >>>>> XMLRPC parameter. Why? > >>>>> > >>>> GeneralizedTime isn't defined as an XML-RPC paramter, just an IPA one > >>>> and XML-RPC just comes along for the ride. We only needed support for > >>>> RFC 4517. > >>>> > >>> Exactly, that's the problem. GeneralizedTime is not known to anybody > >>> who speaks XMLRPC, but iso8601 is known to anybody who does speak > >>> XMLRPC, and since GeneralizedTime is a subset of iso8601 anybody > >>> requiring GeneralizeTime can convert to GeneralizedTime from iso8601. > >>> Whenever possible we should stay within the definitions of the > >>> specifications, since XMLRPC already has a type for iso8601 there is > >>> no need to introduce a private type into XMLRPC which would be known > >>> only to select parties. > >>> > >>> > >>>>> * XMLRPC defines the dateTime.iso8601 parameter value type for passing > >>>>> date/time information > >>>>> > >>>>> * Python has good support for date/time processing in it's datetime > >>>>> module > >>>>> > >>>>> * Python's xmlrpclib supports both xmlrpclib.DateTime and > >>>>> datetime.datetime objects. > >>>>> > >>>>> * Python's xmlrpclib can be configured to use datetime.datetime > >>>>> objects intead of xmlrpclib.DateTime objects if you pass > >>>>> use_datetime=True when invoking xmlrpclib.loads(), however we don't do > >>>>> that. Why? > >>>>> > >>>> Never needed dates. > >>>> > >>> This has nothing to do with needing dates, rather it's an issue of > >>> which date/time object xmlrpclib will use. xmlrpclib apparently was > >>> written prior to the introduction of datetime.datetime so it created > >>> its own date/time type called DateTime. The introduction of > >>> datetime.datetime should supersede xmlrpclib.DateTime but it was left > >>> as the default for backwards compatibility. We have no need for that > >>> backward compatibility, we should be representing date/time > >>> information in Python's native datetime.datetime object. > >>> > >>> > >>>>> * ISO 8601 is an internet standard for passing date time information > >>>>> between cooperating network entities. However GeneralizedTime is only > >>>>> valid in a subset of binary protocols (primarily LDAP and PKI) > >>>>> > >>>> And it is LDAP we end up speaking. > >>>> > >>> No, our API is not speaking native LDAP, we're providing an > >>> abstraction layer over LDAP. > >>> > >>> > >>>>> Given that ISO 8601 is the preferred standard, that's it is directly > >>>>> supported by XMLRPC, is compatible with datetime.datetime and the fact > >>>>> datetime.datetime has excellent support in Python shouldn't we be > >>>>> using datetime.datetime for all our date/time information and only > >>>>> convert to and from GeneralizedTime for the subset of interfaces which > >>>>> require GeneralizedTime? > >>>>> > >>>>> > >>>> This could always be revisited but at the time we didn't need full-blown > >>>> support and in fact don't want timezone information. > >>>> > >>> datetime.datetime can be use with and without timezone information. We > >>> probably want to establish a convention that all date/time information > >>> is exchanged in UTC (effectively the same thing as omitting timezone > >>> information, if that's what you meant). datetime.datetime handles UTC > >>> trivially. > >>> > >>> > >> > > > > > > From dpal at redhat.com Thu Nov 5 18:36:35 2009 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 05 Nov 2009 13:36:35 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF317A1.6040406@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> Message-ID: <4AF31B33.9060806@redhat.com> >> Ok so here are the questions I have: >> >> We send the CSR from cermonger and the principal name separately. Is >> this a right approach? >> Do we send the principal in case admin requests a cert for someone using >> main CLI. I hope not. >> Does that mean that server already knows how to dig into the CSR and get >> the principal out of it? >> Why we can't reuse same approach then? >> >> The more I think about the difference between the admin+CLI use case vs >> cermonger use case >> the more I come to conclusion that they should have a lot in common. >> I.e: >> >> a) Server side part of the XML RPC is probably same >> b) The permissions are checked on the ACI basis so interface and XML-RPC >> code does not know anything about that part at all. >> c) Logic of the client is pretty same. The only main difference it seems >> that certmonger is a C application >> and admin CLI is python but they should do pretty much same thing: >> * create CSR >> * send it to the server >> * when server responds, process the result (save cert to the file or NSS >> database or just spit out as based 64 encoded string). >> >> Am I oversimplifying things and there is really something different on >> the server side between processing certmonger's cert >> request vs processing CLI request initiated by admin? >> > > This is about right. What you're missing is storing the certificate in > the service record. To do this we need to know what the target is. > Yes but it should not be any different. Service principal is just a part of the request and certmonger client can be configured to provide service principal too. So the code on the server should be pretty generic and be able to serve both implementations of clients. It is just that for the CLI case the service target is likely to be provided while for certmonger the service target most like likely is not provided. If it is not provided target defaults to the host principal provided. > Nalin and I simply took two different approaches to sending this. We > can easily support either method by making the principal an optional > attribute and looking for it in the CSR if not provided (assuming I > can get my head around PKCS#10 enough to grab attributes). I hope this can be reconciled. > > rob -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From ssorce at redhat.com Thu Nov 5 19:22:03 2009 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 05 Nov 2009 14:22:03 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF317A1.6040406@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> Message-ID: <1257448924.3553.106.camel@willson.li.ssimo.org> On Thu, 2009-11-05 at 13:21 -0500, Rob Crittenden wrote: > > This is about right. What you're missing is storing the certificate > in > the service record. To do this we need to know what the target is. > > Nalin and I simply took two different approaches to sending this. We > can > easily support either method by making the principal an optional > attribute and looking for it in the CSR if not provided (assuming I > can > get my head around PKCS#10 enough to grab attributes). Given we should prevent "tricks" from people the server side should really parse the CSR and validate it against the ACL IMO. Otherwise do we have any other part that checks that host foo.example.com is asking a certificate for itself and not for bar.example.com ? Simo. -- Simo Sorce * Red Hat, Inc * New York From awnuk at redhat.com Thu Nov 5 19:28:11 2009 From: awnuk at redhat.com (Andrew Wnuk) Date: Thu, 05 Nov 2009 11:28:11 -0800 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <1257448924.3553.106.camel@willson.li.ssimo.org> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> Message-ID: <4AF3274B.6040403@redhat.com> On 11/05/09 11:22, Simo Sorce wrote: > On Thu, 2009-11-05 at 13:21 -0500, Rob Crittenden wrote: > >> This is about right. What you're missing is storing the certificate >> in >> the service record. To do this we need to know what the target is. >> >> Nalin and I simply took two different approaches to sending this. We >> can >> easily support either method by making the principal an optional >> attribute and looking for it in the CSR if not provided (assuming I >> can >> get my head around PKCS#10 enough to grab attributes). >> > Given we should prevent "tricks" from people the server side should > really parse the CSR and validate it against the ACL IMO. > Otherwise do we have any other part that checks that host > foo.example.com is asking a certificate for itself and not for > bar.example.com ? > > Simo. > > CSR is parsed and validated by CA. Andrew From dpal at redhat.com Thu Nov 5 19:30:37 2009 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 05 Nov 2009 14:30:37 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <1257448924.3553.106.camel@willson.li.ssimo.org> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> Message-ID: <4AF327DD.7010209@redhat.com> Simo Sorce wrote: > On Thu, 2009-11-05 at 13:21 -0500, Rob Crittenden wrote: > >> This is about right. What you're missing is storing the certificate >> in >> the service record. To do this we need to know what the target is. >> >> Nalin and I simply took two different approaches to sending this. We >> can >> easily support either method by making the principal an optional >> attribute and looking for it in the CSR if not provided (assuming I >> can >> get my head around PKCS#10 enough to grab attributes). >> > > Given we should prevent "tricks" from people the server side should > really parse the CSR and validate it against the ACL IMO. > Otherwise do we have any other part that checks that host > foo.example.com is asking a certificate for itself and not for > bar.example.com ? > > Simo. > > This is a valid use case and if the ACIs on the server are not configured to allow host "foo" to get certs for host "bar", host "foo" will be denied regardless of the way it asked via CSR or not. -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From ssorce at redhat.com Thu Nov 5 19:34:59 2009 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 05 Nov 2009 14:34:59 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF3274B.6040403@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> <4AF3274B.6040403@redhat.com> Message-ID: <1257449699.3553.107.camel@willson.li.ssimo.org> On Thu, 2009-11-05 at 11:28 -0800, Andrew Wnuk wrote: > On 11/05/09 11:22, Simo Sorce wrote: > > On Thu, 2009-11-05 at 13:21 -0500, Rob Crittenden wrote: > > > >> This is about right. What you're missing is storing the certificate > >> in > >> the service record. To do this we need to know what the target is. > >> > >> Nalin and I simply took two different approaches to sending this. We > >> can > >> easily support either method by making the principal an optional > >> attribute and looking for it in the CSR if not provided (assuming I > >> can > >> get my head around PKCS#10 enough to grab attributes). > >> > > Given we should prevent "tricks" from people the server side should > > really parse the CSR and validate it against the ACL IMO. > > Otherwise do we have any other part that checks that host > > foo.example.com is asking a certificate for itself and not for > > bar.example.com ? > > > > Simo. > > > > > CSR is parsed and validated by CA. How does the CA know "Who" asked for a specific cert ? Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Thu Nov 5 19:37:33 2009 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 05 Nov 2009 14:37:33 -0500 Subject: [Freeipa-devel] DNS schema Message-ID: <4AF3297D.6040508@redhat.com> Hi, Now when we decided to use latest DS that is available from 389 project for IPA v2 we can take advantage of the syntaxes that DS team has added. For example they added numeric syntax that was a part of the original DNS schema we planned for IPA. I remember we replaced the originally planned syntax with some other syntax that is available. Does it make sense to revert to the numeric syntax now or it is not a big deal? I am just concerned that once we will decide to do it the original way because we would think it is the "right" thing to do and will face all sorts of migration issues. Ideas? -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From dpal at redhat.com Thu Nov 5 19:41:07 2009 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 05 Nov 2009 14:41:07 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <1257449699.3553.107.camel@willson.li.ssimo.org> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> <4AF3274B.6040403@redhat.com> <1257449699.3553.107.camel@willson.li.ssimo.org> Message-ID: <4AF32A53.9090804@redhat.com> Simo Sorce wrote: > On Thu, 2009-11-05 at 11:28 -0800, Andrew Wnuk wrote: > >> On 11/05/09 11:22, Simo Sorce wrote: >> >>> On Thu, 2009-11-05 at 13:21 -0500, Rob Crittenden wrote: >>> >>> >>>> This is about right. What you're missing is storing the certificate >>>> in >>>> the service record. To do this we need to know what the target is. >>>> >>>> Nalin and I simply took two different approaches to sending this. We >>>> can >>>> easily support either method by making the principal an optional >>>> attribute and looking for it in the CSR if not provided (assuming I >>>> can >>>> get my head around PKCS#10 enough to grab attributes). >>>> >>>> >>> Given we should prevent "tricks" from people the server side should >>> really parse the CSR and validate it against the ACL IMO. >>> Otherwise do we have any other part that checks that host >>> foo.example.com is asking a certificate for itself and not for >>> bar.example.com ? >>> >>> Simo. >>> >>> >>> >> CSR is parsed and validated by CA. >> > > How does the CA know "Who" asked for a specific cert ? > > Simo. > > The server should look at the identity of the peer who authenticated when the connection was established. It can be admin using his password or a host using his keytab or OTP. I think Rob figured out how to factor this into the aci validation and the code to do it already there. Rob? -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From awnuk at redhat.com Thu Nov 5 19:53:02 2009 From: awnuk at redhat.com (Andrew Wnuk) Date: Thu, 05 Nov 2009 11:53:02 -0800 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <1257449699.3553.107.camel@willson.li.ssimo.org> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> <4AF3274B.6040403@redhat.com> <1257449699.3553.107.camel@willson.li.ssimo.org> Message-ID: <4AF32D1E.4060306@redhat.com> On 11/05/09 11:34, Simo Sorce wrote: > On Thu, 2009-11-05 at 11:28 -0800, Andrew Wnuk wrote: > >> On 11/05/09 11:22, Simo Sorce wrote: >> >>> On Thu, 2009-11-05 at 13:21 -0500, Rob Crittenden wrote: >>> >>> >>>> This is about right. What you're missing is storing the certificate >>>> in >>>> the service record. To do this we need to know what the target is. >>>> >>>> Nalin and I simply took two different approaches to sending this. We >>>> can >>>> easily support either method by making the principal an optional >>>> attribute and looking for it in the CSR if not provided (assuming I >>>> can >>>> get my head around PKCS#10 enough to grab attributes). >>>> >>>> >>> Given we should prevent "tricks" from people the server side should >>> really parse the CSR and validate it against the ACL IMO. >>> Otherwise do we have any other part that checks that host >>> foo.example.com is asking a certificate for itself and not for >>> bar.example.com ? >>> >>> Simo. >>> >>> >>> >> CSR is parsed and validated by CA. >> > How does the CA know "Who" asked for a specific cert ? > > Simo. > > CA authenticates IPA and validates CSR, IPA authenticates and authorizes. Andrew. From ssorce at redhat.com Thu Nov 5 20:57:36 2009 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 05 Nov 2009 15:57:36 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF32D1E.4060306@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> <4AF3274B.6040403@redhat.com> <1257449699.3553.107.camel@willson.li.ssimo.org> <4AF32D1E.4060306@redhat.com> Message-ID: <1257454656.3553.112.camel@willson.li.ssimo.org> On Thu, 2009-11-05 at 11:53 -0800, Andrew Wnuk wrote: > >> CSR is parsed and validated by CA. > >> > > How does the CA know "Who" asked for a specific cert ? > > > > Simo. > > > > > CA authenticates IPA and validates CSR, IPA authenticates and > authorizes. Yes I know, this is the problem. IPA is almost blind about what's in the CSR (which comes straight from the client), and the RA trust IPA to make "correct" requests. So we need to verify within the IPA server that a client is not trying to sneak stuff in fields like subjectAltName and any other field a client may rely on for authentication/authorization purposes. Rob just opened a bug about that. Simo. -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Thu Nov 5 21:00:55 2009 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 05 Nov 2009 16:00:55 -0500 Subject: [Freeipa-devel] DNS schema In-Reply-To: <4AF3297D.6040508@redhat.com> References: <4AF3297D.6040508@redhat.com> Message-ID: <1257454855.3553.115.camel@willson.li.ssimo.org> On Thu, 2009-11-05 at 14:37 -0500, Dmitri Pal wrote: > Hi, > > Now when we decided to use latest DS that is available from 389 project > for IPA v2 we can take advantage of the syntaxes that DS team has added. > For example they added numeric syntax that was a part of the original > DNS schema we planned for IPA. > I remember we replaced the originally planned syntax with some other > syntax that is available. > > Does it make sense to revert to the numeric syntax now or it is not a > big deal? > I am just concerned that once we will decide to do it the original way > because we would think it is the "right" thing to do and will face all > sorts of migration issues. > Ideas? Unless it is a major amount of work we should use the proper syntax. Especially for "standard" schema. Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Thu Nov 5 23:16:15 2009 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 05 Nov 2009 18:16:15 -0500 Subject: [Freeipa-devel] DNS schema In-Reply-To: <1257454855.3553.115.camel@willson.li.ssimo.org> References: <4AF3297D.6040508@redhat.com> <1257454855.3553.115.camel@willson.li.ssimo.org> Message-ID: <4AF35CBF.10701@redhat.com> Simo Sorce wrote: > On Thu, 2009-11-05 at 14:37 -0500, Dmitri Pal wrote: > >> Hi, >> >> Now when we decided to use latest DS that is available from 389 project >> for IPA v2 we can take advantage of the syntaxes that DS team has added. >> For example they added numeric syntax that was a part of the original >> DNS schema we planned for IPA. >> I remember we replaced the originally planned syntax with some other >> syntax that is available. >> >> Does it make sense to revert to the numeric syntax now or it is not a >> big deal? >> I am just concerned that once we will decide to do it the original way >> because we would think it is the "right" thing to do and will face all >> sorts of migration issues. >> Ideas? >> > > Unless it is a major amount of work we should use the proper syntax. > Especially for "standard" schema. > > Simo. > > It should not be a big deal IMO so Martin if you are ok with this I would like to ask Pavel to take it as an action item. -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From pzuna at redhat.com Fri Nov 6 10:46:59 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Fri, 06 Nov 2009 11:46:59 +0100 Subject: [Freeipa-devel] [PATCH] Add 'File' parameter type. Message-ID: <4AF3FEA3.2020305@redhat.com> Accepts filenames and loads file contents as parameter value. In CLI, the 'stdin_if_missing' kwarg can be used to read the file from stdin if no filename has been entered. Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-File-parameter-type.patch Type: application/mbox Size: 3962 bytes Desc: not available URL: From pzuna at redhat.com Fri Nov 6 10:47:55 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Fri, 06 Nov 2009 11:47:55 +0100 Subject: [Freeipa-devel] [PATCH] Use File parameter for CSR in cert_request command plugin. Message-ID: <4AF3FEDB.9080703@redhat.com> Makes use of the new File parameter introduced in my previous patch. Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Use-File-parameter-for-CSR-in-cert_request-command-p.patch Type: application/mbox Size: 2849 bytes Desc: not available URL: From rcritten at redhat.com Fri Nov 6 14:08:50 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 06 Nov 2009 09:08:50 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <1257448924.3553.106.camel@willson.li.ssimo.org> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> Message-ID: <4AF42DF2.2060600@redhat.com> Simo Sorce wrote: > On Thu, 2009-11-05 at 13:21 -0500, Rob Crittenden wrote: >> This is about right. What you're missing is storing the certificate >> in >> the service record. To do this we need to know what the target is. >> >> Nalin and I simply took two different approaches to sending this. We >> can >> easily support either method by making the principal an optional >> attribute and looking for it in the CSR if not provided (assuming I >> can >> get my head around PKCS#10 enough to grab attributes). > > Given we should prevent "tricks" from people the server side should > really parse the CSR and validate it against the ACL IMO. > Otherwise do we have any other part that checks that host > foo.example.com is asking a certificate for itself and not for > bar.example.com ? > > Simo. > When binding using machine credentials, in order to request a certificate for any host they need to be in the managedBy attribute of the target service entry. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From ssorce at redhat.com Fri Nov 6 15:22:51 2009 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 06 Nov 2009 10:22:51 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF42DF2.2060600@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> <4AF42DF2.2060600@redhat.com> Message-ID: <1257520971.3553.137.camel@willson.li.ssimo.org> On Fri, 2009-11-06 at 09:08 -0500, Rob Crittenden wrote: > Simo Sorce wrote: > > On Thu, 2009-11-05 at 13:21 -0500, Rob Crittenden wrote: > >> This is about right. What you're missing is storing the certificate > >> in > >> the service record. To do this we need to know what the target is. > >> > >> Nalin and I simply took two different approaches to sending this. We > >> can > >> easily support either method by making the principal an optional > >> attribute and looking for it in the CSR if not provided (assuming I > >> can > >> get my head around PKCS#10 enough to grab attributes). > > > > Given we should prevent "tricks" from people the server side should > > really parse the CSR and validate it against the ACL IMO. > > Otherwise do we have any other part that checks that host > > foo.example.com is asking a certificate for itself and not for > > bar.example.com ? > > > > Simo. > > > > When binding using machine credentials, in order to request a > certificate for any host they need to be in the managedBy attribute of > the target service entry. I know, but I was referring to stuff like subjectAltName. Not sure if we need to test it against an ACL to be honest, but we need to validate that as well (and any other attribute that can affect client behavior). Whether this is done in IPA or in the CA is not really important as long as it is done by a component that have enough information to determine what is ok or not, depending on the "user" requesting the new cert. For example admin may be allowed to stuff just any random crap in subjectAltName but maybe a host shouldn't be. Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Fri Nov 6 16:16:17 2009 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 06 Nov 2009 11:16:17 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <1257520971.3553.137.camel@willson.li.ssimo.org> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> <4AF42DF2.2060600@redhat.com> <1257520971.3553.137.camel@willson.li.ssimo.org> Message-ID: <4AF44BD1.9090100@redhat.com> Simo Sorce wrote: > On Fri, 2009-11-06 at 09:08 -0500, Rob Crittenden wrote: > >> Simo Sorce wrote: >> >>> On Thu, 2009-11-05 at 13:21 -0500, Rob Crittenden wrote: >>> >>>> This is about right. What you're missing is storing the certificate >>>> in >>>> the service record. To do this we need to know what the target is. >>>> >>>> Nalin and I simply took two different approaches to sending this. We >>>> can >>>> easily support either method by making the principal an optional >>>> attribute and looking for it in the CSR if not provided (assuming I >>>> can >>>> get my head around PKCS#10 enough to grab attributes). >>>> >>> Given we should prevent "tricks" from people the server side should >>> really parse the CSR and validate it against the ACL IMO. >>> Otherwise do we have any other part that checks that host >>> foo.example.com is asking a certificate for itself and not for >>> bar.example.com ? >>> >>> Simo. >>> >>> >> When binding using machine credentials, in order to request a >> certificate for any host they need to be in the managedBy attribute of >> the target service entry. >> > > I know, but I was referring to stuff like subjectAltName. > Not sure if we need to test it against an ACL to be honest, but we need > to validate that as well (and any other attribute that can affect client > behavior). Whether this is done in IPA or in the CA is not really > important as long as it is done by a component that have enough > information to determine what is ok or not, depending on the "user" > requesting the new cert. > For example admin may be allowed to stuff just any random crap in > subjectAltName but maybe a host shouldn't be. > > Simo. > > How important is this check? In the scenario: * we already authenticated the peer - admin or host using kerberos * we checked the access rights of the authenticated entity against the principal name he tries to issue cert to and determined that entity can do the operation * now we also need to check the the subjectAltName? Why? What is the attack scenario? What are we trying to prevent with this check? Thanks Dmitri -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From awnuk at redhat.com Fri Nov 6 16:21:54 2009 From: awnuk at redhat.com (Andrew Wnuk) Date: Fri, 06 Nov 2009 08:21:54 -0800 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <1257520971.3553.137.camel@willson.li.ssimo.org> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> <4AF42DF2.2060600@redhat.com> <1257520971.3553.137.camel@willson.li.ssimo.org> Message-ID: <4AF44D22.3010602@redhat.com> On 11/06/09 07:22, Simo Sorce wrote: > On Fri, 2009-11-06 at 09:08 -0500, Rob Crittenden wrote: > >> Simo Sorce wrote: >> >>> On Thu, 2009-11-05 at 13:21 -0500, Rob Crittenden wrote: >>> >>>> This is about right. What you're missing is storing the certificate >>>> in >>>> the service record. To do this we need to know what the target is. >>>> >>>> Nalin and I simply took two different approaches to sending this. We >>>> can >>>> easily support either method by making the principal an optional >>>> attribute and looking for it in the CSR if not provided (assuming I >>>> can >>>> get my head around PKCS#10 enough to grab attributes). >>>> >>> Given we should prevent "tricks" from people the server side should >>> really parse the CSR and validate it against the ACL IMO. >>> Otherwise do we have any other part that checks that host >>> foo.example.com is asking a certificate for itself and not for >>> bar.example.com ? >>> >>> Simo. >>> >>> >> When binding using machine credentials, in order to request a >> certificate for any host they need to be in the managedBy attribute of >> the target service entry. >> > I know, but I was referring to stuff like subjectAltName. > Not sure if we need to test it against an ACL to be honest, but we need > to validate that as well (and any other attribute that can affect client > behavior). Whether this is done in IPA or in the CA is not really > important as long as it is done by a component that have enough > information to determine what is ok or not, depending on the "user" > requesting the new cert. > For example admin may be allowed to stuff just any random crap in > subjectAltName but maybe a host shouldn't be. > > Simo. > > Some of this random stuff could be filtered on the client side before CSR is generated. Andrew From ssorce at redhat.com Fri Nov 6 16:39:34 2009 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 06 Nov 2009 11:39:34 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF44D22.3010602@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> <4AF42DF2.2060600@redhat.com> <1257520971.3553.137.camel@willson.li.ssimo.org> <4AF44D22.3010602@redhat.com> Message-ID: <1257525574.7156.0.camel@willson.li.ssimo.org> On Fri, 2009-11-06 at 08:21 -0800, Andrew Wnuk wrote: > Some of this random stuff could be filtered on the client side before > CSR is generated. You can't trust the client, period. Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Fri Nov 6 16:48:05 2009 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 06 Nov 2009 11:48:05 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <1257525574.7156.0.camel@willson.li.ssimo.org> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> <4AF42DF2.2060600@redhat.com> <1257520971.3553.137.camel@willson.li.ssimo.org> <4AF44D22.3010602@redhat.com> <1257525574.7156.0.camel@willson.li.ssimo.org> Message-ID: <4AF45345.1070600@redhat.com> Simo Sorce wrote: > On Fri, 2009-11-06 at 08:21 -0800, Andrew Wnuk wrote: > >> Some of this random stuff could be filtered on the client side before >> CSR is generated. >> > > You can't trust the client, period. > > Simo. > > > This is where you lost me. You can't trust the client unless it is authenticated. But it authenticated so it is trusted to say what it wants. If it wants something not allowed it would not be allowed but at least after authentication you trust client's claim that the client is actually the entity it says it is. Where is the problem? That the client sends some garbage? That the client's identity was hijacked? If it was we either have a case of a stolen admin password and then I do not think you can do a lot about it or someone impersonated the host and requested a cert for a service that this host is allowed to request certs for so how you can distinguish it from a valid request? Sorry, may be I am missing something but I still do not get the point. Can someone explain please what we want to prevent from happening? -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From awnuk at redhat.com Fri Nov 6 16:58:02 2009 From: awnuk at redhat.com (Andrew Wnuk) Date: Fri, 06 Nov 2009 08:58:02 -0800 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <1257525574.7156.0.camel@willson.li.ssimo.org> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> <4AF42DF2.2060600@redhat.com> <1257520971.3553.137.camel@willson.li.ssimo.org> <4AF44D22.3010602@redhat.com> <1257525574.7156.0.camel@willson.li.ssimo.org> Message-ID: <4AF4559A.20408@redhat.com> On 11/06/09 08:39, Simo Sorce wrote: > On Fri, 2009-11-06 at 08:21 -0800, Andrew Wnuk wrote: > >> Some of this random stuff could be filtered on the client side before >> CSR is generated. >> > You can't trust the client, period. > > Simo. > > > Yes but that should not stop the client from verification of its inputs. Andrew From ssorce at redhat.com Fri Nov 6 17:04:07 2009 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 06 Nov 2009 12:04:07 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF45345.1070600@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> <4AF42DF2.2060600@redhat.com> <1257520971.3553.137.camel@willson.li.ssimo.org> <4AF44D22.3010602@redhat.com> <1257525574.7156.0.camel@willson.li.ssimo.org> <4AF45345.1070600@redhat.com> Message-ID: <1257527047.7156.8.camel@willson.li.ssimo.org> On Fri, 2009-11-06 at 11:48 -0500, Dmitri Pal wrote: > This is where you lost me. > You can't trust the client unless it is authenticated. > But it authenticated so it is trusted to say what it wants. > If it wants something not allowed it would not be > allowed but at least after authentication you trust > client's claim that the client is actually the entity > it says it is. Where is the problem? That the client sends some > garbage? > That the client's identity was hijacked? Yes. A host can be compromised, we can't entirely trust a CSR coming from a host if we do automatic approval. We need to always validate what clients present us. > If it was we either have a case of a stolen admin password and then I > do not think you can do a lot about it or > someone impersonated the host and requested a cert for a service that > this host is allowed to request certs for > so how you can distinguish it from a valid request? If a (senior enough) admin requests a cert we can allow everything in the CSR, once you have admin credentials you can do whatever you want, I am not concerned about that case too much. If a host is requesting a cert then no, we need some more validation, as host credentials are stored in the hosts themselves, so anyone with root access on the host can use those credentials. > Sorry, may be I am missing something but I still do not get the > point. The point is that the level of trust you can give to a host is not limited. > Can someone explain please what we want to prevent from happening? That we allow to generate and ship certificates that can be used to MITM or otherwise deceive clients. We may want to do some validation also in case cert creation for host is delegated to some lower level admins, actually. Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Fri Nov 6 17:58:20 2009 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 06 Nov 2009 12:58:20 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <1257527047.7156.8.camel@willson.li.ssimo.org> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> <4AF42DF2.2060600@redhat.com> <1257520971.3553.137.camel@willson.li.ssimo.org> <4AF44D22.3010602@redhat.com> <1257525574.7156.0.camel@willson.li.ssimo.org> <4AF45345.1070600@redhat.com> <1257527047.7156.8.camel@willson.li.ssimo.org> Message-ID: <4AF463BC.6040305@redhat.com> Simo Sorce wrote: > On Fri, 2009-11-06 at 11:48 -0500, Dmitri Pal wrote: > >> This is where you lost me. >> You can't trust the client unless it is authenticated. >> But it authenticated so it is trusted to say what it wants. >> If it wants something not allowed it would not be >> allowed but at least after authentication you trust >> client's claim that the client is actually the entity >> it says it is. Where is the problem? That the client sends some >> garbage? >> That the client's identity was hijacked? >> > > Yes. A host can be compromised, we can't entirely trust a CSR coming > from a host if we do automatic approval. We need to always validate what > clients present us. > > >> If it was we either have a case of a stolen admin password and then I >> do not think you can do a lot about it or >> someone impersonated the host and requested a cert for a service that >> this host is allowed to request certs for >> so how you can distinguish it from a valid request? >> > > If a (senior enough) admin requests a cert we can allow everything in > the CSR, once you have admin credentials you can do whatever you want, I > am not concerned about that case too much. > > If a host is requesting a cert then no, we need some more validation, as > host credentials are stored in the hosts themselves, so anyone with root > access on the host can use those credentials. > > >> Sorry, may be I am missing something but I still do not get the >> point. >> > > The point is that the level of trust you can give to a host is not > limited. > > >> Can someone explain please what we want to prevent from happening? >> > > That we allow to generate and ship certificates that can be used to MITM > or otherwise deceive clients. > > We may want to do some validation also in case cert creation for host is > delegated to some lower level admins, actually. > > Simo. > > This all makes sense but Rob's question from the bug is still open: "The question is, even if we can look at subjectAltName in the IPA backend how will we prevent users from mis-using subjectAltName, such as requesting a cert for a host they don't control?" -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From ssorce at redhat.com Fri Nov 6 18:24:34 2009 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 06 Nov 2009 13:24:34 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF463BC.6040305@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> <4AF42DF2.2060600@redhat.com> <1257520971.3553.137.camel@willson.li.ssimo.org> <4AF44D22.3010602@redhat.com> <1257525574.7156.0.camel@willson.li.ssimo.org> <4AF45345.1070600@redhat.com> <1257527047.7156.8.camel@willson.li.ssimo.org> <4AF463BC.6040305@redhat.com> Message-ID: <1257531874.7156.13.camel@willson.li.ssimo.org> On Fri, 2009-11-06 at 12:58 -0500, Dmitri Pal wrote: > > This all makes sense but Rob's question from the bug is still open: > > "The question is, even if we can look at subjectAltName in the IPA > backend how > will we prevent users from mis-using subjectAltName, such as > requesting > a cert > for a host they don't control?" My natural paranoid answer is: by making a white list of what is allowed and refusing anything else. Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Fri Nov 6 19:19:08 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 06 Nov 2009 14:19:08 -0500 Subject: [Freeipa-devel] Re: Certificate enrollment, principal names In-Reply-To: <4AF463BC.6040305@redhat.com> References: <20091103233711.GC9409@redhat.com> <4AF1F49C.3080900@redhat.com> <20091105001658.GA7176@redhat.com> <4AF2299B.7010007@redhat.com> <4AF22AD6.7090004@redhat.com> <4AF26629.8040906@redhat.com> <20091105152839.GA17324@redhat.com> <4AF2F194.1050302@redhat.com> <20091105170443.GB17324@redhat.com> <4AF30D41.8060400@redhat.com> <4AF3135B.2070303@redhat.com> <4AF317A1.6040406@redhat.com> <1257448924.3553.106.camel@willson.li.ssimo.org> <4AF42DF2.2060600@redhat.com> <1257520971.3553.137.camel@willson.li.ssimo.org> <4AF44D22.3010602@redhat.com> <1257525574.7156.0.camel@willson.li.ssimo.org> <4AF45345.1070600@redhat.com> <1257527047.7156.8.camel@willson.li.ssimo.org> <4AF463BC.6040305@redhat.com> Message-ID: <4AF476AC.5010208@redhat.com> Dmitri Pal wrote: > Simo Sorce wrote: >> On Fri, 2009-11-06 at 11:48 -0500, Dmitri Pal wrote: >> >>> This is where you lost me. >>> You can't trust the client unless it is authenticated. >>> But it authenticated so it is trusted to say what it wants. >>> If it wants something not allowed it would not be >>> allowed but at least after authentication you trust >>> client's claim that the client is actually the entity >>> it says it is. Where is the problem? That the client sends some >>> garbage? >>> That the client's identity was hijacked? >>> >> Yes. A host can be compromised, we can't entirely trust a CSR coming >> from a host if we do automatic approval. We need to always validate what >> clients present us. >> >> >>> If it was we either have a case of a stolen admin password and then I >>> do not think you can do a lot about it or >>> someone impersonated the host and requested a cert for a service that >>> this host is allowed to request certs for >>> so how you can distinguish it from a valid request? >>> >> If a (senior enough) admin requests a cert we can allow everything in >> the CSR, once you have admin credentials you can do whatever you want, I >> am not concerned about that case too much. >> >> If a host is requesting a cert then no, we need some more validation, as >> host credentials are stored in the hosts themselves, so anyone with root >> access on the host can use those credentials. >> >> >>> Sorry, may be I am missing something but I still do not get the >>> point. >>> >> The point is that the level of trust you can give to a host is not >> limited. >> >> >>> Can someone explain please what we want to prevent from happening? >>> >> That we allow to generate and ship certificates that can be used to MITM >> or otherwise deceive clients. >> >> We may want to do some validation also in case cert creation for host is >> delegated to some lower level admins, actually. >> >> Simo. >> >> > > This all makes sense but Rob's question from the bug is still open: > > "The question is, even if we can look at subjectAltName in the IPA > backend how > will we prevent users from mis-using subjectAltName, such as requesting > a cert > for a host they don't control?" Right. I'm not entirely sure how I'll implement this check. And I think the answer is different depending on whether an admin or a machine is doing the request. I'd think that for a host we'd want a list of acceptable hostnames. For an admin user I'm not so sure. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From mnagy at redhat.com Mon Nov 9 13:39:34 2009 From: mnagy at redhat.com (Martin Nagy) Date: Mon, 09 Nov 2009 14:39:34 +0100 Subject: [Freeipa-devel] DNS schema In-Reply-To: <4AF35CBF.10701@redhat.com> References: <4AF3297D.6040508@redhat.com> <1257454855.3553.115.camel@willson.li.ssimo.org> <4AF35CBF.10701@redhat.com> Message-ID: <1257773974.1895.0.camel@wolverine.englab.brq.redhat.com> On Thu, 2009-11-05 at 18:16 -0500, Dmitri Pal wrote: > Simo Sorce wrote: > > On Thu, 2009-11-05 at 14:37 -0500, Dmitri Pal wrote: > > > >> Hi, > >> > >> Now when we decided to use latest DS that is available from 389 project > >> for IPA v2 we can take advantage of the syntaxes that DS team has added. > >> For example they added numeric syntax that was a part of the original > >> DNS schema we planned for IPA. > >> I remember we replaced the originally planned syntax with some other > >> syntax that is available. > >> > >> Does it make sense to revert to the numeric syntax now or it is not a > >> big deal? > >> I am just concerned that once we will decide to do it the original way > >> because we would think it is the "right" thing to do and will face all > >> sorts of migration issues. > >> Ideas? > >> > > > > Unless it is a major amount of work we should use the proper syntax. > > Especially for "standard" schema. > > > > Simo. > > > > > It should not be a big deal IMO so Martin if you are ok with this I > would like to ask Pavel > to take it as an action item. Dmitri, I'm doing something "in the neighborhood" right now, so let me take care of it, it'll only take little time. Martin From pzuna at redhat.com Mon Nov 9 13:43:58 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Mon, 09 Nov 2009 14:43:58 +0100 Subject: [Freeipa-devel] [PATCH] Add BIND pre-op for DS->IPA password migration to ipa-pwd-extop DS plugin. Message-ID: <4AF81C9E.8040305@redhat.com> Re-post from ipa-and-samba-team-list. This patch makes DS generate new Kerberos keys for eligible users upon a successful simple bind. We need this for password migration. Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-BIND-pre-op-for-DS-IPA-password-migration-to-ip.patch Type: application/mbox Size: 8462 bytes Desc: not available URL: From dpal at redhat.com Mon Nov 9 14:45:14 2009 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 09 Nov 2009 09:45:14 -0500 Subject: [Freeipa-devel] DNS schema In-Reply-To: <1257773974.1895.0.camel@wolverine.englab.brq.redhat.com> References: <4AF3297D.6040508@redhat.com> <1257454855.3553.115.camel@willson.li.ssimo.org> <4AF35CBF.10701@redhat.com> <1257773974.1895.0.camel@wolverine.englab.brq.redhat.com> Message-ID: <4AF82AFA.1080509@redhat.com> Martin Nagy wrote: > On Thu, 2009-11-05 at 18:16 -0500, Dmitri Pal wrote: > >> Simo Sorce wrote: >> >>> On Thu, 2009-11-05 at 14:37 -0500, Dmitri Pal wrote: >>> >>> >>>> Hi, >>>> >>>> Now when we decided to use latest DS that is available from 389 project >>>> for IPA v2 we can take advantage of the syntaxes that DS team has added. >>>> For example they added numeric syntax that was a part of the original >>>> DNS schema we planned for IPA. >>>> I remember we replaced the originally planned syntax with some other >>>> syntax that is available. >>>> >>>> Does it make sense to revert to the numeric syntax now or it is not a >>>> big deal? >>>> I am just concerned that once we will decide to do it the original way >>>> because we would think it is the "right" thing to do and will face all >>>> sorts of migration issues. >>>> Ideas? >>>> >>>> >>> Unless it is a major amount of work we should use the proper syntax. >>> Especially for "standard" schema. >>> >>> Simo. >>> >>> >>> >> It should not be a big deal IMO so Martin if you are ok with this I >> would like to ask Pavel >> to take it as an action item. >> > > Dmitri, > I'm doing something "in the neighborhood" right now, so let me take care > of it, it'll only take little time. > > Martin > > Works for me, thanks! -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Mon Nov 9 16:27:07 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 09 Nov 2009 11:27:07 -0500 Subject: [Freeipa-devel] Thoughts on client configuration Message-ID: <4AF842DB.90604@redhat.com> I've got all the pieces together to create a host principal and keytab when a machine joins an IPA realm and am thinking about how I'm going to tie it altogether. My plan revolves around enhancing ipa-client-install to call ipa-join and ipa-rmkeytab (for uninstall). The question then becomes, is the client configuration dependent upon successful machine join? We have a bit of a chicken and egg problem right now with join in terms of validating argument inputs. Joining a machine can happen one of two ways, using kerberos credentials (an admin) or using a one-time password (OTP). The OTP method is easy enough, we can call that really early in the client configuration process. If it fails (wrong password, host not created, whatever) we can simply quit and not configure the client at all. With the admin method we have to first configure the machine, then get the credentials, then try to do the join. It could easily fail here for a number of reasons. Do we roll back the configuration upon failure? I'm thinking the answer should be yes, otherwise some machines will have host service principals and some won't making a support nightmare. But should we have a --force option to let the client be configured anyway, in sort of a degraded mode? Or a --no-keytab option to be more explicit? Or both? I'm all for flexibility, just not sure what the implications of this are other than support headaches like "I can log into machine A but not machine B, why not?" Well, you're missing the host keytab for some reason... rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From dpal at redhat.com Mon Nov 9 16:35:30 2009 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 09 Nov 2009 11:35:30 -0500 Subject: [Freeipa-devel] Thoughts on client configuration In-Reply-To: <4AF842DB.90604@redhat.com> References: <4AF842DB.90604@redhat.com> Message-ID: <4AF844D2.9070206@redhat.com> Rob Crittenden wrote: > I've got all the pieces together to create a host principal and keytab > when a machine joins an IPA realm and am thinking about how I'm going > to tie it altogether. > > My plan revolves around enhancing ipa-client-install to call ipa-join > and ipa-rmkeytab (for uninstall). The question then becomes, is the > client configuration dependent upon successful machine join? > > We have a bit of a chicken and egg problem right now with join in > terms of validating argument inputs. Joining a machine can happen one > of two ways, using kerberos credentials (an admin) or using a one-time > password (OTP). > > The OTP method is easy enough, we can call that really early in the > client configuration process. If it fails (wrong password, host not > created, whatever) we can simply quit and not configure the client at > all. > > With the admin method we have to first configure the machine, then get > the credentials, then try to do the join. It could easily fail here > for a number of reasons. Do we roll back the configuration upon failure? Why do you need first configure the machine? Why the sequence of operations is different in the OTP vs admin case. Can we have the same sequence? Then the difference is the authenticated entity and its permissions only. > > I'm thinking the answer should be yes, otherwise some machines will > have host service principals and some won't making a support > nightmare. But should we have a --force option to let the client be > configured anyway, in sort of a degraded mode? Or a --no-keytab option > to be more explicit? Or both? > > I'm all for flexibility, just not sure what the implications of this > are other than support headaches like "I can log into machine A but > not machine B, why not?" Well, you're missing the host keytab for some > reason... > > rob > ------------------------------------------------------------------------ > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From ssorce at redhat.com Mon Nov 9 16:35:52 2009 From: ssorce at redhat.com (Simo Sorce) Date: Mon, 09 Nov 2009 11:35:52 -0500 Subject: [Freeipa-devel] Thoughts on client configuration In-Reply-To: <4AF842DB.90604@redhat.com> References: <4AF842DB.90604@redhat.com> Message-ID: <1257784553.2547.56.camel@willson.li.ssimo.org> On Mon, 2009-11-09 at 11:27 -0500, Rob Crittenden wrote: > I've got all the pieces together to create a host principal and keytab > when a machine joins an IPA realm and am thinking about how I'm going to > tie it altogether. > > My plan revolves around enhancing ipa-client-install to call ipa-join > and ipa-rmkeytab (for uninstall). The question then becomes, is the > client configuration dependent upon successful machine join? > > We have a bit of a chicken and egg problem right now with join in terms > of validating argument inputs. Joining a machine can happen one of two > ways, using kerberos credentials (an admin) or using a one-time password > (OTP). > > The OTP method is easy enough, we can call that really early in the > client configuration process. If it fails (wrong password, host not > created, whatever) we can simply quit and not configure the client at all. > > With the admin method we have to first configure the machine, then get > the credentials, then try to do the join. It could easily fail here for > a number of reasons. Do we roll back the configuration upon failure? Uhmm you can do admin stuff w/o necessarily configuring the machine first, for the kerberos part we can set temporary environment variables and get the admin password through a prompt. If all is successful we save the configuration in the real krb5.conf and all. I actually would really prefer to do it this way so that we do not touch permanent configuration files until the join procedure is successful. > I'm thinking the answer should be yes, otherwise some machines will have > host service principals and some won't making a support nightmare. But > should we have a --force option to let the client be configured anyway, > in sort of a degraded mode? Or a --no-keytab option to be more explicit? > Or both? Good question, in what case would it make sense to run ipa-client-install and not get the machine enrolled in ? > I'm all for flexibility, just not sure what the implications of this are > other than support headaches like "I can log into machine A but not > machine B, why not?" Well, you're missing the host keytab for some reason... Yeah, I think we should avoid half configured machines. If someone has special needs he can script his own installation procedure the way he wants, IMO. Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Mon Nov 9 16:54:48 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 09 Nov 2009 11:54:48 -0500 Subject: [Freeipa-devel] Thoughts on client configuration In-Reply-To: <1257784553.2547.56.camel@willson.li.ssimo.org> References: <4AF842DB.90604@redhat.com> <1257784553.2547.56.camel@willson.li.ssimo.org> Message-ID: <4AF84958.1000209@redhat.com> Simo Sorce wrote: > On Mon, 2009-11-09 at 11:27 -0500, Rob Crittenden wrote: >> I've got all the pieces together to create a host principal and keytab >> when a machine joins an IPA realm and am thinking about how I'm going to >> tie it altogether. >> >> My plan revolves around enhancing ipa-client-install to call ipa-join >> and ipa-rmkeytab (for uninstall). The question then becomes, is the >> client configuration dependent upon successful machine join? >> >> We have a bit of a chicken and egg problem right now with join in terms >> of validating argument inputs. Joining a machine can happen one of two >> ways, using kerberos credentials (an admin) or using a one-time password >> (OTP). >> >> The OTP method is easy enough, we can call that really early in the >> client configuration process. If it fails (wrong password, host not >> created, whatever) we can simply quit and not configure the client at all. >> >> With the admin method we have to first configure the machine, then get >> the credentials, then try to do the join. It could easily fail here for >> a number of reasons. Do we roll back the configuration upon failure? > > Uhmm you can do admin stuff w/o necessarily configuring the machine > first, for the kerberos part we can set temporary environment variables > and get the admin password through a prompt. If all is successful we > save the configuration in the real krb5.conf and all. Wow, fantastic idea. > I actually would really prefer to do it this way so that we do not touch > permanent configuration files until the join procedure is successful. Right, that's what I was afraid of. Having to fire up install machine only to then back it all out. > >> I'm thinking the answer should be yes, otherwise some machines will have >> host service principals and some won't making a support nightmare. But >> should we have a --force option to let the client be configured anyway, >> in sort of a degraded mode? Or a --no-keytab option to be more explicit? >> Or both? > > Good question, in what case would it make sense to run > ipa-client-install and not get the machine enrolled in ? I think that some desktop machines perhaps. Then again it is fairly trivial to manually configure the pam stack to do LDAP/kerberos. There actually already is a --force option but it applies to the DNS discovery failing. >> I'm all for flexibility, just not sure what the implications of this are >> other than support headaches like "I can log into machine A but not >> machine B, why not?" Well, you're missing the host keytab for some reason... > > Yeah, I think we should avoid half configured machines. If someone has > special needs he can script his own installation procedure the way he > wants, IMO. That or we need to write a very good FAQ :-) rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From jderose at redhat.com Mon Nov 9 17:49:52 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Mon, 09 Nov 2009 10:49:52 -0700 Subject: [Freeipa-devel] [PATCH] Add 'File' parameter type. In-Reply-To: <4AF3FEA3.2020305@redhat.com> References: <4AF3FEA3.2020305@redhat.com> Message-ID: <1257788992.7901.0.camel@jgd-dsk> On Fri, 2009-11-06 at 11:46 +0100, Pavel Zuna wrote: > Accepts filenames and loads file contents as parameter value. > > In CLI, the 'stdin_if_missing' kwarg can be used to read the file from stdin if > no filename has been entered. > > Pavel ack. pushed to master. From jderose at redhat.com Mon Nov 9 17:50:03 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Mon, 09 Nov 2009 10:50:03 -0700 Subject: [Freeipa-devel] [PATCH] Use File parameter for CSR in cert_request command plugin. In-Reply-To: <4AF3FEDB.9080703@redhat.com> References: <4AF3FEDB.9080703@redhat.com> Message-ID: <1257789003.7901.1.camel@jgd-dsk> On Fri, 2009-11-06 at 11:47 +0100, Pavel Zuna wrote: > Makes use of the new File parameter introduced in my previous patch. > > Pavel ack. pushed to master. From rcritten at redhat.com Tue Nov 10 17:28:46 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 10 Nov 2009 12:28:46 -0500 Subject: [Freeipa-devel] [PATCH] 308 manage arbitrary attributes In-Reply-To: <1257398378.20441.1.camel@jgd-dsk> References: <4AF1EC42.5000603@redhat.com> <1257398378.20441.1.camel@jgd-dsk> Message-ID: <4AF9A2CE.2070900@redhat.com> Jason Gerard DeRose wrote: > Oops, was this missing the attachment? ;) Bah, here it is. rob > > On Wed, 2009-11-04 at 16:04 -0500, Rob Crittenden wrote: >> This adds 2 new parameters, --setattr and --addattr and lets you manage >> whatever attribute you want (within the given set of objectclasses). >> >> Both take a name/value pair. >> >> --setattr sets the attribute to the given value >> --addattr adds the value to an attribute. Can be used to manage >> multi-valued attributes >> >> For example: >> >> ipa user-mod --addattr=postalcode=90210 jsmith >> >> If the attribute to be modified is an another param then the value is >> silently dropped. >> >> You can include multiples of these on a single command-line: >> >> ipa user-mod --addattr=postalcode=20601 --addattr=postalcode=30330 jsmith >> >> Setting an attribute to "" deletes it: >> >> ipa user-mod --setattr=postalcode= jsmith >> >> rob >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-308-attr.patch Type: application/mbox Size: 6803 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From jdennis at redhat.com Wed Nov 11 01:42:21 2009 From: jdennis at redhat.com (John Dennis) Date: Tue, 10 Nov 2009 20:42:21 -0500 Subject: [Freeipa-devel] localized strings Message-ID: <4AFA167D.5010301@redhat.com> I was writing some new code and wanted to return an error message. Error messages need to be localized. Typically this is done via the _() method. But _() has to be defined and there are several ways that can happen. So I went looking for existing usage in our code base. I was surprised at how little I found which makes me wonder if I'm not understanding how we're handling messages. errors.py does define _() which makes sense, but a lot of the usage of the error classes in errors.py involves providing a customized error detail message, usually via the 'reason' keyword, for example: raise errors.NotFound(reason="Group '%s' does not exist" % kw['group']) but it seems to me this should be: raise errors.NotFound(reason=_("Group '%s' does not exist") % kw['group']) Virtually all of the examples I found of returning a detailed error message omitted the _() localization method. Why? Am I missing something about how we're supposed to be coding this? This also raised the question in my mind about how we're binding _(), the only example I could find was this: from request import ugettext as _ Is this what all the modules are supposed to be doing to bind _()? -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Wed Nov 11 16:32:55 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 11 Nov 2009 11:32:55 -0500 Subject: [Freeipa-devel] [PATCH] 309 make exception from ipautil.run() optional Message-ID: <4AFAE737.7090507@redhat.com> There are probably occasions where a caller will want more control over what happens when running a command fails. I've added an optional argument to run where it will not raise an exception on errors. I've also added returncode to the tuple of things returned. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-309-run.patch Type: application/mbox Size: 7263 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Wed Nov 11 16:36:54 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 11 Nov 2009 11:36:54 -0500 Subject: [Freeipa-devel] [PATCH] 310 clean up ipa-join return values Message-ID: <4AFAE826.9030708@redhat.com> ipa-join calls ipa-getkeytab and returns whatever return value it does so I want to be careful not to overlap the values and keep things unique or meaning the same thing in both. This patch cleans up a few places. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-310-join.patch Type: application/mbox Size: 3951 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Wed Nov 11 16:39:46 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 11 Nov 2009 11:39:46 -0500 Subject: [Freeipa-devel] [PATCH] 311 more integrated client install Message-ID: <4AFAE8D2.3050701@redhat.com> This patch integrates ipa-join and ipa-rmkeytab into the client installer. This will join a machine to the IPA realm and fetch a host principal for /etc/krb5.keytab. On uninstall all principals for the realm will be removed from /etc/krb5.keytab. By default installation will fail if a host principal cannot be obtained. Use the --force option to continue anyway. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-311-install.patch Type: application/mbox Size: 19506 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Wed Nov 11 16:41:12 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 11 Nov 2009 11:41:12 -0500 Subject: [Freeipa-devel] [PATCH] 309 make exception from ipautil.run() optional In-Reply-To: <4AFAE737.7090507@redhat.com> References: <4AFAE737.7090507@redhat.com> Message-ID: <4AFAE928.1020801@redhat.com> Rob Crittenden wrote: > There are probably occasions where a caller will want more control over > what happens when running a command fails. I've added an optional > argument to run where it will not raise an exception on errors. > > I've also added returncode to the tuple of things returned. > > rob I forgot to include this additional change in the patch. When acked I'll add this bit too and commit it. --- a/ipaserver/install/httpinstance.py +++ b/ipaserver/install/httpinstance.py @@ -100,7 +100,7 @@ class HTTPInstance(service.Service): if selinux: try: # returns e.g. "httpd_can_network_connect --> off" - (stdout, stderr) = ipautil.run(["/usr/sbin/getsebool", + (stdout, stderr, returncode) = ipautil.run(["/usr/sbin/getsebool", "httpd_can_network_connect"]) self.backup_state("httpd_can_network_connect", stdout.split()[2]) except: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From dpal at redhat.com Wed Nov 11 17:03:17 2009 From: dpal at redhat.com (Dmitri Pal) Date: Wed, 11 Nov 2009 12:03:17 -0500 Subject: [Freeipa-devel] localized strings In-Reply-To: <4AFA167D.5010301@redhat.com> References: <4AFA167D.5010301@redhat.com> Message-ID: <4AFAEE55.5060609@redhat.com> John Dennis wrote: > I was writing some new code and wanted to return an error message. > Error messages need to be localized. Typically this is done via the > _() method. But _() has to be defined and there are several ways that > can happen. So I went looking for existing usage in our code base. I > was surprised at how little I found which makes me wonder if I'm not > understanding how we're handling messages. errors.py does define _() > which makes sense, but a lot of the usage of the error classes in > errors.py involves providing a customized error detail message, > usually via the 'reason' keyword, for example: > > raise errors.NotFound(reason="Group '%s' does not exist" % kw['group']) > > but it seems to me this should be: > > raise errors.NotFound(reason=_("Group '%s' does not exist") % > kw['group']) > > > Virtually all of the examples I found of returning a detailed error > message omitted the _() localization method. Why? Am I missing > something about how we're supposed to be coding this? This also raised > the question in my mind about how we're binding _(), the only example > I could find was this: > > from request import ugettext as _ > > Is this what all the modules are supposed to be doing to bind _()? > > Jason, Pavel, Rob we talked about this many months ago and agreed that we will be writing "translation ready code". I think we need to clean it up. I will add a task to the iteration to clean this up. Pavel I suspect it will fall on your shoulders. Ok? -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jderose at redhat.com Wed Nov 11 20:45:09 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Wed, 11 Nov 2009 13:45:09 -0700 Subject: [Freeipa-devel] localized strings In-Reply-To: <4AFA167D.5010301@redhat.com> References: <4AFA167D.5010301@redhat.com> Message-ID: <1257972309.2460.6.camel@jgd-dsk> On Tue, 2009-11-10 at 20:42 -0500, John Dennis wrote: > I was writing some new code and wanted to return an error message. Error > messages need to be localized. Typically this is done via the _() > method. But _() has to be defined and there are several ways that can > happen. So I went looking for existing usage in our code base. I was > surprised at how little I found which makes me wonder if I'm not > understanding how we're handling messages. errors.py does define _() > which makes sense, but a lot of the usage of the error classes in > errors.py involves providing a customized error detail message, usually > via the 'reason' keyword, for example: > > raise errors.NotFound(reason="Group '%s' does not exist" % kw['group']) This is a practice I've tried to discourage, but we still use it a lot. > but it seems to me this should be: > > raise errors.NotFound(reason=_("Group '%s' does not exist") % kw['group']) > > > Virtually all of the examples I found of returning a detailed error > message omitted the _() localization method. Why? Am I missing something > about how we're supposed to be coding this? This also raised the > question in my mind about how we're binding _(), the only example I > could find was this: > > from request import ugettext as _ Yes, this is the current mechanism. The next UI patch adds a more generalized mechanism so we can have translation from multiple gettext domains. > Is this what all the modules are supposed to be doing to bind _()? Yes. From jdennis at redhat.com Wed Nov 11 21:16:43 2009 From: jdennis at redhat.com (John Dennis) Date: Wed, 11 Nov 2009 16:16:43 -0500 Subject: [Freeipa-devel] localized strings In-Reply-To: <1257972309.2460.6.camel@jgd-dsk> References: <4AFA167D.5010301@redhat.com> <1257972309.2460.6.camel@jgd-dsk> Message-ID: <4AFB29BB.9040108@redhat.com> On 11/11/2009 03:45 PM, Jason Gerard DeRose wrote: > On Tue, 2009-11-10 at 20:42 -0500, John Dennis wrote: >> I was writing some new code and wanted to return an error message. Error >> messages need to be localized. Typically this is done via the _() >> method. But _() has to be defined and there are several ways that can >> happen. So I went looking for existing usage in our code base. I was >> surprised at how little I found which makes me wonder if I'm not >> understanding how we're handling messages. errors.py does define _() >> which makes sense, but a lot of the usage of the error classes in >> errors.py involves providing a customized error detail message, usually >> via the 'reason' keyword, for example: >> >> raise errors.NotFound(reason="Group '%s' does not exist" % kw['group']) > > This is a practice I've tried to discourage, but we still use it a lot. Could you provide a bit more explanation for why you would like to discourage this practice? Do you want to see an exception class for each possible error and no customization of the error strings? >> but it seems to me this should be: >> >> raise errors.NotFound(reason=_("Group '%s' does not exist") % kw['group']) >> >> >> Virtually all of the examples I found of returning a detailed error >> message omitted the _() localization method. Why? Am I missing something >> about how we're supposed to be coding this? This also raised the >> question in my mind about how we're binding _(), the only example I >> could find was this: >> >> from request import ugettext as _ > > Yes, this is the current mechanism. The next UI patch adds a more > generalized mechanism so we can have translation from multiple gettext > domains. > >> Is this what all the modules are supposed to be doing to bind _()? > > Yes. > -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From ssorce at redhat.com Wed Nov 11 23:54:24 2009 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 11 Nov 2009 18:54:24 -0500 Subject: [Freeipa-devel] How to implement Magic Private Groups in FreeIPA ? Message-ID: <1257983664.12148.135.camel@willson.li.ssimo.org> For some time we have been thinking of implementing a thing called Magic Private Groups in FreeIPA. As some of you may have realized in v1 we don't create user private groups, instead we set ipausers as the default primary group for all new users. This was done for a few reasons: - avoid the bloat of having thousands of groups just because you have thousands of users - avoid synchronization and name/gid conflict problems both at creation time and at modification time Unfortunately some of you complained that these are not reasons enough to abandon private groups, as it make it more difficult to implement freeipa in places where people is relying on them. Truth be told nothing prevents admins from manually creating a private groups when a user is created, but having to do so explicitly is kind of annoying. There is also another downside of it, the ipausers group quickly grows and become a behemoth. Yet simply adding "real" groups for each user would be quite annoying. So after some thinking I came up with the idea of creating what I call Magic Private Groups. The "magic" is in the fact that they are not actually separate objects but use a neat property of LDAP and objectclasses: you can add objectclasses and attributes to existing entries (no surprise I know :). That means we can add the posixGroup objectclass (which determines and object is a group) to the user along side the posixAccount one. The MUSTs for posixGroup are cn and gidNumber which are already present so with some "magic" we have turned an user object from a being a user account in being *also* a private group for itself. So, where is the devil ? You'd ask .. Well there are 1 issue and 1 potential problem. The "potential" problem is that some client may not like the fact an entry is both a user and a group at the same time. Clients that behave that way could be technically considered buggy as there is nothing that says a user can't be a group at the same time but we need to check if at least the most common client (nss_ldap) will have any problem with that. Anyway thanks to the schema compatibility plugin, "bad" clients can always be told to look at the schema compat tree for groups (like we do for Solaris clients already as they do not understand rfc2307bis). The real issue is the user/group name. Traditionally users use the "uid" attribute to hold the user name, while groups use the "cn" attribute. Now on a user account "cn" is used to hold the fullname so we can't just override it with the group name. Another problem we have traditionally with user and group names is that both uid and cn have case insensitive syntax, but _posix_ users and groups are treated as case sensitive on the system (what an irony, posixAccount and posixGroup haven't been traditionally been really much posixy after all ...). This already causes problems with nss_ldap and is in general an issue for unix clients. So killing two birds with one stone we are thinking of introducing a new attribute called posixName that has a case sensitive syntax and does not conflict with other uses of uid and cn. We will probably still set uid on users and cn on groups but they will be kept in sync with posixName (except for cn on user accounts that holds the full name). nss_ldap can do attribute mapping, so it shouldn't have problem starting using posixName instead of cn for groups. This would solve the problem for MPG as at that point all we need is to have clients search for posixName. As a plus we can set the uniquness plugin to enforce posixName to be unique across the whole tree making it impossible to create "real" groups that have a name conflict with a magic private group. We are going to make some experiments during this and the next week to see if we can implement this and still have happy clients, but we would like to have opinions if anyone have comments about this approach. Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Thu Nov 12 15:37:54 2009 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 12 Nov 2009 10:37:54 -0500 Subject: [Freeipa-devel] How to implement Magic Private Groups in FreeIPA ? In-Reply-To: <1257983664.12148.135.camel@willson.li.ssimo.org> References: <1257983664.12148.135.camel@willson.li.ssimo.org> Message-ID: <4AFC2BD2.1030200@redhat.com> Simo Sorce wrote: > For some time we have been thinking of implementing a thing called Magic > Private Groups in FreeIPA. > > As some of you may have realized in v1 we don't create user private > groups, instead we set ipausers as the default primary group for all new > users. > > This was done for a few reasons: > - avoid the bloat of having thousands of groups just because you have > thousands of users > - avoid synchronization and name/gid conflict problems both at creation > time and at modification time > > Unfortunately some of you complained that these are not reasons enough > to abandon private groups, as it make it more difficult to implement > freeipa in places where people is relying on them. Truth be told nothing > prevents admins from manually creating a private groups when a user is > created, but having to do so explicitly is kind of annoying. > > There is also another downside of it, the ipausers group quickly grows > and become a behemoth. > > > Yet simply adding "real" groups for each user would be quite annoying. > > So after some thinking I came up with the idea of creating what I call > Magic Private Groups. > The "magic" is in the fact that they are not actually separate objects > but use a neat property of LDAP and objectclasses: you can add > objectclasses and attributes to existing entries (no surprise I know :). > That means we can add the posixGroup objectclass (which determines and > object is a group) to the user along side the posixAccount one. > The MUSTs for posixGroup are cn and gidNumber which are already present > so with some "magic" we have turned an user object from a being a user > account in being *also* a private group for itself. > > So, where is the devil ? You'd ask .. > > Well there are 1 issue and 1 potential problem. > > The "potential" problem is that some client may not like the fact an > entry is both a user and a group at the same time. Clients that behave > that way could be technically considered buggy as there is nothing that > says a user can't be a group at the same time but we need to check if at > least the most common client (nss_ldap) will have any problem with that. > Anyway thanks to the schema compatibility plugin, "bad" clients can > always be told to look at the schema compat tree for groups (like we do > for Solaris clients already as they do not understand rfc2307bis). > > The real issue is the user/group name. > > Traditionally users use the "uid" attribute to hold the user name, while > groups use the "cn" attribute. Now on a user account "cn" is used to > hold the fullname so we can't just override it with the group name. > > Another problem we have traditionally with user and group names is that > both uid and cn have case insensitive syntax, but _posix_ users and > groups are treated as case sensitive on the system (what an irony, > posixAccount and posixGroup haven't been traditionally been really much > posixy after all ...). > > This already causes problems with nss_ldap and is in general an issue > for unix clients. > > So killing two birds with one stone we are thinking of introducing a new > attribute called posixName that has a case sensitive syntax and does not > conflict with other uses of uid and cn. We will probably still set uid > on users and cn on groups but they will be kept in sync with posixName > (except for cn on user accounts that holds the full name). > > So posixName will be a part of the user account object and group object, right? Can you please add more details here? > nss_ldap can do attribute mapping, so it shouldn't have problem starting > using posixName instead of cn for groups. > > This would solve the problem for MPG as at that point all we need is to > have clients search for posixName. > > As a plus we can set the uniquness plugin to enforce posixName to be > unique across the whole tree making it impossible to create "real" > groups that have a name conflict with a magic private group. > > > > We are going to make some experiments during this and the next week to > see if we can implement this and still have happy clients, but we would > like to have opinions if anyone have comments about this approach. > > Simo. > > -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From ssorce at redhat.com Thu Nov 12 16:08:40 2009 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 12 Nov 2009 11:08:40 -0500 Subject: [Freeipa-devel] How to implement Magic Private Groups in FreeIPA ? In-Reply-To: <4AFC2BD2.1030200@redhat.com> References: <1257983664.12148.135.camel@willson.li.ssimo.org> <4AFC2BD2.1030200@redhat.com> Message-ID: <1258042120.12148.1128.camel@willson.li.ssimo.org> On Thu, 2009-11-12 at 10:37 -0500, Dmitri Pal wrote: > > So killing two birds with one stone we are thinking of introducing a > new > > attribute called posixName that has a case sensitive syntax and does > not > > conflict with other uses of uid and cn. We will probably still set > uid > > on users and cn on groups but they will be kept in sync with > posixName > > (except for cn on user accounts that holds the full name). > > > > > > So posixName will be a part of the user account object and group > object, > right? > Can you please add more details here? Correct, we would switch to primarily use posixName for users and groups names. A group entry would probably look like this (from memory): cn=newgroup,cn=groups,cn=accounts,dc=example,dc=com objectclass: nestedgroup objectclass: posixGroup objectclass: ipaPosixName cn: newgroup posixName: newgroup member: ... member: ... When searching for this group we would use a query like: '(&(objectClass=posixGroup)(posixName=newgroup))' Same for users. Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Thu Nov 12 18:17:41 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 12 Nov 2009 13:17:41 -0500 Subject: [Freeipa-devel] [PATCH] 312 fix some ACI parsing bugs Message-ID: <4AFC5145.6080605@redhat.com> This fixes 2 bugs in the ACI parser: 1. When looking for the version section I wasn't specific enough. If the aci had the attribute nsosversion in it this was found instead. I switched it to look for version 3.0 instead. This removes a tiny amount of flexibility but since we create all the ACIs ourselves I think it is ok. 2. In isequal() if one of the ACIs had no targetattrs it was trying to create a set out of None which is illegal. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-312-aci.patch Type: application/mbox Size: 2110 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Thu Nov 12 18:23:09 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 12 Nov 2009 13:23:09 -0500 Subject: [Freeipa-devel] [PATCH] 313 fix aci plugin host helper Message-ID: <4AFC528D.40601@redhat.com> When creating an aci to cover host objects the wrong attribute is used in the DN. It should be using fqdn, not cn. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-313-aci.patch Type: application/mbox Size: 848 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Thu Nov 12 21:50:43 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 12 Nov 2009 16:50:43 -0500 Subject: [Freeipa-devel] [PATCH] 314 handle LDAP bind failures gracefully Message-ID: <4AFC8333.1050402@redhat.com> Gracefully handle a valid kerberos ticket for a deleted entry. I saw this with a host where I joined a host, obtained a host principal, kinited to that principal, then deleted the host from the IPA server. The ticket was still valid so Apache let it through but it failed to bind to LDAP. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-314-bind.patch Type: application/mbox Size: 2503 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Fri Nov 13 15:30:39 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 13 Nov 2009 10:30:39 -0500 Subject: [Freeipa-devel] How to implement Magic Private Groups in FreeIPA ? In-Reply-To: <1258042120.12148.1128.camel@willson.li.ssimo.org> References: <1257983664.12148.135.camel@willson.li.ssimo.org> <4AFC2BD2.1030200@redhat.com> <1258042120.12148.1128.camel@willson.li.ssimo.org> Message-ID: <4AFD7B9F.5080405@redhat.com> Simo Sorce wrote: > On Thu, 2009-11-12 at 10:37 -0500, Dmitri Pal wrote: >>> So killing two birds with one stone we are thinking of introducing a >> new >>> attribute called posixName that has a case sensitive syntax and does >> not >>> conflict with other uses of uid and cn. We will probably still set >> uid >>> on users and cn on groups but they will be kept in sync with >> posixName >>> (except for cn on user accounts that holds the full name). >>> >>> >> So posixName will be a part of the user account object and group >> object, >> right? >> Can you please add more details here? > > Correct, > we would switch to primarily use posixName for users and groups names. > > A group entry would probably look like this (from memory): > > cn=newgroup,cn=groups,cn=accounts,dc=example,dc=com > objectclass: nestedgroup > objectclass: posixGroup > objectclass: ipaPosixName > cn: newgroup > posixName: newgroup > member: ... > member: ... > > > When searching for this group we would use a query like: > '(&(objectClass=posixGroup)(posixName=newgroup))' > > Same for users. > > Simo. > > FYI, here is the new schema I've come up with: dn: cn=schema attributeTypes: ( 2.16.840.1.113730.3.8.3.54 NAME 'posixName' EQUALITY caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE) objectClasses: ( 2.16.840.1.113730.3.8.3.55 NAME 'ipaPosixName' DESC 'Case-sensitive name common to users and groups' AUXILIARY MUST ( posixName ) X-ORIGIN 'IPA v2' ) It also occurs to me that we'll need to prevent any modifications to the posixName attribute unless the cn/uid is also being modified. On other word, sync needs to be 2-way. rob rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From pzuna at redhat.com Fri Nov 13 15:33:39 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Fri, 13 Nov 2009 16:33:39 +0100 Subject: [Freeipa-devel] [PATCH] Add --all to LDAPCreate and make LDAP commands always display default attributes. In-Reply-To: <4AFD6DB1.6010805@redhat.com> References: <4AFD0C35.2050309@redhat.com> <4AFD6DB1.6010805@redhat.com> Message-ID: <4AFD7C53.2010904@redhat.com> This should fix the issue: Rob Crittenden wrote: > Michael Gregg wrote: >> >> Rob, did the support for posix groups change? >> >> If I create a group specifying "--posix" the cli does create the group. >> Then, using ipa group-find, I do not see any way to determine if that >> group is a posixgroup or not. >> group-find -all used to reveal a PosixGroup field. >> >> How do I determine if a group is a posix group or not? >> >> Michael- > > Ok, I suppose I could have looked at this before firing off an e-mail :-) > > I do see the group number when showing a group: > > $ ./ipa group-show g9 > ----------- > group-show: > ----------- > Group: g9 > name: g9 > description: test posix group > group id: 1117 > > But when adding it this doesn't appear. Oddly enough we show the > ipquniqueid when adding a group but not when showing it! > > Pavel, do you have time to investigate this inconsistency? > > rob Pavel From pzuna at redhat.com Fri Nov 13 15:35:05 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Fri, 13 Nov 2009 16:35:05 +0100 Subject: [Freeipa-devel] Re: [PATCH] Add --all to LDAPCreate and make LDAP commands always display default attributes. In-Reply-To: <4AFD7C53.2010904@redhat.com> References: <4AFD0C35.2050309@redhat.com> <4AFD6DB1.6010805@redhat.com> <4AFD7C53.2010904@redhat.com> Message-ID: <4AFD7CA9.3040902@redhat.com> And here's the actual patch. :) Pavel Zuna wrote: > This should fix the issue: > > Rob Crittenden wrote: >> Michael Gregg wrote: >>> >>> Rob, did the support for posix groups change? >>> >>> If I create a group specifying "--posix" the cli does create the group. >>> Then, using ipa group-find, I do not see any way to determine if that >>> group is a posixgroup or not. >>> group-find -all used to reveal a PosixGroup field. >>> >>> How do I determine if a group is a posix group or not? >>> >>> Michael- >> >> Ok, I suppose I could have looked at this before firing off an e-mail :-) >> >> I do see the group number when showing a group: >> >> $ ./ipa group-show g9 >> ----------- >> group-show: >> ----------- >> Group: g9 >> name: g9 >> description: test posix group >> group id: 1117 >> >> But when adding it this doesn't appear. Oddly enough we show the >> ipquniqueid when adding a group but not when showing it! >> >> Pavel, do you have time to investigate this inconsistency? >> >> rob > > Pavel > Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-all-to-LDAPCreate-and-make-LDAP-commands-alway.patch Type: application/mbox Size: 6987 bytes Desc: not available URL: From pzuna at redhat.com Fri Nov 13 15:39:20 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Fri, 13 Nov 2009 16:39:20 +0100 Subject: [Freeipa-devel] [PATCH] Fix a bunch of unit tests. Message-ID: <4AFD7DA8.5020606@redhat.com> Only pwpolicy test is still broken - I'm looking into it. Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-a-bunch-of-unit-tests.patch Type: application/mbox Size: 4800 bytes Desc: not available URL: From ssorce at redhat.com Fri Nov 13 15:51:06 2009 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 13 Nov 2009 10:51:06 -0500 Subject: [Freeipa-devel] How to implement Magic Private Groups in FreeIPA ? In-Reply-To: <4AFD7B9F.5080405@redhat.com> References: <1257983664.12148.135.camel@willson.li.ssimo.org> <4AFC2BD2.1030200@redhat.com> <1258042120.12148.1128.camel@willson.li.ssimo.org> <4AFD7B9F.5080405@redhat.com> Message-ID: <1258127466.12148.2605.camel@willson.li.ssimo.org> On Fri, 2009-11-13 at 10:30 -0500, Rob Crittenden wrote: > Simo Sorce wrote: > > On Thu, 2009-11-12 at 10:37 -0500, Dmitri Pal wrote: > >>> So killing two birds with one stone we are thinking of introducing a > >> new > >>> attribute called posixName that has a case sensitive syntax and does > >> not > >>> conflict with other uses of uid and cn. We will probably still set > >> uid > >>> on users and cn on groups but they will be kept in sync with > >> posixName > >>> (except for cn on user accounts that holds the full name). > >>> > >>> > >> So posixName will be a part of the user account object and group > >> object, > >> right? > >> Can you please add more details here? > > > > Correct, > > we would switch to primarily use posixName for users and groups names. > > > > A group entry would probably look like this (from memory): > > > > cn=newgroup,cn=groups,cn=accounts,dc=example,dc=com > > objectclass: nestedgroup > > objectclass: posixGroup > > objectclass: ipaPosixName > > cn: newgroup > > posixName: newgroup > > member: ... > > member: ... > > > > > > When searching for this group we would use a query like: > > '(&(objectClass=posixGroup)(posixName=newgroup))' > > > > Same for users. > > > > Simo. > > > > > > FYI, here is the new schema I've come up with: > > dn: cn=schema > attributeTypes: ( 2.16.840.1.113730.3.8.3.54 NAME 'posixName' EQUALITY > caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE) > objectClasses: ( 2.16.840.1.113730.3.8.3.55 NAME 'ipaPosixName' DESC > 'Case-sensitive name common to users and groups' AUXILIARY MUST ( > posixName ) X-ORIGIN 'IPA v2' ) > > It also occurs to me that we'll need to prevent any modifications to the > posixName attribute unless the cn/uid is also being modified. On other > word, sync needs to be 2-way. Yes, these attributes need to be "linked" somehow. Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Fri Nov 13 18:04:26 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 13 Nov 2009 13:04:26 -0500 Subject: [Freeipa-devel] Re: [PATCH] Add --all to LDAPCreate and make LDAP commands always display default attributes. In-Reply-To: <4AFD7CA9.3040902@redhat.com> References: <4AFD0C35.2050309@redhat.com> <4AFD6DB1.6010805@redhat.com> <4AFD7C53.2010904@redhat.com> <4AFD7CA9.3040902@redhat.com> Message-ID: <4AFD9FAA.1080707@redhat.com> Pavel Zuna wrote: > And here's the actual patch. :) > > Pavel Zuna wrote: >> This should fix the issue: >> >> Rob Crittenden wrote: >>> Michael Gregg wrote: >>>> >>>> Rob, did the support for posix groups change? >>>> >>>> If I create a group specifying "--posix" the cli does create the group. >>>> Then, using ipa group-find, I do not see any way to determine if >>>> that group is a posixgroup or not. >>>> group-find -all used to reveal a PosixGroup field. >>>> >>>> How do I determine if a group is a posix group or not? >>>> >>>> Michael- >>> >>> Ok, I suppose I could have looked at this before firing off an e-mail >>> :-) >>> >>> I do see the group number when showing a group: >>> >>> $ ./ipa group-show g9 >>> ----------- >>> group-show: >>> ----------- >>> Group: g9 >>> name: g9 >>> description: test posix group >>> group id: 1117 >>> >>> But when adding it this doesn't appear. Oddly enough we show the >>> ipquniqueid when adding a group but not when showing it! >>> >>> Pavel, do you have time to investigate this inconsistency? >>> >>> rob >> >> Pavel >> I'm not sure how this addresses the issue that when adding a group different values are returned than when you show one. This also causes a whole ton of tests to fail. I think in baseldap.oy instead of: if options['all']: You want: if options.get('all', False): rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Fri Nov 13 18:07:58 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 13 Nov 2009 13:07:58 -0500 Subject: [Freeipa-devel] [PATCH] Fix a bunch of unit tests. In-Reply-To: <4AFD7DA8.5020606@redhat.com> References: <4AFD7DA8.5020606@redhat.com> Message-ID: <4AFDA07E.9050008@redhat.com> Pavel Zuna wrote: > Only pwpolicy test is still broken - I'm looking into it. > > Pavel This brings up the return values question again. I thought we had decided that any attribute that had only one value would be returned as a scalar. In this case userCertificate is being returned as a list which is causing things to fail. Now arguably userCertificate is a multi-valued attribute but we will only store one certificate at a time there so I think we're ok. Also, why the change to the principal name in the service tests? rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Fri Nov 13 20:14:32 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 13 Nov 2009 15:14:32 -0500 Subject: [Freeipa-devel] [PATCH] 315 tab completion for ipa command Message-ID: <4AFDBE28.5070108@redhat.com> This adds bash tab completion for the ipa command. Note that this is sourced when you log in, so installing the package isn't enough to load this file. Alternatively you can do: % source /etc/bash_completion.d/ipa % ipa user user-add user-find user-mod user-unlock user-del user-lock user-show rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-315-tab.patch Type: application/mbox Size: 2843 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From pzuna at redhat.com Mon Nov 16 16:48:05 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Mon, 16 Nov 2009 17:48:05 +0100 Subject: [Freeipa-devel] Re: [PATCH] Add --all to LDAPCreate and make LDAP commands always display default attributes. In-Reply-To: <4AFD9FAA.1080707@redhat.com> References: <4AFD0C35.2050309@redhat.com> <4AFD6DB1.6010805@redhat.com> <4AFD7C53.2010904@redhat.com> <4AFD7CA9.3040902@redhat.com> <4AFD9FAA.1080707@redhat.com> Message-ID: <4B018245.5010308@redhat.com> Rob Crittenden wrote: > Pavel Zuna wrote: >> And here's the actual patch. :) >> >> Pavel Zuna wrote: >>> This should fix the issue: >>> >>> Rob Crittenden wrote: >>>> Michael Gregg wrote: >>>>> >>>>> Rob, did the support for posix groups change? >>>>> >>>>> If I create a group specifying "--posix" the cli does create the >>>>> group. >>>>> Then, using ipa group-find, I do not see any way to determine if >>>>> that group is a posixgroup or not. >>>>> group-find -all used to reveal a PosixGroup field. >>>>> >>>>> How do I determine if a group is a posix group or not? >>>>> >>>>> Michael- >>>> >>>> Ok, I suppose I could have looked at this before firing off an >>>> e-mail :-) >>>> >>>> I do see the group number when showing a group: >>>> >>>> $ ./ipa group-show g9 >>>> ----------- >>>> group-show: >>>> ----------- >>>> Group: g9 >>>> name: g9 >>>> description: test posix group >>>> group id: 1117 >>>> >>>> But when adding it this doesn't appear. Oddly enough we show the >>>> ipquniqueid when adding a group but not when showing it! >>>> >>>> Pavel, do you have time to investigate this inconsistency? >>>> >>>> rob >>> >>> Pavel >>> > > I'm not sure how this addresses the issue that when adding a group > different values are returned than when you show one. When an entry is created, we show the default attributes and all attributes that were created explicitly. Before this patch, it was just all attributes, that were created explicitly, so for example gid didn't show up on groups, because it was created by the DNA plugin. When showing an entry, we return the default attributes. Should I change LDAPCreate to only return default attributes? > This also causes a whole ton of tests to fail. I think in baseldap.oy > instead of: > > if options['all']: > > You want: > > if options.get('all', False): Some of the tests were failing before this patch. I submitted a fix for most of them. if options['all'] is fine, because --all is a Flag parameters and is required. > rob Pavel From pzuna at redhat.com Mon Nov 16 16:59:51 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Mon, 16 Nov 2009 17:59:51 +0100 Subject: [Freeipa-devel] [PATCH] Fix a bunch of unit tests. In-Reply-To: <4AFDA07E.9050008@redhat.com> References: <4AFD7DA8.5020606@redhat.com> <4AFDA07E.9050008@redhat.com> Message-ID: <4B018507.90403@redhat.com> Rob Crittenden wrote: > Pavel Zuna wrote: >> Only pwpolicy test is still broken - I'm looking into it. >> >> Pavel > > This brings up the return values question again. I thought we had > decided that any attribute that had only one value would be returned as > a scalar. In this case userCertificate is being returned as a list which > is causing things to fail. Now arguably userCertificate is a > multi-valued attribute but we will only store one certificate at a time > there so I think we're ok. Yeah, I remember, but I'm not sure if we agreed on the logic. There are 2 ways of doing this: 1) Make ldap backend check the schema. If it's multi-value - leave it as a list. If it's single-value - convert it to a scalar. 2) Make ldap backend check if the attribute contains 1 or more values. If there's only one, convert it to a scalar. With 1) plugin authors can depend on the schema when manipulating attributes, but they have to know the schema and handle multi/single attributes differently. With 2) plugin authors have to always check, if the attribute is a list or a scalar. I think that having attributes always returned as list makes things easier on plugin authors - no checks required, everything is handled the same way. What's the advantage of returning attribute values as 2 different types? > > Also, why the change to the principal name in the service tests? At first I didn't know where the problem in this test was. So, I tried a few different things and this is a leftover. Doesn't hurt anything, but I can always change it back. > rob Pavel From jderose at redhat.com Tue Nov 17 02:14:22 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Mon, 16 Nov 2009 19:14:22 -0700 Subject: [Freeipa-devel] Return values, CRUD, webUI Message-ID: <1258424062.6624.759.camel@jgd-dsk> The vast majority of our Command plugins subclass from one of the CRUD base classes, so in terms of return value consistency and API style, we need to focus most on them (and then adapt their style to the few non-CRUD commands). While hooking up the webUI there have been many, many small problems in the core library and plugins that have caused unexpected setbacks for me. Some features that I needed got changed without me noticing, some of my half-baked designs needed more baking, some features were missing, and some new code I was just unfamiliar with. Point is, I've spent a lot of time battling little gotchas and thinking about how best to clean these things up. Here are the guidelines I propose we follow: A return value dict =================== As much as possible, I want to keep our return values very simple and regular. This 1) makes our API easy to learn and use, and 2) makes it easy to use the return values to drive UI logic on both the CLI and webUI. One current source of irregularity is the need to pass the "this isn't all the entries" flag from LDAP when we do searches. For example, `user_find` returns an (entry_list, more_remains) tuple. The problem is that most of the code paths don't care about the `more_remains` flag... they just need to know whether a list of entries was returned (result is a list) or whether a single entry was returned (result is a dict). At the same time, we obviously need a way to pass extra data like the `more_remains` flag and it would be nice to be able to extend a return value with additional special data without breaking code or causing an explosion of special cases. So I propose that our return values always be a dict containing at least a 'result', leaving us the option to extend the return value without breaking code that just looks at ret['result']. So in the case of a search, instead of: ([{'uid': 'foo'}, {'uid': 'bar'}, ...], True) We should return: { 'result': [{'uid': 'foo'}, {'uid': 'bar'}, ...], 'more_remains': True, ... 'extend': 'as-needed', } The following all assume we are returning {'result': blah} even though they don't show it... Entries ======= 95% of our return values are LDAP entries. Currently we're returning pretty much the raw value from python-ldap (although we are decoding UTF-8 into `unicode` objects for use in the Python pipeline and encoding back to UTF-8 on the way out, which is good). But the data structure returned from python-ldap is pretty awkward to work with. First, at the top it's typically a (dn, entry) tuple. Assuming the 'dn' key doesn't conflict with any sane LDAP attribute names, I think we should return a single dict with the dn stored under the 'dn' key. So instead of: ('uid=jdoe,cn=users,cn=accounts,dc=example,dc=com', {'sn': ['Doe']}) We should return: {'dn': 'uid=jdoe,cn=users,cn=accounts,dc=example,dc=com', 'sn': ['Doe']} Second, currently we return all attribute values inside a list whether or not they're multi-value. This leads to lots of special cases throughout the code that would be better dealt with in a single place, in LDAP Backend adapter, IHMO. So instead of: {'uid': ['jdoe'], 'group': ['foo', 'bar']} We should return: {'uid': 'jdoe', 'group': ['foo', 'bar']} Lists of Entries ================ When a command returns multiple entries, the entries should be in the same form as they are from commands that return only one entry. For example, currently user-find returns each entry as a (uid, entry) tuple. I think this should again be replaced with a single dict without the uid being duplicated. Create ====== If successful, we should return the resulting entry in standard form. If any error occurs, we should raise an appropriate exception. Retrieve ======== If successful, we should return the entry in standard form. If no such entry exists we should raise a NotFound exception. If any other error occurs, we should raise an appropriate exception. Update ====== (Same as Create.) Delete ====== (Same as Retrieve.) Search ====== If one or more entries matches the search criteria, we should return a list of entries, where the each entry is in standard form. If no entries match, we should return an empty list. If an error occurs, we should raise an appropriate exception. Thoughts? -Jason From rcritten at redhat.com Tue Nov 17 16:08:03 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 17 Nov 2009 11:08:03 -0500 Subject: [Freeipa-devel] Return values, CRUD, webUI In-Reply-To: <1258424062.6624.759.camel@jgd-dsk> References: <1258424062.6624.759.camel@jgd-dsk> Message-ID: <4B02CA63.5050905@redhat.com> Jason Gerard DeRose wrote: > The vast majority of our Command plugins subclass from one of the CRUD > base classes, so in terms of return value consistency and API style, we > need to focus most on them (and then adapt their style to the few > non-CRUD commands). > > While hooking up the webUI there have been many, many small problems in > the core library and plugins that have caused unexpected setbacks for > me. Some features that I needed got changed without me noticing, some > of my half-baked designs needed more baking, some features were missing, > and some new code I was just unfamiliar with. Point is, I've spent a > lot of time battling little gotchas and thinking about how best to clean > these things up. Here are the guidelines I propose we follow: > > > A return value dict > =================== > > As much as possible, I want to keep our return values very simple and > regular. This 1) makes our API easy to learn and use, and 2) makes it > easy to use the return values to drive UI logic on both the CLI and > webUI. > > One current source of irregularity is the need to pass the "this isn't > all the entries" flag from LDAP when we do searches. For example, > `user_find` returns an (entry_list, more_remains) tuple. The problem is > that most of the code paths don't care about the `more_remains` flag... > they just need to know whether a list of entries was returned (result is > a list) or whether a single entry was returned (result is a dict). > > At the same time, we obviously need a way to pass extra data like the > `more_remains` flag and it would be nice to be able to extend a return > value with additional special data without breaking code or causing an > explosion of special cases. So I propose that our return values always > be a dict containing at least a 'result', leaving us the option to > extend the return value without breaking code that just looks at > ret['result']. > > So in the case of a search, instead of: > > ([{'uid': 'foo'}, {'uid': 'bar'}, ...], True) > > We should return: > > { > 'result': [{'uid': 'foo'}, {'uid': 'bar'}, ...], > 'more_remains': True, > ... > 'extend': 'as-needed', > } > > The following all assume we are returning {'result': blah} even though > they don't show it... Well, I think that we'll always need some sort of special value to indicate when a limit has been exceeded. In v1 we decided to return what we got instead of failing the entire search. This proposal works for me. > > > Entries > ======= > > 95% of our return values are LDAP entries. Currently we're returning > pretty much the raw value from python-ldap (although we are decoding > UTF-8 into `unicode` objects for use in the Python pipeline and encoding > back to UTF-8 on the way out, which is good). But the data structure > returned from python-ldap is pretty awkward to work with. > > First, at the top it's typically a (dn, entry) tuple. Assuming the 'dn' > key doesn't conflict with any sane LDAP attribute names, I think we > should return a single dict with the dn stored under the 'dn' key. > > So instead of: > > ('uid=jdoe,cn=users,cn=accounts,dc=example,dc=com', {'sn': ['Doe']}) > > We should return: > > {'dn': 'uid=jdoe,cn=users,cn=accounts,dc=example,dc=com', 'sn': ['Doe']} I don't know, it is kinda handy to have the dn separate. But if it makes things more uniform I can deal with it. The advantage of having it separate is it draws a clear line between what you can change and what you can't. We'd have to strip out dn from updates and that might be confusing for some users. I suppose what we could do is have the LDAP module return a specific error if dn is included in an update. > > Second, currently we return all attribute values inside a list whether > or not they're multi-value. This leads to lots of special cases > throughout the code that would be better dealt with in a single place, > in LDAP Backend adapter, IHMO. > > So instead of: > > {'uid': ['jdoe'], 'group': ['foo', 'bar']} > > We should return: > > {'uid': 'jdoe', 'group': ['foo', 'bar']} Yes, I agree. What do you propose for a multi-valued attribute with a single value, returned as a list or a scalar? In v1 we returned any single value as a scalar and any multi-valued attr as a list. > > > Lists of Entries > ================ > > When a command returns multiple entries, the entries should be in the > same form as they are from commands that return only one entry. For > example, currently user-find returns each entry as a (uid, entry) tuple. > I think this should again be replaced with a single dict without the uid > being duplicated. Same but just within a list, right? So multiple entries are a list of dicts that represent the result. > > Create > ====== > > If successful, we should return the resulting entry in standard form. > If any error occurs, we should raise an appropriate exception. I think that in early versions of the framework we would return the results of Retrieve if the creation was successful. I think we should do that again in order to provide uniformity. > > Retrieve > ======== > > If successful, we should return the entry in standard form. If no such > entry exists we should raise a NotFound exception. If any other error > occurs, we should raise an appropriate exception. +1 > > > Update > ====== > > (Same as Create.) +1, with a Retrieve. > > Delete > ====== > > (Same as Retrieve.) Well, delete is a bit different. It either succeeds or fails. I'm not sure what we should return here, perhaps a boolean True. > > Search > ====== > > If one or more entries matches the search criteria, we should return a > list of entries, where the each entry is in standard form. If no > entries match, we should return an empty list. If an error occurs, we > should raise an appropriate exception. Sure. I'm ok with either return an empty list or an exception. In any case the caller is going to have to handle this case. I don't think this represents a lot of work but will go a long way to helping our API. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From jderose at redhat.com Tue Nov 17 16:58:31 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Tue, 17 Nov 2009 09:58:31 -0700 Subject: [Freeipa-devel] [PATCH] 307 enforce scalar In-Reply-To: <4AF193C0.8030606@redhat.com> References: <4AF193C0.8030606@redhat.com> Message-ID: <1258477111.15021.0.camel@jgd-dsk> On Wed, 2009-11-04 at 09:46 -0500, Rob Crittenden wrote: > _convert_scalar() should not handle tuples/lists (by definition). A > parameter may be mutivalued but even then _convert_scalar() gets the > values one at a time. > > rob ack. pushed to master. From jderose at redhat.com Tue Nov 17 16:58:52 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Tue, 17 Nov 2009 09:58:52 -0700 Subject: [Freeipa-devel] [PATCH] 308 manage arbitrary attributes In-Reply-To: <4AF9A2CE.2070900@redhat.com> References: <4AF1EC42.5000603@redhat.com> <1257398378.20441.1.camel@jgd-dsk> <4AF9A2CE.2070900@redhat.com> Message-ID: <1258477132.15021.1.camel@jgd-dsk> On Tue, 2009-11-10 at 12:28 -0500, Rob Crittenden wrote: > Jason Gerard DeRose wrote: > > Oops, was this missing the attachment? ;) > > Bah, here it is. > > rob ack. pushed to master. > > > > On Wed, 2009-11-04 at 16:04 -0500, Rob Crittenden wrote: > >> This adds 2 new parameters, --setattr and --addattr and lets you manage > >> whatever attribute you want (within the given set of objectclasses). > >> > >> Both take a name/value pair. > >> > >> --setattr sets the attribute to the given value > >> --addattr adds the value to an attribute. Can be used to manage > >> multi-valued attributes > >> > >> For example: > >> > >> ipa user-mod --addattr=postalcode=90210 jsmith > >> > >> If the attribute to be modified is an another param then the value is > >> silently dropped. > >> > >> You can include multiples of these on a single command-line: > >> > >> ipa user-mod --addattr=postalcode=20601 --addattr=postalcode=30330 jsmith > >> > >> Setting an attribute to "" deletes it: > >> > >> ipa user-mod --setattr=postalcode= jsmith > >> > >> rob > >> _______________________________________________ > >> Freeipa-devel mailing list > >> Freeipa-devel at redhat.com > >> https://www.redhat.com/mailman/listinfo/freeipa-devel > > > From jderose at redhat.com Tue Nov 17 17:03:41 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Tue, 17 Nov 2009 10:03:41 -0700 Subject: [Freeipa-devel] [PATCH] 309 make exception from ipautil.run() optional In-Reply-To: <4AFAE928.1020801@redhat.com> References: <4AFAE737.7090507@redhat.com> <4AFAE928.1020801@redhat.com> Message-ID: <1258477421.15021.2.camel@jgd-dsk> On Wed, 2009-11-11 at 11:41 -0500, Rob Crittenden wrote: > Rob Crittenden wrote: > > There are probably occasions where a caller will want more control over > > what happens when running a command fails. I've added an optional > > argument to run where it will not raise an exception on errors. > > > > I've also added returncode to the tuple of things returned. > > > > rob > > I forgot to include this additional change in the patch. When acked I'll > add this bit too and commit it. > > --- a/ipaserver/install/httpinstance.py > +++ b/ipaserver/install/httpinstance.py > @@ -100,7 +100,7 @@ class HTTPInstance(service.Service): > if selinux: > try: > # returns e.g. "httpd_can_network_connect --> off" > - (stdout, stderr) = ipautil.run(["/usr/sbin/getsebool", > + (stdout, stderr, returncode) = > ipautil.run(["/usr/sbin/getsebool", > > "httpd_can_network_connect"]) > self.backup_state("httpd_can_network_connect", > stdout.split()[2]) > except: ack. It all looks fine to me, although I can't get this patch to apply. From rcritten at redhat.com Tue Nov 17 19:58:52 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 17 Nov 2009 14:58:52 -0500 Subject: [Freeipa-devel] [PATCH] 309 make exception from ipautil.run() optional In-Reply-To: <1258477421.15021.2.camel@jgd-dsk> References: <4AFAE737.7090507@redhat.com> <4AFAE928.1020801@redhat.com> <1258477421.15021.2.camel@jgd-dsk> Message-ID: <4B03007C.6050109@redhat.com> Jason Gerard DeRose wrote: > On Wed, 2009-11-11 at 11:41 -0500, Rob Crittenden wrote: >> Rob Crittenden wrote: >>> There are probably occasions where a caller will want more control over >>> what happens when running a command fails. I've added an optional >>> argument to run where it will not raise an exception on errors. >>> >>> I've also added returncode to the tuple of things returned. >>> >>> rob >> I forgot to include this additional change in the patch. When acked I'll >> add this bit too and commit it. >> >> --- a/ipaserver/install/httpinstance.py >> +++ b/ipaserver/install/httpinstance.py >> @@ -100,7 +100,7 @@ class HTTPInstance(service.Service): >> if selinux: >> try: >> # returns e.g. "httpd_can_network_connect --> off" >> - (stdout, stderr) = ipautil.run(["/usr/sbin/getsebool", >> + (stdout, stderr, returncode) = >> ipautil.run(["/usr/sbin/getsebool", >> >> "httpd_can_network_connect"]) >> self.backup_state("httpd_can_network_connect", >> stdout.split()[2]) >> except: > > ack. It all looks fine to me, although I can't get this patch to apply. > Ok, it is failing because it relies on a patch I didn't submit yet. I'll send that one out shortly and hold onto this one for now. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Tue Nov 17 20:06:00 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 17 Nov 2009 15:06:00 -0500 Subject: [Freeipa-devel] [PATCH] 285 CRL publishing Message-ID: <4B030228.8010409@redhat.com> This enables CRL publishing by dogtag to a place where Apache can get the files. I have to do a couple of tricks here because dogtag is an optional component. This is why in the installer I first see if the dogtag SELinux policy is installed and if not add it. Similarly the installer will remove it upon uninstall. The policy itself just lets dogtag write to some Apache-labeled directories. dogtag uses symlinks to mark the latest CRL hence the permissions for links. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-285-crl.patch Type: application/mbox Size: 5620 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Tue Nov 17 20:45:02 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 17 Nov 2009 15:45:02 -0500 Subject: [Freeipa-devel] [PATCH] Fix a bunch of unit tests. In-Reply-To: <4B018507.90403@redhat.com> References: <4AFD7DA8.5020606@redhat.com> <4AFDA07E.9050008@redhat.com> <4B018507.90403@redhat.com> Message-ID: <4B030B4E.2040400@redhat.com> Pavel Zuna wrote: > Rob Crittenden wrote: >> Pavel Zuna wrote: >>> Only pwpolicy test is still broken - I'm looking into it. >>> >>> Pavel >> >> This brings up the return values question again. I thought we had >> decided that any attribute that had only one value would be returned >> as a scalar. In this case userCertificate is being returned as a list >> which is causing things to fail. Now arguably userCertificate is a >> multi-valued attribute but we will only store one certificate at a >> time there so I think we're ok. > Yeah, I remember, but I'm not sure if we agreed on the logic. > > There are 2 ways of doing this: > > 1) Make ldap backend check the schema. If it's multi-value - leave it as > a list. If it's single-value - convert it to a scalar. > > 2) Make ldap backend check if the attribute contains 1 or more values. > If there's only one, convert it to a scalar. > > With 1) plugin authors can depend on the schema when manipulating > attributes, but they have to know the schema and handle multi/single > attributes differently. Yes but they already have to deal with it/be aware of it because updates may fail if you try to add another value to a single-valued attribute. > > With 2) plugin authors have to always check, if the attribute is a list > or a scalar. Not necessarly. If the author has some awareness of the schema they can get by ok. > I think that having attributes always returned as list makes things > easier on plugin authors - no checks required, everything is handled the > same way. What's the advantage of returning attribute values as 2 > different types? Because some values are single-value by nature and we're treating them like multi-values. >> >> Also, why the change to the principal name in the service tests? > At first I didn't know where the problem in this test was. So, I tried a > few different things and this is a leftover. Doesn't hurt anything, but > I can always change it back. Yes, please do. You're effectively adding a subdomain onto the hostname. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Tue Nov 17 20:56:34 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 17 Nov 2009 15:56:34 -0500 Subject: [Freeipa-devel] Re: [PATCH] Add --all to LDAPCreate and make LDAP commands always display default attributes. In-Reply-To: <4B018245.5010308@redhat.com> References: <4AFD0C35.2050309@redhat.com> <4AFD6DB1.6010805@redhat.com> <4AFD7C53.2010904@redhat.com> <4AFD7CA9.3040902@redhat.com> <4AFD9FAA.1080707@redhat.com> <4B018245.5010308@redhat.com> Message-ID: <4B030E02.4040100@redhat.com> Pavel Zuna wrote: > Rob Crittenden wrote: >> Pavel Zuna wrote: >>> And here's the actual patch. :) >>> >>> Pavel Zuna wrote: >>>> This should fix the issue: >>>> >>>> Rob Crittenden wrote: >>>>> Michael Gregg wrote: >>>>>> >>>>>> Rob, did the support for posix groups change? >>>>>> >>>>>> If I create a group specifying "--posix" the cli does create the >>>>>> group. >>>>>> Then, using ipa group-find, I do not see any way to determine if >>>>>> that group is a posixgroup or not. >>>>>> group-find -all used to reveal a PosixGroup field. >>>>>> >>>>>> How do I determine if a group is a posix group or not? >>>>>> >>>>>> Michael- >>>>> >>>>> Ok, I suppose I could have looked at this before firing off an >>>>> e-mail :-) >>>>> >>>>> I do see the group number when showing a group: >>>>> >>>>> $ ./ipa group-show g9 >>>>> ----------- >>>>> group-show: >>>>> ----------- >>>>> Group: g9 >>>>> name: g9 >>>>> description: test posix group >>>>> group id: 1117 >>>>> >>>>> But when adding it this doesn't appear. Oddly enough we show the >>>>> ipquniqueid when adding a group but not when showing it! >>>>> >>>>> Pavel, do you have time to investigate this inconsistency? >>>>> >>>>> rob >>>> >>>> Pavel >>>> >> >> I'm not sure how this addresses the issue that when adding a group >> different values are returned than when you show one. > When an entry is created, we show the default attributes and all > attributes that were created explicitly. Before this patch, it was just > all attributes, that were created explicitly, so for example gid didn't > show up on groups, because it was created by the DNA plugin. > > When showing an entry, we return the default attributes. > > Should I change LDAPCreate to only return default attributes? No. I understand the problem now. I think in earlier versions we were actually doing a lookup of the entry after creation and returning that. This would resolve the problem. >> This also causes a whole ton of tests to fail. I think in baseldap.oy >> instead of: >> >> if options['all']: >> >> You want: >> >> if options.get('all', False): > Some of the tests were failing before this patch. I submitted a fix for > most of them. > > if options['all'] is fine, because --all is a Flag parameters and is > required. The service plugin overrides takes_options() in some cases, hence no 'all. Probably something to fix but we still should handle this case (all not in options). rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From pzuna at redhat.com Wed Nov 18 12:53:13 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Wed, 18 Nov 2009 13:53:13 +0100 Subject: [Freeipa-devel] [PATCH] Fix a bunch of unit tests. In-Reply-To: <4B030B4E.2040400@redhat.com> References: <4AFD7DA8.5020606@redhat.com> <4AFDA07E.9050008@redhat.com> <4B018507.90403@redhat.com> <4B030B4E.2040400@redhat.com> Message-ID: <4B03EE39.6070409@redhat.com> Rob Crittenden wrote: > Pavel Zuna wrote: >> Rob Crittenden wrote: >>> Pavel Zuna wrote: >>>> Only pwpolicy test is still broken - I'm looking into it. >>>> >>>> Pavel >>> >>> This brings up the return values question again. I thought we had >>> decided that any attribute that had only one value would be returned >>> as a scalar. In this case userCertificate is being returned as a list >>> which is causing things to fail. Now arguably userCertificate is a >>> multi-valued attribute but we will only store one certificate at a >>> time there so I think we're ok. >> Yeah, I remember, but I'm not sure if we agreed on the logic. >> >> There are 2 ways of doing this: >> >> 1) Make ldap backend check the schema. If it's multi-value - leave it >> as a list. If it's single-value - convert it to a scalar. >> >> 2) Make ldap backend check if the attribute contains 1 or more values. >> If there's only one, convert it to a scalar. >> >> With 1) plugin authors can depend on the schema when manipulating >> attributes, but they have to know the schema and handle multi/single >> attributes differently. > > Yes but they already have to deal with it/be aware of it because updates > may fail if you try to add another value to a single-valued attribute. Yeah, the command will fail, but at least the code won't blow up. >> >> With 2) plugin authors have to always check, if the attribute is a >> list or a scalar. > > Not necessarly. If the author has some awareness of the schema they can > get by ok. That's true for single-value attributes. Multi-value attributes with only one value will get converted to scalars. Also, when handling a set of attributes (in a loop for example, which is the case most of the time), we still have to check every single one: for a in attrs: if isinstance(a, (list, tuple)): # do something else: # do something Or (and I've seen this A LOT in old code): for a in attrs: if not isinstance(a, (list, tuple)): a = [a] # do something Instead of: for a in attrs: # do something > >> I think that having attributes always returned as list makes things >> easier on plugin authors - no checks required, everything is handled >> the same way. What's the advantage of returning attribute values as 2 >> different types? > > Because some values are single-value by nature and we're treating them > like multi-values. Ok, fair enough, I'll make the change. I don't really mind that much, I just prefer generic solutions, where everything handles the same. :) >>> >>> Also, why the change to the principal name in the service tests? >> At first I didn't know where the problem in this test was. So, I tried >> a few different things and this is a leftover. Doesn't hurt anything, >> but I can always change it back. > > Yes, please do. You're effectively adding a subdomain onto the hostname. I just checked and I'm not adding any subdomains. This patch changes the hostname from 'ipatest.$IPA_DOMAIN' to 'ipatest.test' and we no longer make the assumption the host already exists (the test was failing on my machine, because it didn't exist). > rob Pavel From pzuna at redhat.com Wed Nov 18 13:03:20 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Wed, 18 Nov 2009 14:03:20 +0100 Subject: [Freeipa-devel] Re: [PATCH] Add --all to LDAPCreate and make LDAP commands always display default attributes. In-Reply-To: <4B030E02.4040100@redhat.com> References: <4AFD0C35.2050309@redhat.com> <4AFD6DB1.6010805@redhat.com> <4AFD7C53.2010904@redhat.com> <4AFD7CA9.3040902@redhat.com> <4AFD9FAA.1080707@redhat.com> <4B018245.5010308@redhat.com> <4B030E02.4040100@redhat.com> Message-ID: <4B03F098.9060707@redhat.com> Rob Crittenden wrote: > Pavel Zuna wrote: >> Rob Crittenden wrote: >>> Pavel Zuna wrote: >>>> And here's the actual patch. :) >>>> >>>> Pavel Zuna wrote: >>>>> This should fix the issue: >>>>> >>>>> Rob Crittenden wrote: >>>>>> Michael Gregg wrote: >>>>>>> >>>>>>> Rob, did the support for posix groups change? >>>>>>> >>>>>>> If I create a group specifying "--posix" the cli does create the >>>>>>> group. >>>>>>> Then, using ipa group-find, I do not see any way to determine if >>>>>>> that group is a posixgroup or not. >>>>>>> group-find -all used to reveal a PosixGroup field. >>>>>>> >>>>>>> How do I determine if a group is a posix group or not? >>>>>>> >>>>>>> Michael- >>>>>> >>>>>> Ok, I suppose I could have looked at this before firing off an >>>>>> e-mail :-) >>>>>> >>>>>> I do see the group number when showing a group: >>>>>> >>>>>> $ ./ipa group-show g9 >>>>>> ----------- >>>>>> group-show: >>>>>> ----------- >>>>>> Group: g9 >>>>>> name: g9 >>>>>> description: test posix group >>>>>> group id: 1117 >>>>>> >>>>>> But when adding it this doesn't appear. Oddly enough we show the >>>>>> ipquniqueid when adding a group but not when showing it! >>>>>> >>>>>> Pavel, do you have time to investigate this inconsistency? >>>>>> >>>>>> rob >>>>> >>>>> Pavel >>>>> >>> >>> I'm not sure how this addresses the issue that when adding a group >>> different values are returned than when you show one. >> When an entry is created, we show the default attributes and all >> attributes that were created explicitly. Before this patch, it was >> just all attributes, that were created explicitly, so for example gid >> didn't show up on groups, because it was created by the DNA plugin. >> >> When showing an entry, we return the default attributes. >> >> Should I change LDAPCreate to only return default attributes? > > No. I understand the problem now. I think in earlier versions we were > actually doing a lookup of the entry after creation and returning that. > This would resolve the problem. We're still doing it the same way. Add, then lookup the entry we just created, but when we lookup an entry, we have to specify a list of attributes we are interested in. Currently: In *-show, we request LDAPObject.default_attributes. In *-add, we request LDAPObject.default_attributes + attributes passed to add_entry. Or all attributes in both, if --all was present. > >>> This also causes a whole ton of tests to fail. I think in baseldap.oy >>> instead of: >>> >>> if options['all']: >>> >>> You want: >>> >>> if options.get('all', False): >> Some of the tests were failing before this patch. I submitted a fix >> for most of them. >> >> if options['all'] is fine, because --all is a Flag parameters and is >> required. > > The service plugin overrides takes_options() in some cases, hence no > 'all. Probably something to fix but we still should handle this case > (all not in options). Okey, I'll go the fail-safe way and assume 'all' might not be there. takes_options in the service plugin need fixing. > > rob Pavel From pzuna at redhat.com Wed Nov 18 14:15:57 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Wed, 18 Nov 2009 15:15:57 +0100 Subject: [Freeipa-devel] Return values, CRUD, webUI In-Reply-To: <1258424062.6624.759.camel@jgd-dsk> References: <1258424062.6624.759.camel@jgd-dsk> Message-ID: <4B04019D.2060405@redhat.com> Jason Gerard DeRose wrote: > The vast majority of our Command plugins subclass from one of the CRUD > base classes, so in terms of return value consistency and API style, we > need to focus most on them (and then adapt their style to the few > non-CRUD commands). > > While hooking up the webUI there have been many, many small problems in > the core library and plugins that have caused unexpected setbacks for > me. Some features that I needed got changed without me noticing, some > of my half-baked designs needed more baking, some features were missing, > and some new code I was just unfamiliar with. Point is, I've spent a > lot of time battling little gotchas and thinking about how best to clean > these things up. Here are the guidelines I propose we follow: > > > A return value dict > =================== > > As much as possible, I want to keep our return values very simple and > regular. This 1) makes our API easy to learn and use, and 2) makes it > easy to use the return values to drive UI logic on both the CLI and > webUI. > > One current source of irregularity is the need to pass the "this isn't > all the entries" flag from LDAP when we do searches. For example, > `user_find` returns an (entry_list, more_remains) tuple. The problem is > that most of the code paths don't care about the `more_remains` flag... > they just need to know whether a list of entries was returned (result is > a list) or whether a single entry was returned (result is a dict). > > At the same time, we obviously need a way to pass extra data like the > `more_remains` flag and it would be nice to be able to extend a return > value with additional special data without breaking code or causing an > explosion of special cases. So I propose that our return values always > be a dict containing at least a 'result', leaving us the option to > extend the return value without breaking code that just looks at > ret['result']. > > So in the case of a search, instead of: > > ([{'uid': 'foo'}, {'uid': 'bar'}, ...], True) > > We should return: > > { > 'result': [{'uid': 'foo'}, {'uid': 'bar'}, ...], > 'more_remains': True, > ... > 'extend': 'as-needed', > } > > The following all assume we are returning {'result': blah} even though > they don't show it... > > > Entries > ======= > > 95% of our return values are LDAP entries. Currently we're returning > pretty much the raw value from python-ldap (although we are decoding > UTF-8 into `unicode` objects for use in the Python pipeline and encoding > back to UTF-8 on the way out, which is good). But the data structure > returned from python-ldap is pretty awkward to work with. > > First, at the top it's typically a (dn, entry) tuple. Assuming the 'dn' > key doesn't conflict with any sane LDAP attribute names, I think we > should return a single dict with the dn stored under the 'dn' key. > > So instead of: > > ('uid=jdoe,cn=users,cn=accounts,dc=example,dc=com', {'sn': ['Doe']}) > > We should return: > > {'dn': 'uid=jdoe,cn=users,cn=accounts,dc=example,dc=com', 'sn': ['Doe']} > > Second, currently we return all attribute values inside a list whether > or not they're multi-value. This leads to lots of special cases > throughout the code that would be better dealt with in a single place, > in LDAP Backend adapter, IHMO. > > So instead of: > > {'uid': ['jdoe'], 'group': ['foo', 'bar']} > > We should return: > > {'uid': 'jdoe', 'group': ['foo', 'bar']} This is the only part I don't really agree with. See this thread for my reasons: [Freeipa-devel] [PATCH] Add --all to LDAPCreate and make LDAP commands always display default attributes. But since both you and Rob seem to agree, I won't stand in the way and just make the change. :) > > Lists of Entries > ================ > > When a command returns multiple entries, the entries should be in the > same form as they are from commands that return only one entry. For > example, currently user-find returns each entry as a (uid, entry) tuple. > I think this should again be replaced with a single dict without the uid > being duplicated. > > > Create > ====== > > If successful, we should return the resulting entry in standard form. > If any error occurs, we should raise an appropriate exception. > > > Retrieve > ======== > > If successful, we should return the entry in standard form. If no such > entry exists we should raise a NotFound exception. If any other error > occurs, we should raise an appropriate exception. > > > Update > ====== > > (Same as Create.) > > > Delete > ====== > > (Same as Retrieve.) > After a delete, there is no entry to return. Unless an exception was raised, I think we should just report 'delete successful' to the user and be done with it. > > Search > ====== > > If one or more entries matches the search criteria, we should return a > list of entries, where the each entry is in standard form. If no > entries match, we should return an empty list. If an error occurs, we > should raise an appropriate exception. > > > Thoughts? > > -Jason > It's going to take a while to implement (and test!) all these changes, but I should have it done by next week. Pavel From rcritten at redhat.com Wed Nov 18 15:04:54 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 18 Nov 2009 10:04:54 -0500 Subject: [Freeipa-devel] [PATCH] Fix a bunch of unit tests. In-Reply-To: <4B03EE39.6070409@redhat.com> References: <4AFD7DA8.5020606@redhat.com> <4AFDA07E.9050008@redhat.com> <4B018507.90403@redhat.com> <4B030B4E.2040400@redhat.com> <4B03EE39.6070409@redhat.com> Message-ID: <4B040D16.30700@redhat.com> Pavel Zuna wrote: > Rob Crittenden wrote: >> Pavel Zuna wrote: >>> Rob Crittenden wrote: >>>> Pavel Zuna wrote: >>>>> Only pwpolicy test is still broken - I'm looking into it. >>>>> >>>>> Pavel >>>> >>>> This brings up the return values question again. I thought we had >>>> decided that any attribute that had only one value would be returned >>>> as a scalar. In this case userCertificate is being returned as a >>>> list which is causing things to fail. Now arguably userCertificate >>>> is a multi-valued attribute but we will only store one certificate >>>> at a time there so I think we're ok. >>> Yeah, I remember, but I'm not sure if we agreed on the logic. >>> >>> There are 2 ways of doing this: >>> >>> 1) Make ldap backend check the schema. If it's multi-value - leave it >>> as a list. If it's single-value - convert it to a scalar. >>> >>> 2) Make ldap backend check if the attribute contains 1 or more >>> values. If there's only one, convert it to a scalar. >>> >>> With 1) plugin authors can depend on the schema when manipulating >>> attributes, but they have to know the schema and handle multi/single >>> attributes differently. >> >> Yes but they already have to deal with it/be aware of it because >> updates may fail if you try to add another value to a single-valued >> attribute. > Yeah, the command will fail, but at least the code won't blow up. > >>> >>> With 2) plugin authors have to always check, if the attribute is a >>> list or a scalar. >> >> Not necessarly. If the author has some awareness of the schema they >> can get by ok. > That's true for single-value attributes. Multi-value attributes with > only one value will get converted to scalars. Also, when handling a set > of attributes (in a loop for example, which is the case most of the > time), we still have to check every single one: > > for a in attrs: > if isinstance(a, (list, tuple)): > # do something > else: > # do something > > Or (and I've seen this A LOT in old code): > > for a in attrs: > if not isinstance(a, (list, tuple)): > a = [a] > # do something > > Instead of: > > for a in attrs: > # do something > >> >>> I think that having attributes always returned as list makes things >>> easier on plugin authors - no checks required, everything is handled >>> the same way. What's the advantage of returning attribute values as 2 >>> different types? >> >> Because some values are single-value by nature and we're treating them >> like multi-values. > Ok, fair enough, I'll make the change. I don't really mind that much, I > just prefer generic solutions, where everything handles the same. :) > >>>> >>>> Also, why the change to the principal name in the service tests? >>> At first I didn't know where the problem in this test was. So, I >>> tried a few different things and this is a leftover. Doesn't hurt >>> anything, but I can always change it back. >> >> Yes, please do. You're effectively adding a subdomain onto the hostname. > I just checked and I'm not adding any subdomains. This patch changes the > hostname from 'ipatest.$IPA_DOMAIN' to 'ipatest.test' and we no longer > make the assumption the host already exists (the test was failing on my > machine, because it didn't exist). Yeah, you're right, I misread the patch. I though it was still sticking in the domain name. In any case hostname this change isn't needed. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From pzuna at redhat.com Wed Nov 18 16:50:03 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Wed, 18 Nov 2009 17:50:03 +0100 Subject: [Freeipa-devel] [PATCH] Add {user, host, sourcehost}Category to HBAC and make accessTime multivalue. Message-ID: <4B0425BB.3030700@redhat.com> Due to the format of accessTime (it has commas and spaces in it), we can't use the List parameter type. I made it so that accessTime values have to be entered one by one using new commands. We also agreed, that we're going to rename GeneralizedTime parameter to AccessTime to prevent confusion with RFC 4517 standard. I attached a separate patch for clarity. Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-user-host-sourcehost-Category-to-HBAC-and-make.patch Type: application/mbox Size: 6230 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Rename-GeneralizedTime-to-AccessTime.patch Type: application/mbox Size: 3299 bytes Desc: not available URL: From rcritten at redhat.com Wed Nov 18 19:14:08 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 18 Nov 2009 14:14:08 -0500 Subject: [Freeipa-devel] [PATCH] 270 handle all exceptions in XML-RPC server In-Reply-To: <1252679726.3046.76.camel@localhost.localdomain> References: <4AAA5812.7030102@redhat.com> <1252679726.3046.76.camel@localhost.localdomain> Message-ID: <4B044780.1070400@redhat.com> Simo Sorce wrote: > On Fri, 2009-09-11 at 10:00 -0400, Rob Crittenden wrote: >> Add a fail-safe exception handler to ensure that we destroy the >> request >> context and don't return dirty laundry to a client. If we don't do >> this >> then this process/thread will blow up the next time it handles a >> request >> because a context already exists. > > ack > pushed to master -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Wed Nov 18 19:26:46 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 18 Nov 2009 14:26:46 -0500 Subject: [Freeipa-devel] [PATCH] Fix sasl mappings In-Reply-To: <20091005152057.GH3006@localhost.localdomain> References: <1254242595.3017.10.camel@localhost.localdomain> <20091005152057.GH3006@localhost.localdomain> Message-ID: <4B044A76.5010006@redhat.com> Sumit Bose wrote: > On Tue, Sep 29, 2009 at 12:43:15PM -0400, Simo Sorce wrote: >> Sasl mappings never worked properly with full principal names. >> This patch fixes the problem. >> >> See bug#526284 >> >> Should probably commit this patch also against v1. >> >> Simo. > > this patch works for me with v1 and v2. > > ACK pushed to master -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Wed Nov 18 19:28:54 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 18 Nov 2009 14:28:54 -0500 Subject: [Freeipa-devel] [PATCH] 286 cache installer questions In-Reply-To: <20091012205220.3e65a09a@notas> References: <4ACCAC74.1080204@redhat.com> <1255328557.29221.35.camel@wolverine.englab.brq.redhat.com> <4AD328DF.4090106@redhat.com> <20091012205220.3e65a09a@notas> Message-ID: <4B044AF6.30904@redhat.com> Martin Nagy wrote: > Rob Crittenden wrote: >> Martin Nagy wrote: >>> Hi Rob, >>> >>> On Wed, 2009-10-07 at 10:57 -0400, Rob Crittenden wrote: >>>> Installing a CA that is signed by another CA is a 2-step process. The >>>> first step is to generate a CSR for the CA and the second step is to >>>> install the certificate issued by the external CA. To avoid asking >>>> questions over and over (and potentially getting different answers) the >>>> answers are cached. >>>> >>>> rob >>> - try: >>> - from ipaserver.install import cainstance >>> - except ImportError: >>> - print >> sys.stderr, "Import failed: %s" % sys.exc_value >>> - sys.exit(1) >>> >>> What's this? From just a quick look, it seems to me that this will cause >>> troubles. >>> >>> Martin >>> >> It was duplicated code, we check for that elsewhere. >> >> rob > > Ah, right. Not thrilled that it's not in a separate patch but what the > heck. Ack. > > Martin pushed to master -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Wed Nov 18 19:34:32 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 18 Nov 2009 14:34:32 -0500 Subject: [Freeipa-devel] [PATCH] Filter all NULL values in ldap2.add_entry. python-ldap doesn't like'em. In-Reply-To: <4AE1BF4D.9030905@redhat.com> References: <4AE1BF4D.9030905@redhat.com> Message-ID: <4B044C48.8070901@redhat.com> Pavel Z?na wrote: > Previously we only filtered None values, but it turns out that's not > enough. For example, empty lists/tuples generate ProtocolErrors. > > Pavel ack, pushed to master -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Wed Nov 18 19:35:57 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 18 Nov 2009 14:35:57 -0500 Subject: [Freeipa-devel] [PATCH] Make ldap2.convert_attr_synonyms more robust against schema lookup fails. In-Reply-To: <4AE9C376.9090906@redhat.com> References: <4AE9C376.9090906@redhat.com> Message-ID: <4B044C9D.7020803@redhat.com> Pavel Zuna wrote: > Rob Crittenden wrote: >> The user plugin is crapping out on line 317 of ldap2.py because attr >> is coming back None. The attribute it is looking for is member. >> >> I think the fix involves setting member_attributes = ['member'] to the >> user plugin. >> >> I wonder if we need to make the ldap2 plugin a bit more robust too so >> it can handle it better if the schema lookup returns None. >> >> rob > This should fix the issue. > Yes, this will fix it (I did a similar fix to work around it) but what does it mean if there is no attribute found? Is that possible? Should we catch it and return a more specific error message instead? rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Wed Nov 18 19:37:30 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 18 Nov 2009 14:37:30 -0500 Subject: [Freeipa-devel] [PATCH] Remove 'ipaObject' objectClass from rolegroups and taskgroups. In-Reply-To: <4AE9D115.9000006@redhat.com> References: <4AE9D115.9000006@redhat.com> Message-ID: <4B044CFA.50002@redhat.com> Pavel Zuna wrote: > We don't need it there and rolegroups/taskgroup created on installation > don't have it, so they didn't show up in *-find output before this patch. > > Pavel ack, pushed to master -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Wed Nov 18 19:41:10 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 18 Nov 2009 14:41:10 -0500 Subject: [Freeipa-devel] [PATCH] Handle ipaEnabledFlag as bool (TRUE/FALSE) instead of string (enabled/disabled). In-Reply-To: <4AEECA88.7050207@redhat.com> References: <4AEECA88.7050207@redhat.com> Message-ID: <4B044DD6.6010009@redhat.com> Pavel Zuna wrote: > Syntax in schema is 1.3.6.1.4.1.1466.115.121.1.7 == Boolean syntax. > > Thanks to Sumit for pointing this out. > > Pavel ack, pushed to master -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Wed Nov 18 19:52:59 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 18 Nov 2009 14:52:59 -0500 Subject: [Freeipa-devel] [PATCH] Add BIND pre-op for DS->IPA password migration to ipa-pwd-extop DS plugin. In-Reply-To: <4AF81C9E.8040305@redhat.com> References: <4AF81C9E.8040305@redhat.com> Message-ID: <4B04509B.8020402@redhat.com> Pavel Zuna wrote: > Re-post from ipa-and-samba-team-list. > > This patch makes DS generate new Kerberos keys for eligible users upon a > successful simple bind. We need this for password migration. > > Pavel Some minor points: - The comment block of ipa_pwd_pre_bind() doesn't match the code: it doesn't add krbPrincipalAux or set the principal name. - It probably shouldn't use log level SLAPI_LOG_FATAL rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Wed Nov 18 21:51:48 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 18 Nov 2009 16:51:48 -0500 Subject: [Freeipa-devel] [PATCH] 316 add tips to --help Message-ID: <4B046C74.9050705@redhat.com> Add some tips for getting more help from ipa command. % ipa --help Usage: ipa [options] Options: -h, --help show this help message and exit -e KEY=VAL Set environment variable KEY to VAL -c FILE Load configuration from FILE -d, --debug Produce full debuging output -v, --verbose Produce more verbose output -a, --prompt-all Prompt for ALL values (even if optional) -n, --no-prompt Prompt for NO values (even if required) Available help topics: ipa help topics Available commands: ipa help commands -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-316-help.patch Type: application/mbox Size: 1007 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From jderose at redhat.com Thu Nov 19 09:45:49 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Thu, 19 Nov 2009 02:45:49 -0700 Subject: [Freeipa-devel] Return values, CRUD, webUI In-Reply-To: <4B04019D.2060405@redhat.com> References: <1258424062.6624.759.camel@jgd-dsk> <4B04019D.2060405@redhat.com> Message-ID: <1258623949.19095.7.camel@jgd-dsk> On Wed, 2009-11-18 at 15:15 +0100, Pavel Zuna wrote: > Jason Gerard DeRose wrote: > > The vast majority of our Command plugins subclass from one of the CRUD > > base classes, so in terms of return value consistency and API style, we > > need to focus most on them (and then adapt their style to the few > > non-CRUD commands). > > > > While hooking up the webUI there have been many, many small problems in > > the core library and plugins that have caused unexpected setbacks for > > me. Some features that I needed got changed without me noticing, some > > of my half-baked designs needed more baking, some features were missing, > > and some new code I was just unfamiliar with. Point is, I've spent a > > lot of time battling little gotchas and thinking about how best to clean > > these things up. Here are the guidelines I propose we follow: > > > > > > A return value dict > > =================== > > > > As much as possible, I want to keep our return values very simple and > > regular. This 1) makes our API easy to learn and use, and 2) makes it > > easy to use the return values to drive UI logic on both the CLI and > > webUI. > > > > One current source of irregularity is the need to pass the "this isn't > > all the entries" flag from LDAP when we do searches. For example, > > `user_find` returns an (entry_list, more_remains) tuple. The problem is > > that most of the code paths don't care about the `more_remains` flag... > > they just need to know whether a list of entries was returned (result is > > a list) or whether a single entry was returned (result is a dict). > > > > At the same time, we obviously need a way to pass extra data like the > > `more_remains` flag and it would be nice to be able to extend a return > > value with additional special data without breaking code or causing an > > explosion of special cases. So I propose that our return values always > > be a dict containing at least a 'result', leaving us the option to > > extend the return value without breaking code that just looks at > > ret['result']. > > > > So in the case of a search, instead of: > > > > ([{'uid': 'foo'}, {'uid': 'bar'}, ...], True) > > > > We should return: > > > > { > > 'result': [{'uid': 'foo'}, {'uid': 'bar'}, ...], > > 'more_remains': True, > > ... > > 'extend': 'as-needed', > > } > > > > The following all assume we are returning {'result': blah} even though > > they don't show it... > > > > > > Entries > > ======= > > > > 95% of our return values are LDAP entries. Currently we're returning > > pretty much the raw value from python-ldap (although we are decoding > > UTF-8 into `unicode` objects for use in the Python pipeline and encoding > > back to UTF-8 on the way out, which is good). But the data structure > > returned from python-ldap is pretty awkward to work with. > > > > First, at the top it's typically a (dn, entry) tuple. Assuming the 'dn' > > key doesn't conflict with any sane LDAP attribute names, I think we > > should return a single dict with the dn stored under the 'dn' key. > > > > So instead of: > > > > ('uid=jdoe,cn=users,cn=accounts,dc=example,dc=com', {'sn': ['Doe']}) > > > > We should return: > > > > {'dn': 'uid=jdoe,cn=users,cn=accounts,dc=example,dc=com', 'sn': ['Doe']} > > > > Second, currently we return all attribute values inside a list whether > > or not they're multi-value. This leads to lots of special cases > > throughout the code that would be better dealt with in a single place, > > in LDAP Backend adapter, IHMO. > > > > So instead of: > > > > {'uid': ['jdoe'], 'group': ['foo', 'bar']} > > > > We should return: > > > > {'uid': 'jdoe', 'group': ['foo', 'bar']} > This is the only part I don't really agree with. See this thread for my reasons: > > [Freeipa-devel] [PATCH] Add --all to LDAPCreate and make LDAP commands always > display default attributes. > > But since both you and Rob seem to agree, I won't stand in the way and just make > the change. :) Well, you've made some good points, and this is something I don't feel super strongly about either way (although I lean toward not putting scalar values in a list). Rob, has this swayed you either way? > > > > Lists of Entries > > ================ > > > > When a command returns multiple entries, the entries should be in the > > same form as they are from commands that return only one entry. For > > example, currently user-find returns each entry as a (uid, entry) tuple. > > I think this should again be replaced with a single dict without the uid > > being duplicated. > > > > > > Create > > ====== > > > > If successful, we should return the resulting entry in standard form. > > If any error occurs, we should raise an appropriate exception. > > > > > > Retrieve > > ======== > > > > If successful, we should return the entry in standard form. If no such > > entry exists we should raise a NotFound exception. If any other error > > occurs, we should raise an appropriate exception. > > > > > > Update > > ====== > > > > (Same as Create.) > > > > > > Delete > > ====== > > > > (Same as Retrieve.) > > > After a delete, there is no entry to return. Unless an exception was raised, I > think we should just report 'delete successful' to the user and be done with it. > > > > > Search > > ====== > > > > If one or more entries matches the search criteria, we should return a > > list of entries, where the each entry is in standard form. If no > > entries match, we should return an empty list. If an error occurs, we > > should raise an appropriate exception. > > > > > > Thoughts? > > > > -Jason > > > > It's going to take a while to implement (and test!) all these changes, but I > should have it done by next week. > > Pavel We need to coordinate closely on these changes so we don't duplicate. I have some proof of concept changes I made just while exploring the idea, but these were just scratch really. From jderose at redhat.com Thu Nov 19 10:14:39 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Thu, 19 Nov 2009 03:14:39 -0700 Subject: [Freeipa-devel] [PATCH] 310 clean up ipa-join return values In-Reply-To: <4AFAE826.9030708@redhat.com> References: <4AFAE826.9030708@redhat.com> Message-ID: <1258625679.19095.8.camel@jgd-dsk> On Wed, 2009-11-11 at 11:36 -0500, Rob Crittenden wrote: > ipa-join calls ipa-getkeytab and returns whatever return value it does > so I want to be careful not to overlap the values and keep things unique > or meaning the same thing in both. > > This patch cleans up a few places. > > rob ack. pushed to master. From jderose at redhat.com Thu Nov 19 10:48:31 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Thu, 19 Nov 2009 03:48:31 -0700 Subject: [Freeipa-devel] [PATCH] 311 more integrated client install In-Reply-To: <4AFAE8D2.3050701@redhat.com> References: <4AFAE8D2.3050701@redhat.com> Message-ID: <1258627711.19095.9.camel@jgd-dsk> On Wed, 2009-11-11 at 11:39 -0500, Rob Crittenden wrote: > This patch integrates ipa-join and ipa-rmkeytab into the client > installer. This will join a machine to the IPA realm and fetch a host > principal for /etc/krb5.keytab. > > On uninstall all principals for the realm will be removed from > /etc/krb5.keytab. > > By default installation will fail if a host principal cannot be > obtained. Use the --force option to continue anyway. > > rob nack. this is breaking the installer: [6/12]: creating a keytab for httpd [7/12]: Setting up ssl [8/12]: Setting up browser autoconfig [9/12]: publish CA cert [10/12]: configuring SELinux for httpd [11/12]: restarting httpd [12/12]: configuring httpd to start on boot done configuring httpd. Applying LDAP updates restarting the directory server restarting the KDC Sample zone file for bind has been created in /tmp/sample.zone.WUedsi.db Configuration of client side components failed! ipa-client-install returned: Command '/usr/sbin/ipa-client-install --on-master --unattended --domain example.com --server fedora11.example.com --realm EXAMPLE.COM' returned non-zero exit status 1 From pzuna at redhat.com Thu Nov 19 14:38:02 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Thu, 19 Nov 2009 15:38:02 +0100 Subject: [Freeipa-devel] [PATCH] Make ldap2.convert_attr_synonyms more robust against schema lookup fails. In-Reply-To: <4B044C9D.7020803@redhat.com> References: <4AE9C376.9090906@redhat.com> <4B044C9D.7020803@redhat.com> Message-ID: <4B05584A.50603@redhat.com> Rob Crittenden wrote: > Pavel Zuna wrote: >> Rob Crittenden wrote: >>> The user plugin is crapping out on line 317 of ldap2.py because attr >>> is coming back None. The attribute it is looking for is member. >>> >>> I think the fix involves setting member_attributes = ['member'] to >>> the user plugin. >>> >>> I wonder if we need to make the ldap2 plugin a bit more robust too so >>> it can handle it better if the schema lookup returns None. >>> >>> rob >> This should fix the issue. >> > > Yes, this will fix it (I did a similar fix to work around it) but what > does it mean if there is no attribute found? Is that possible? > > Should we catch it and return a more specific error message instead? > > rob If it doesn't find the attribute, PROBABLY nothing will happen... Fortunately, we don't have to worry about it anymore. I played with python-ldap a bit today and it seems to have the convert_attr_synonyms functionality built-in. :) Here's a replacement patch. Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Remove-ldap2.convert_attr_synonyms.-Turns-out-python.patch Type: application/mbox Size: 2779 bytes Desc: not available URL: From pzuna at redhat.com Thu Nov 19 14:41:19 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Thu, 19 Nov 2009 15:41:19 +0100 Subject: [Freeipa-devel] [PATCH] 314 handle LDAP bind failures gracefully In-Reply-To: <4AFC8333.1050402@redhat.com> References: <4AFC8333.1050402@redhat.com> Message-ID: <4B05590F.7050105@redhat.com> Rob Crittenden wrote: > Gracefully handle a valid kerberos ticket for a deleted entry. > > I saw this with a host where I joined a host, obtained a host principal, > kinited to that principal, then deleted the host from the IPA server. > The ticket was still valid so Apache let it through but > it failed to bind to LDAP. > > rob > ack. Pavel From pzuna at redhat.com Thu Nov 19 14:55:15 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Thu, 19 Nov 2009 15:55:15 +0100 Subject: [Freeipa-devel] [PATCH] 316 add tips to --help In-Reply-To: <4B046C74.9050705@redhat.com> References: <4B046C74.9050705@redhat.com> Message-ID: <4B055C53.3070407@redhat.com> Rob Crittenden wrote: > Add some tips for getting more help from ipa command. > > % ipa --help > Usage: ipa [options] > > Options: > -h, --help show this help message and exit > -e KEY=VAL Set environment variable KEY to VAL > -c FILE Load configuration from FILE > -d, --debug Produce full debuging output > -v, --verbose Produce more verbose output > -a, --prompt-all Prompt for ALL values (even if optional) > -n, --no-prompt Prompt for NO values (even if required) > > Available help topics: > ipa help topics > > Available commands: > ipa help commands > ack. Pavel From pzuna at redhat.com Thu Nov 19 14:57:24 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Thu, 19 Nov 2009 15:57:24 +0100 Subject: [Freeipa-devel] [PATCH] Print only one line of docstrings in command listings. Message-ID: <4B055CD4.30804@redhat.com> Full docstring is shown on `ipa help COMMAND` Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Print-only-one-line-of-docstrings-in-command-listing.patch Type: application/mbox Size: 1288 bytes Desc: not available URL: From jdennis at redhat.com Thu Nov 19 15:33:50 2009 From: jdennis at redhat.com (John Dennis) Date: Thu, 19 Nov 2009 10:33:50 -0500 Subject: [Freeipa-devel] [PATCH] respect debug arg during server install Message-ID: <20091119153211.13791.20970.stgit@jdennis> The debug flag (e.g. -d) was not being respected during server install. This patch corrects that. --- install/tools/ipa-server-install | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) -------------- next part -------------- diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install index 809d69a..44fc5fd 100755 --- a/install/tools/ipa-server-install +++ b/install/tools/ipa-server-install @@ -484,6 +484,7 @@ def main(): cfg = dict( in_server=True, webui_assets_dir=ASSETS_DIR, + debug=options.debug ) if not options.uninstall: if options.ca: From jdennis at redhat.com Thu Nov 19 16:09:51 2009 From: jdennis at redhat.com (John Dennis) Date: Thu, 19 Nov 2009 11:09:51 -0500 Subject: [Freeipa-devel] [PATCH] error strings in documentation were missing unicode specifier Message-ID: <20091119160927.14253.39023.stgit@jdennis> error strings in documentation were missing unicode specifier --- ipalib/errors.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) -------------- next part -------------- diff --git a/ipalib/errors.py b/ipalib/errors.py index 1c358cd..ca1851c 100644 --- a/ipalib/errors.py +++ b/ipalib/errors.py @@ -390,7 +390,7 @@ class NetworkError(PublicError): For example: - >>> raise NetworkError(uri='ldap://localhost:389', error='Connection refused') + >>> raise NetworkError(uri='ldap://localhost:389', error=u'Connection refused') Traceback (most recent call last): ... NetworkError: cannot connect to 'ldap://localhost:389': Connection refused @@ -694,7 +694,7 @@ class ConversionError(InvocationError): For example: - >>> raise ConversionError(name='age', error='must be an integer') + >>> raise ConversionError(name='age', error=u'must be an integer') Traceback (most recent call last): ... ConversionError: invalid 'age': must be an integer @@ -710,7 +710,7 @@ class ValidationError(InvocationError): For example: - >>> raise ValidationError(name='sn', error='can be at most 128 characters') + >>> raise ValidationError(name='sn', error=u'can be at most 128 characters') Traceback (most recent call last): ... ValidationError: invalid 'sn': can be at most 128 characters From jdennis at redhat.com Thu Nov 19 16:16:30 2009 From: jdennis at redhat.com (John Dennis) Date: Thu, 19 Nov 2009 11:16:30 -0500 Subject: [Freeipa-devel] [PATCH] add new error class for certificate operations Message-ID: <20091119161555.14470.14746.stgit@jdennis> add new error class for certificate operations --- ipalib/errors.py | 29 ++++++++++++++++++++++++++++- 1 files changed, 28 insertions(+), 1 deletions(-) -------------- next part -------------- diff --git a/ipalib/errors.py b/ipalib/errors.py index 1c358cd..f23a636 100644 --- a/ipalib/errors.py +++ b/ipalib/errors.py @@ -89,7 +89,9 @@ current block assignments: - **4200 - 4299** `LDAPError` and its subclasses - - **4300 - 4999** *Reserved for future use* + - **4300 - 4399** `CertificateError` and its subclasses + + - **4400 - 4999** *Reserved for future use* - **5000 - 5999** `GenericError` and its subclasses @@ -1155,6 +1157,31 @@ class ObjectclassViolation(ExecutionError): format = _('%(info)s') +class CertificateError(ExecutionError): + """ + **4300** Base class for Certificate execution errors (*4300 - 4399*). + """ + + errno = 4300 + + +class CertificateOperationError(ExecutionError): + """ + **4301** Raised when a certificate operation cannot be completed + + For example: + + >>> raise CertificateOperationError(error=u'bad serial number') + Traceback (most recent call last): + ... + CertificateOperationError: Certificate operation cannot be completed: bad serial number + + """ + + errno = 4301 + format = _('Certificate operation cannot be completed: %(error)s') + + ############################################################################## # 5000 - 5999: Generic errors From jdennis at redhat.com Thu Nov 19 16:39:49 2009 From: jdennis at redhat.com (John Dennis) Date: Thu, 19 Nov 2009 11:39:49 -0500 Subject: [Freeipa-devel] [PATCH] Reading Int parameter class should respect radix prefix Message-ID: <20091119163846.14864.83858.stgit@jdennis> Signed-off-by: John Dennis Reading INT parameter class should respect radix prefix The Int parameter class was not respecting any radix prefix (e.g. 0x) the user may have supplied. This patch implements _convert_scalar method for the Int class so that we can pass the special radix base of zero to the int constructor telling it to determine the radix from the prefix (if present). --- ipalib/parameters.py | 22 ++++++++++++++++++++++ 1 files changed, 22 insertions(+), 0 deletions(-) -------------- next part -------------- diff --git a/ipalib/parameters.py b/ipalib/parameters.py index 227757d..00862f0 100644 --- a/ipalib/parameters.py +++ b/ipalib/parameters.py @@ -909,6 +909,28 @@ class Int(Number): self.nice, self.minvalue, self.maxvalue) ) + def _convert_scalar(self, value, index=None): + """ + Convert a single scalar value. + """ + if type(value) is int: + return value + if type(value) is unicode: + try: + # 2nd arg is radix base, 2nd arg only accepted for strings. + # Zero means determine radix base from prefix (e.g. 0x for hex) + return int(value, 0) + except ValueError: + pass + if type(value) is float: + try: + return int(value) + except ValueError: + pass + raise ConversionError(name=self.name, index=index, + error=ugettext(self.type_error), + ) + def _rule_minvalue(self, _, value): """ Check min constraint. From rcritten at redhat.com Thu Nov 19 16:40:06 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Nov 2009 11:40:06 -0500 Subject: [Freeipa-devel] [PATCH] 317 fix location of CA Message-ID: <4B0574E6.60006@redhat.com> The output of ipa-server-install pointed to the old location of the self-signed database, the 389-DS instance. It is now stored in the Apache NSS database. Also set a db password on the 389-DS NSS database. It was using a blank password. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-317-ca.patch Type: application/mbox Size: 2178 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From jdennis at redhat.com Thu Nov 19 16:56:14 2009 From: jdennis at redhat.com (John Dennis) Date: Thu, 19 Nov 2009 11:56:14 -0500 Subject: [Freeipa-devel] [PATCH] If plugin fails to load log the traceback Message-ID: <20091119165535.15148.5173.stgit@jdennis> Signed-off-by: John Dennis If plugin fails to load log the traceback If a plugin fails to load due to some kind of error it would be nice if the error log contained the traceback so you can examine what went wrong rather than being left blind as to why it failed to load. --- ipalib/plugable.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) -------------- next part -------------- diff --git a/ipalib/plugable.py b/ipalib/plugable.py index 12746c1..d44d688 100644 --- a/ipalib/plugable.py +++ b/ipalib/plugable.py @@ -575,7 +575,8 @@ class API(DictProxy): 'skipping plugin module %s: %s', fullname, e.reason ) except StandardError, e: - self.log.error('could not load plugin module %r', pyfile) + import traceback + self.log.error('could not load plugin module %r\n%s', pyfile, traceback.format_exc()) raise e def finalize(self): From jdennis at redhat.com Thu Nov 19 17:14:30 2009 From: jdennis at redhat.com (John Dennis) Date: Thu, 19 Nov 2009 12:14:30 -0500 Subject: [Freeipa-devel] [PATCH] along with stdout, stderr also log the initial command Message-ID: <20091119171350.15432.68898.stgit@jdennis> Signed-off-by: John Dennis along with stdout,stderr also log the initial command This implements better logging of external commands. Formerly we were just outputting stdout and stderr without labeling which was which. We also omitted the initial command and it's arguments. This made it difficult when reviewing the logs to know what the command was and what was stdout vs. stderr. This patch fixes that. --- ipapython/ipautil.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) -------------- next part -------------- diff --git a/ipapython/ipautil.py b/ipapython/ipautil.py index 47b651c..1066879 100644 --- a/ipapython/ipautil.py +++ b/ipapython/ipautil.py @@ -97,8 +97,9 @@ def run(args, stdin=None): p = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True) stdout,stderr = p.communicate() - logging.info(stdout) - logging.info(stderr) + logging.info('args=%s' % ' '.join(args)) + logging.info('stdout=%s' % stdout) + logging.info('stderr=%s' % stderr) if p.returncode != 0: raise CalledProcessError(p.returncode, ' '.join(args)) From jdennis at redhat.com Thu Nov 19 18:24:27 2009 From: jdennis at redhat.com (John Dennis) Date: Thu, 19 Nov 2009 13:24:27 -0500 Subject: [Freeipa-devel] [PATCH] Make NotImplementedError in rabase return the correct function name Message-ID: <20091119182348.16247.12608.stgit@jdennis> Signed-off-by: John Dennis Make NotImplementedError in rabase return the correct function name The NotImplementedError's in rabase were returning the wrong function name, looks like a simple cut-n-paste bug. --- ipaserver/plugins/rabase.py | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) -------------- next part -------------- diff --git a/ipaserver/plugins/rabase.py b/ipaserver/plugins/rabase.py index 3fc0ec5..c6aabf7 100644 --- a/ipaserver/plugins/rabase.py +++ b/ipaserver/plugins/rabase.py @@ -66,7 +66,7 @@ class rabase(Backend): :param serial_number: certificate serial number """ - raise errors.NotImplementedError(name='%s.check_request_status' % self.name) + raise errors.NotImplementedError(name='%s.get_certificate' % self.name) def request_certificate(self, csr, request_type='pkcs10'): """ @@ -75,7 +75,7 @@ class rabase(Backend): :param csr: The certificate signing request. :param request_type: The request type (defaults to ``'pkcs10'``). """ - raise errors.NotImplementedError(name='%s.check_request_status' % self.name) + raise errors.NotImplementedError(name='%s.request_certificate' % self.name) def revoke_certificate(self, serial_number, revocation_reason=0): """ @@ -101,7 +101,7 @@ class rabase(Backend): :param serial_number: Certificate serial number. :param revocation_reason: Integer code of revocation reason. """ - raise errors.NotImplementedError(name='%s.check_request_status' % self.name) + raise errors.NotImplementedError(name='%s.revoke_certificate' % self.name) def take_certificate_off_hold(self, serial_number): """ @@ -109,5 +109,5 @@ class rabase(Backend): :param serial_number: Certificate serial number. """ - raise errors.NotImplementedError('%s.check_request_status' % self.name) + raise errors.NotImplementedError('%s.take_certificate_off_hold' % self.name) From rcritten at redhat.com Thu Nov 19 19:15:46 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Nov 2009 14:15:46 -0500 Subject: [Freeipa-devel] [PATCH] 311 more integrated client install In-Reply-To: <1258627711.19095.9.camel@jgd-dsk> References: <4AFAE8D2.3050701@redhat.com> <1258627711.19095.9.camel@jgd-dsk> Message-ID: <4B059962.1000904@redhat.com> Jason Gerard DeRose wrote: > On Wed, 2009-11-11 at 11:39 -0500, Rob Crittenden wrote: >> This patch integrates ipa-join and ipa-rmkeytab into the client >> installer. This will join a machine to the IPA realm and fetch a host >> principal for /etc/krb5.keytab. >> >> On uninstall all principals for the realm will be removed from >> /etc/krb5.keytab. >> >> By default installation will fail if a host principal cannot be >> obtained. Use the --force option to continue anyway. >> >> rob > > nack. this is breaking the installer: > > [6/12]: creating a keytab for httpd > [7/12]: Setting up ssl > [8/12]: Setting up browser autoconfig > [9/12]: publish CA cert > [10/12]: configuring SELinux for httpd > [11/12]: restarting httpd > [12/12]: configuring httpd to start on boot > done configuring httpd. > Applying LDAP updates > restarting the directory server > restarting the KDC > Sample zone file for bind has been created in /tmp/sample.zone.WUedsi.db > Configuration of client side components failed! > ipa-client-install returned: Command '/usr/sbin/ipa-client-install > --on-master --unattended --domain example.com --server > fedora11.example.com --realm EXAMPLE.COM' returned non-zero exit status > 1 It shouldn't require a password/principal when on the master. New patch attached. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-311.2-install.patch Type: application/mbox Size: 19435 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Thu Nov 19 19:37:04 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Nov 2009 14:37:04 -0500 Subject: [Freeipa-devel] [PATCH] Make ldap2.convert_attr_synonyms more robust against schema lookup fails. In-Reply-To: <4B05584A.50603@redhat.com> References: <4AE9C376.9090906@redhat.com> <4B044C9D.7020803@redhat.com> <4B05584A.50603@redhat.com> Message-ID: <4B059E60.1030606@redhat.com> Pavel Zuna wrote: > Rob Crittenden wrote: >> Pavel Zuna wrote: >>> Rob Crittenden wrote: >>>> The user plugin is crapping out on line 317 of ldap2.py because attr >>>> is coming back None. The attribute it is looking for is member. >>>> >>>> I think the fix involves setting member_attributes = ['member'] to >>>> the user plugin. >>>> >>>> I wonder if we need to make the ldap2 plugin a bit more robust too >>>> so it can handle it better if the schema lookup returns None. >>>> >>>> rob >>> This should fix the issue. >>> >> >> Yes, this will fix it (I did a similar fix to work around it) but what >> does it mean if there is no attribute found? Is that possible? >> >> Should we catch it and return a more specific error message instead? >> >> rob > > If it doesn't find the attribute, PROBABLY nothing will happen... > > Fortunately, we don't have to worry about it anymore. I played with > python-ldap a bit today and it seems to have the convert_attr_synonyms > functionality built-in. :) > > Here's a replacement patch. > > Pavel nack. I don't see where python-ldap is replacing it. We weren't seeing it done before were we? Also, we need to request the 'aci' attribute for the aci plugin to work. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Thu Nov 19 19:37:52 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Nov 2009 14:37:52 -0500 Subject: [Freeipa-devel] [PATCH] 314 handle LDAP bind failures gracefully In-Reply-To: <4B05590F.7050105@redhat.com> References: <4AFC8333.1050402@redhat.com> <4B05590F.7050105@redhat.com> Message-ID: <4B059E90.9010403@redhat.com> Pavel Zuna wrote: > Rob Crittenden wrote: >> Gracefully handle a valid kerberos ticket for a deleted entry. >> >> I saw this with a host where I joined a host, obtained a host >> principal, kinited to that principal, then deleted the host from the >> IPA server. The ticket was still valid so Apache let it through but >> it failed to bind to LDAP. >> >> rob >> > ack. > > Pavel pushed to master -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Thu Nov 19 19:38:12 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Nov 2009 14:38:12 -0500 Subject: [Freeipa-devel] [PATCH] 316 add tips to --help In-Reply-To: <4B055C53.3070407@redhat.com> References: <4B046C74.9050705@redhat.com> <4B055C53.3070407@redhat.com> Message-ID: <4B059EA4.4070902@redhat.com> Pavel Zuna wrote: > Rob Crittenden wrote: >> Add some tips for getting more help from ipa command. >> >> % ipa --help >> Usage: ipa [options] >> >> Options: >> -h, --help show this help message and exit >> -e KEY=VAL Set environment variable KEY to VAL >> -c FILE Load configuration from FILE >> -d, --debug Produce full debuging output >> -v, --verbose Produce more verbose output >> -a, --prompt-all Prompt for ALL values (even if optional) >> -n, --no-prompt Prompt for NO values (even if required) >> >> Available help topics: >> ipa help topics >> >> Available commands: >> ipa help commands >> > ack. > > Pavel pushed to master -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Thu Nov 19 19:52:42 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Nov 2009 14:52:42 -0500 Subject: [Freeipa-devel] [PATCH] respect debug arg during server install In-Reply-To: <20091119153211.13791.20970.stgit@jdennis> References: <20091119153211.13791.20970.stgit@jdennis> Message-ID: <4B05A20A.1050207@redhat.com> John Dennis wrote: > The debug flag (e.g. -d) was not being respected during server install. This > patch corrects that. > > > > --- > > install/tools/ipa-server-install | 1 + > 1 files changed, 1 insertions(+), 0 deletions(-) > > > ------------------------------------------------------------------------ > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel ack, pushed to master -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Thu Nov 19 19:52:51 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Nov 2009 14:52:51 -0500 Subject: [Freeipa-devel] [PATCH] error strings in documentation were missing unicode specifier In-Reply-To: <20091119160927.14253.39023.stgit@jdennis> References: <20091119160927.14253.39023.stgit@jdennis> Message-ID: <4B05A213.3070208@redhat.com> John Dennis wrote: > error strings in documentation were missing unicode specifier > > --- > > ipalib/errors.py | 6 +++--- > 1 files changed, 3 insertions(+), 3 deletions(-) > > > ------------------------------------------------------------------------ > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel ack, pushed to master -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Thu Nov 19 19:53:00 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Nov 2009 14:53:00 -0500 Subject: [Freeipa-devel] [PATCH] add new error class for certificate operations In-Reply-To: <20091119161555.14470.14746.stgit@jdennis> References: <20091119161555.14470.14746.stgit@jdennis> Message-ID: <4B05A21C.3080900@redhat.com> John Dennis wrote: > add new error class for certificate operations > > --- > > ipalib/errors.py | 29 ++++++++++++++++++++++++++++- > 1 files changed, 28 insertions(+), 1 deletions(-) > > > ------------------------------------------------------------------------ > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel ack, pushed to master -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Thu Nov 19 19:59:57 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Nov 2009 14:59:57 -0500 Subject: [Freeipa-devel] [PATCH] Reading Int parameter class should respect radix prefix In-Reply-To: <20091119163846.14864.83858.stgit@jdennis> References: <20091119163846.14864.83858.stgit@jdennis> Message-ID: <4B05A3BD.2020904@redhat.com> John Dennis wrote: > Signed-off-by: John Dennis > > Reading INT parameter class should respect radix prefix > > The Int parameter class was not respecting any radix prefix (e.g. 0x) the user > may have supplied. This patch implements _convert_scalar method for the Int > class so that we can pass the special radix base of zero to the int constructor > telling it to determine the radix from the prefix (if present). > > --- > > ipalib/parameters.py | 22 ++++++++++++++++++++++ > 1 files changed, 22 insertions(+), 0 deletions(-) The patch looks good. Can you provide some unit tests for it as well? rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From rcritten at redhat.com Thu Nov 19 21:19:46 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Nov 2009 16:19:46 -0500 Subject: [Freeipa-devel] [PATCH] Make NotImplementedError in rabase return the correct function name In-Reply-To: <20091119182348.16247.12608.stgit@jdennis> References: <20091119182348.16247.12608.stgit@jdennis> Message-ID: <4B05B672.8090405@redhat.com> John Dennis wrote: > Signed-off-by: John Dennis > > Make NotImplementedError in rabase return the correct function name > > The NotImplementedError's in rabase were returning the wrong function > name, looks like a simple cut-n-paste bug. > > --- > > ipaserver/plugins/rabase.py | 8 ++++---- > 1 files changed, 4 insertions(+), 4 deletions(-) > > > ------------------------------------------------------------------------ > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel ack, pushed to master -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From jdennis at redhat.com Thu Nov 19 23:37:35 2009 From: jdennis at redhat.com (John Dennis) Date: Thu, 19 Nov 2009 18:37:35 -0500 Subject: [Freeipa-devel] [PATCH] Require current versions of python-nss & python-lxml Message-ID: <20091119233505.20374.89637.stgit@jdennis> This adds a new dependency on python-lxml. It also requires that python-nss be the current version (0.8). This is needed for the http authentication work I did for Jason which was added in a recent version of python-nss. This patch also fixes a deprecation warning concerning nssinit() which is dependent on a newer version of python-nss. To the best of my knowledge both python-lxml and python-nss.0-8 are currently available in the RHEL5 IPA compose, they are both definately available in fedora F-11, F-12, rawhide. --- From: John Dennis Subject: [PATCH] Require current versions of python-nss & python-lxml ipa.spec.in | 3 ++- ipapython/nsslib.py | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/ipa.spec.in b/ipa.spec.in index ce2c0a6..659a76a 100644 --- a/ipa.spec.in +++ b/ipa.spec.in @@ -173,7 +173,8 @@ Requires: python-kerberos >= 1.1-3 Requires: authconfig Requires: gnupg Requires: pyOpenSSL -Requires: python-nss +Requires: python-nss >= 0.8 +Requires: python-lxml %description python IPA is an integrated solution to provide centrally managed Identity (machine, diff --git a/ipapython/nsslib.py b/ipapython/nsslib.py index 05b45c1..2052843 100644 --- a/ipapython/nsslib.py +++ b/ipapython/nsslib.py @@ -95,7 +95,7 @@ class NSSConnection(httplib.HTTPConnection): if not dbdir: raise RuntimeError("dbdir is required") - ssl.nssinit(dbdir) + nss.nss_init(dbdir) ssl.set_domestic_policy() nss.set_password_callback(self.password_callback) From pzuna at redhat.com Fri Nov 20 12:35:33 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Fri, 20 Nov 2009 13:35:33 +0100 Subject: [Freeipa-devel] [PATCH] Make ldap2.convert_attr_synonyms more robust against schema lookup fails. In-Reply-To: <4B059E60.1030606@redhat.com> References: <4AE9C376.9090906@redhat.com> <4B044C9D.7020803@redhat.com> <4B05584A.50603@redhat.com> <4B059E60.1030606@redhat.com> Message-ID: <4B068D15.7030904@redhat.com> Rob Crittenden wrote: > Pavel Zuna wrote: >> Rob Crittenden wrote: >>> Pavel Zuna wrote: >>>> Rob Crittenden wrote: >>>>> The user plugin is crapping out on line 317 of ldap2.py because >>>>> attr is coming back None. The attribute it is looking for is member. >>>>> >>>>> I think the fix involves setting member_attributes = ['member'] to >>>>> the user plugin. >>>>> >>>>> I wonder if we need to make the ldap2 plugin a bit more robust too >>>>> so it can handle it better if the schema lookup returns None. >>>>> >>>>> rob >>>> This should fix the issue. >>>> >>> >>> Yes, this will fix it (I did a similar fix to work around it) but >>> what does it mean if there is no attribute found? Is that possible? >>> >>> Should we catch it and return a more specific error message instead? >>> >>> rob >> >> If it doesn't find the attribute, PROBABLY nothing will happen... >> >> Fortunately, we don't have to worry about it anymore. I played with >> python-ldap a bit today and it seems to have the convert_attr_synonyms >> functionality built-in. :) >> >> Here's a replacement patch. >> >> Pavel > > nack. I don't see where python-ldap is replacing it. We weren't seeing > it done before were we? That's because we were doing it wrong. We were requesting all attributes ('*') + ACIs ('aci'). After this patch we explicitly request all attributes in the new entry (i.e. all attributes that are going to be updated) and python-ldap will always return them named as they were requested. In other words: If we request localityName as l, python-ldap will return it as l, if we request it as localityName, python-ldap will return it as localityName. > Also, we need to request the 'aci' attribute for the aci plugin to work. And we do so, because after this patch, we're requesting all attributes explicitly. > rob Pavel From rcritten at redhat.com Fri Nov 20 14:32:18 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 20 Nov 2009 09:32:18 -0500 Subject: [Freeipa-devel] [PATCH] Make ldap2.convert_attr_synonyms more robust against schema lookup fails. In-Reply-To: <4B068D15.7030904@redhat.com> References: <4AE9C376.9090906@redhat.com> <4B044C9D.7020803@redhat.com> <4B05584A.50603@redhat.com> <4B059E60.1030606@redhat.com> <4B068D15.7030904@redhat.com> Message-ID: <4B06A872.8080608@redhat.com> Pavel Zuna wrote: > Rob Crittenden wrote: >> Pavel Zuna wrote: >>> Rob Crittenden wrote: >>>> Pavel Zuna wrote: >>>>> Rob Crittenden wrote: >>>>>> The user plugin is crapping out on line 317 of ldap2.py because >>>>>> attr is coming back None. The attribute it is looking for is member. >>>>>> >>>>>> I think the fix involves setting member_attributes = ['member'] to >>>>>> the user plugin. >>>>>> >>>>>> I wonder if we need to make the ldap2 plugin a bit more robust too >>>>>> so it can handle it better if the schema lookup returns None. >>>>>> >>>>>> rob >>>>> This should fix the issue. >>>>> >>>> >>>> Yes, this will fix it (I did a similar fix to work around it) but >>>> what does it mean if there is no attribute found? Is that possible? >>>> >>>> Should we catch it and return a more specific error message instead? >>>> >>>> rob >>> >>> If it doesn't find the attribute, PROBABLY nothing will happen... >>> >>> Fortunately, we don't have to worry about it anymore. I played with >>> python-ldap a bit today and it seems to have the >>> convert_attr_synonyms functionality built-in. :) >>> >>> Here's a replacement patch. >>> >>> Pavel >> >> nack. I don't see where python-ldap is replacing it. We weren't seeing >> it done before were we? > That's because we were doing it wrong. > > We were requesting all attributes ('*') + ACIs ('aci'). After this patch > we explicitly request all attributes in the new entry (i.e. all > attributes that are going to be updated) and python-ldap will always > return them named as they were requested. In other words: If we request > localityName as l, python-ldap will return it as l, if we request it as > localityName, python-ldap will return it as localityName. > >> Also, we need to request the 'aci' attribute for the aci plugin to work. > And we do so, because after this patch, we're requesting all attributes > explicitly. > Well, no, you're requesting all attributes in the current entry. The code looked like this once before and caused the aci plugin to break. I guess some other change fixed that, things are working as expected. ack rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From pzuna at redhat.com Fri Nov 20 15:08:21 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Fri, 20 Nov 2009 16:08:21 +0100 Subject: [Freeipa-devel] [PATCH] Make ldap2.convert_attr_synonyms more robust against schema lookup fails. In-Reply-To: <4B06A872.8080608@redhat.com> References: <4AE9C376.9090906@redhat.com> <4B044C9D.7020803@redhat.com> <4B05584A.50603@redhat.com> <4B059E60.1030606@redhat.com> <4B068D15.7030904@redhat.com> <4B06A872.8080608@redhat.com> Message-ID: <4B06B0E5.3090904@redhat.com> Rob Crittenden wrote: > Pavel Zuna wrote: >> Rob Crittenden wrote: >>> Pavel Zuna wrote: >>>> Rob Crittenden wrote: >>>>> Pavel Zuna wrote: >>>>>> Rob Crittenden wrote: >>>>>>> The user plugin is crapping out on line 317 of ldap2.py because >>>>>>> attr is coming back None. The attribute it is looking for is member. >>>>>>> >>>>>>> I think the fix involves setting member_attributes = ['member'] >>>>>>> to the user plugin. >>>>>>> >>>>>>> I wonder if we need to make the ldap2 plugin a bit more robust >>>>>>> too so it can handle it better if the schema lookup returns None. >>>>>>> >>>>>>> rob >>>>>> This should fix the issue. >>>>>> >>>>> >>>>> Yes, this will fix it (I did a similar fix to work around it) but >>>>> what does it mean if there is no attribute found? Is that possible? >>>>> >>>>> Should we catch it and return a more specific error message instead? >>>>> >>>>> rob >>>> >>>> If it doesn't find the attribute, PROBABLY nothing will happen... >>>> >>>> Fortunately, we don't have to worry about it anymore. I played with >>>> python-ldap a bit today and it seems to have the >>>> convert_attr_synonyms functionality built-in. :) >>>> >>>> Here's a replacement patch. >>>> >>>> Pavel >>> >>> nack. I don't see where python-ldap is replacing it. We weren't >>> seeing it done before were we? >> That's because we were doing it wrong. >> >> We were requesting all attributes ('*') + ACIs ('aci'). After this >> patch we explicitly request all attributes in the new entry (i.e. all >> attributes that are going to be updated) and python-ldap will always >> return them named as they were requested. In other words: If we >> request localityName as l, python-ldap will return it as l, if we >> request it as localityName, python-ldap will return it as localityName. >> >>> Also, we need to request the 'aci' attribute for the aci plugin to work. >> And we do so, because after this patch, we're requesting all >> attributes explicitly. >> > > Well, no, you're requesting all attributes in the current entry. The > code looked like this once before and caused the aci plugin to break. I > guess some other change fixed that, things are working as expected. If by current entry you mean the dict containing the changes passed to ldap2.update_entry then yes. When modifying ACIs, the 'aci' attribute is present and therefore requested. > ack > > rob Pavel From jdennis at redhat.com Fri Nov 20 18:41:44 2009 From: jdennis at redhat.com (John Dennis) Date: Fri, 20 Nov 2009 13:41:44 -0500 Subject: [Freeipa-devel] [PATCH] [Resend] Reading INT parameter class should respect radix prefix Message-ID: <20091120183928.31838.86725.stgit@jdennis> This modifies the original patch by including a unit test, handling floats when passed as unicode, and handling large magnitude values beyond maxint. The INT parameter class was not respecting any radix prefix (e.g. 0x) the user may have supplied. This patch implements _convert_scalar method for the Int class so that we can pass the special radix base of zero to the int constructor telling it to determine the radix from the prefix (if present). --- From: John Dennis Subject: [PATCH] Reading INT parameter class should respect radix prefix ipalib/parameters.py | 29 +++++++++++++++++++++++++++++ tests/test_ipalib/test_parameters.py | 27 +++++++++++++++++++++++++++ 2 files changed, 56 insertions(+), 0 deletions(-) diff --git a/ipalib/parameters.py b/ipalib/parameters.py index 227757d..8a1aede 100644 --- a/ipalib/parameters.py +++ b/ipalib/parameters.py @@ -909,6 +909,35 @@ class Int(Number): self.nice, self.minvalue, self.maxvalue) ) + def _convert_scalar(self, value, index=None): + """ + Convert a single scalar value. + """ + if type(value) in (int, long): + return value + if type(value) is unicode: + # permit floating point strings + if value.find(u'.') >= 0: + try: + return int(float(value)) + except ValueError: + pass + else: + try: + # 2nd arg is radix base, 2nd arg only accepted for strings. + # Zero means determine radix base from prefix (e.g. 0x for hex) + return int(value, 0) + except ValueError: + pass + if type(value) is float: + try: + return int(value) + except ValueError: + pass + raise ConversionError(name=self.name, index=index, + error=ugettext(self.type_error), + ) + def _rule_minvalue(self, _, value): """ Check min constraint. diff --git a/tests/test_ipalib/test_parameters.py b/tests/test_ipalib/test_parameters.py index f6bc66f..d1c5f7f 100644 --- a/tests/test_ipalib/test_parameters.py +++ b/tests/test_ipalib/test_parameters.py @@ -22,6 +22,7 @@ Test the `ipalib.parameters` module. """ import re +import sys from types import NoneType from inspect import isclass from tests.util import raises, ClassChecker, read_only @@ -1225,6 +1226,32 @@ class test_Int(ClassChecker): assert dummy.called() is True dummy.reset() + def test_convert_scalar(self): + """ + Test the `ipalib.parameters.Int._convert_scalar` method. + Assure radix prefixes work, str objects fail, + floats (native & string) are truncated, + large magnitude values are promoted to long, + empty strings & invalid numerical representations fail + """ + o = self.cls('my_number') + # Assure invalid inputs raise error + for bad in ['hello', u'hello', True, None, '10', u'', u'.']: + e = raises(errors.ConversionError, o._convert_scalar, bad) + assert e.name == 'my_number' + assert e.index is None + # Assure large magnatude values are handled correctly + assert type(o._convert_scalar(sys.maxint*2)) == long + assert o._convert_scalar(sys.maxint*2) == sys.maxint*2 + assert o._convert_scalar(unicode(sys.maxint*2)) == sys.maxint*2 + assert o._convert_scalar(long(16)) == 16 + # Assure normal conversions produce expected result + assert o._convert_scalar(u'16.99') == 16 + assert o._convert_scalar(16.99) == 16 + assert o._convert_scalar(u'16') == 16 + assert o._convert_scalar(u'0x10') == 16 + assert o._convert_scalar(u'020') == 16 + class test_Float(ClassChecker): """ Test the `ipalib.parameters.Float` class. From jdennis at redhat.com Fri Nov 20 19:28:45 2009 From: jdennis at redhat.com (John Dennis) Date: Fri, 20 Nov 2009 14:28:45 -0500 Subject: [Freeipa-devel] [PATCH] Fix raising nonexistent errros.NotImplemented Message-ID: <20091120192831.32511.85522.stgit@jdennis> The correct exception is the built-in NotImplemented exception --- From: John Dennis Subject: [PATCH] Fix raising nonexistent errros.NotImplemented 0 files changed, 0 insertions(+), 0 deletions(-) diff --git a/ipaserver/plugins/rabase.py b/ipaserver/plugins/rabase.py index 3fc0ec5..59f0ea8 100644 --- a/ipaserver/plugins/rabase.py +++ b/ipaserver/plugins/rabase.py @@ -58,7 +58,7 @@ class rabase(Backend): :param request_id: request ID """ - raise errors.NotImplementedError(name='%s.check_request_status' % self.name) + raise NotImplementedError('%s.check_request_status' % self.name) def get_certificate(self, serial_number=None): """ @@ -66,7 +66,7 @@ class rabase(Backend): :param serial_number: certificate serial number """ - raise errors.NotImplementedError(name='%s.check_request_status' % self.name) + raise NotImplementedError('%s.check_request_status' % self.name) def request_certificate(self, csr, request_type='pkcs10'): """ @@ -75,7 +75,7 @@ class rabase(Backend): :param csr: The certificate signing request. :param request_type: The request type (defaults to ``'pkcs10'``). """ - raise errors.NotImplementedError(name='%s.check_request_status' % self.name) + raise NotImplementedError('%s.check_request_status' % self.name) def revoke_certificate(self, serial_number, revocation_reason=0): """ @@ -101,7 +101,7 @@ class rabase(Backend): :param serial_number: Certificate serial number. :param revocation_reason: Integer code of revocation reason. """ - raise errors.NotImplementedError(name='%s.check_request_status' % self.name) + raise NotImplementedError('%s.check_request_status' % self.name) def take_certificate_off_hold(self, serial_number): """ @@ -109,5 +109,5 @@ class rabase(Backend): :param serial_number: Certificate serial number. """ - raise errors.NotImplementedError('%s.check_request_status' % self.name) + raise NotImplementedError('%s.check_request_status' % self.name) From rcritten at redhat.com Fri Nov 20 19:50:33 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 20 Nov 2009 14:50:33 -0500 Subject: [Freeipa-devel] [PATCH] Fix raising nonexistent errros.NotImplemented In-Reply-To: <20091120192831.32511.85522.stgit@jdennis> References: <20091120192831.32511.85522.stgit@jdennis> Message-ID: <4B06F309.6000906@redhat.com> John Dennis wrote: > The correct exception is the built-in NotImplemented exception > --- > From: John Dennis > Subject: [PATCH] Fix raising nonexistent errros.NotImplemented > > > 0 files changed, 0 insertions(+), 0 deletions(-) > > diff --git a/ipaserver/plugins/rabase.py b/ipaserver/plugins/rabase.py > index 3fc0ec5..59f0ea8 100644 > --- a/ipaserver/plugins/rabase.py > +++ b/ipaserver/plugins/rabase.py > @@ -58,7 +58,7 @@ class rabase(Backend): > > :param request_id: request ID > """ > - raise errors.NotImplementedError(name='%s.check_request_status' % self.name) > + raise NotImplementedError('%s.check_request_status' % self.name) > > def get_certificate(self, serial_number=None): > """ > @@ -66,7 +66,7 @@ class rabase(Backend): > > :param serial_number: certificate serial number > """ > - raise errors.NotImplementedError(name='%s.check_request_status' % self.name) > + raise NotImplementedError('%s.check_request_status' % self.name) > > def request_certificate(self, csr, request_type='pkcs10'): > """ > @@ -75,7 +75,7 @@ class rabase(Backend): > :param csr: The certificate signing request. > :param request_type: The request type (defaults to ``'pkcs10'``). > """ > - raise errors.NotImplementedError(name='%s.check_request_status' % self.name) > + raise NotImplementedError('%s.check_request_status' % self.name) > > def revoke_certificate(self, serial_number, revocation_reason=0): > """ > @@ -101,7 +101,7 @@ class rabase(Backend): > :param serial_number: Certificate serial number. > :param revocation_reason: Integer code of revocation reason. > """ > - raise errors.NotImplementedError(name='%s.check_request_status' % self.name) > + raise NotImplementedError('%s.check_request_status' % self.name) > > def take_certificate_off_hold(self, serial_number): > """ > @@ -109,5 +109,5 @@ class rabase(Backend): > > :param serial_number: Certificate serial number. > """ > - raise errors.NotImplementedError('%s.check_request_status' % self.name) > + raise NotImplementedError('%s.check_request_status' % self.name) This will go to the client as an Internal Error. Is this what we want? I must've forgotten to include errors.py when I created the original patch for this. I created a new error class for this but it is lost now. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From jdennis at redhat.com Fri Nov 20 19:55:18 2009 From: jdennis at redhat.com (John Dennis) Date: Fri, 20 Nov 2009 14:55:18 -0500 Subject: [Freeipa-devel] [PATCH] Fix raising nonexistent errros.NotImplemented In-Reply-To: <4B06F309.6000906@redhat.com> References: <20091120192831.32511.85522.stgit@jdennis> <4B06F309.6000906@redhat.com> Message-ID: <4B06F426.8050009@redhat.com> On 11/20/2009 02:50 PM, Rob Crittenden wrote: > John Dennis wrote: >> - raise errors.NotImplementedError('%s.check_request_status' % self.name) >> + raise NotImplementedError('%s.check_request_status' % self.name) > > This will go to the client as an Internal Error. Is this what we want? > > I must've forgotten to include errors.py when I created the original > patch for this. I created a new error class for this but it is lost now. Good question, FWIW it seems like all the other uses of NotImplemented use the built-in exception rather one derived from PublicError. In a way I could imagine this is actually an internal error however it would really result from misconfiguration of the server. I think having it be an internal error is O.K. but I don't have strong feelings either way. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Mon Nov 23 21:20:46 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 23 Nov 2009 16:20:46 -0500 Subject: [Freeipa-devel] [PATCH] 315 tab completion for ipa command In-Reply-To: <4AFDBE28.5070108@redhat.com> References: <4AFDBE28.5070108@redhat.com> Message-ID: <4B0AFCAE.3030005@redhat.com> Rob Crittenden wrote: > This adds bash tab completion for the ipa command. > > Note that this is sourced when you log in, so installing the package > isn't enough to load this file. Alternatively you can do: > > % source /etc/bash_completion.d/ipa > % ipa user > user-add user-find user-mod user-unlock > user-del user-lock user-show > > rob re-based patch should now apply cleanly. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-315.2-tab.patch Type: application/mbox Size: 2876 bytes Desc: not available URL: From rcritten at redhat.com Mon Nov 23 21:25:12 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 23 Nov 2009 16:25:12 -0500 Subject: [Freeipa-devel] [PATCH] Add {user, host, sourcehost}Category to HBAC and make accessTime multivalue. In-Reply-To: <4B0425BB.3030700@redhat.com> References: <4B0425BB.3030700@redhat.com> Message-ID: <4B0AFDB8.5080407@redhat.com> Pavel Zuna wrote: > Due to the format of accessTime (it has commas and spaces in it), we > can't use the List parameter type. I made it so that accessTime values > have to be entered one by one using new commands. > > We also agreed, that we're going to rename GeneralizedTime parameter to > AccessTime to prevent confusion with RFC 4517 standard. I attached a > separate patch for clarity. > > Pavel A couple of questions: - Would it make sense to leave time in as an option that takes a singular value? If someone wants multiple times they can use the new add interface, right? - What are these new enums for? If there is only one choice do you really have a choice? - We still need some tests for GeneralizedTime/AccessTime. rob From rcritten at redhat.com Mon Nov 23 21:30:09 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 23 Nov 2009 16:30:09 -0500 Subject: [Freeipa-devel] [PATCH] If plugin fails to load log the traceback In-Reply-To: <20091119165535.15148.5173.stgit@jdennis> References: <20091119165535.15148.5173.stgit@jdennis> Message-ID: <4B0AFEE1.2060501@redhat.com> John Dennis wrote: > Signed-off-by: John Dennis > > If plugin fails to load log the traceback > > If a plugin fails to load due to some kind of error it would be nice > if the error log contained the traceback so you can examine what went > wrong rather than being left blind as to why it failed to load. > > --- > > ipalib/plugable.py | 3 ++- > 1 files changed, 2 insertions(+), 1 deletions(-) > > > ------------------------------------------------------------------------ > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel ack, pushed to master From rcritten at redhat.com Mon Nov 23 21:31:15 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 23 Nov 2009 16:31:15 -0500 Subject: [Freeipa-devel] [PATCH] along with stdout, stderr also log the initial command In-Reply-To: <20091119171350.15432.68898.stgit@jdennis> References: <20091119171350.15432.68898.stgit@jdennis> Message-ID: <4B0AFF23.8000305@redhat.com> John Dennis wrote: > Signed-off-by: John Dennis > > along with stdout,stderr also log the initial command > > This implements better logging of external commands. Formerly we were > just outputting stdout and stderr without labeling which was which. We > also omitted the initial command and it's arguments. This made it > difficult when reviewing the logs to know what the command was and > what was stdout vs. stderr. This patch fixes that. > > --- > > ipapython/ipautil.py | 5 +++-- > 1 files changed, 3 insertions(+), 2 deletions(-) > > > ------------------------------------------------------------------------ > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel ack, pushed to master From rcritten at redhat.com Mon Nov 23 21:53:58 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 23 Nov 2009 16:53:58 -0500 Subject: [Freeipa-devel] [PATCH] [Resend] Reading INT parameter class should respect radix prefix In-Reply-To: <20091120183928.31838.86725.stgit@jdennis> References: <20091120183928.31838.86725.stgit@jdennis> Message-ID: <4B0B0476.7080908@redhat.com> John Dennis wrote: > This modifies the original patch by including a unit test, handling floats > when passed as unicode, and handling large magnitude values beyond maxint. > > The INT parameter class was not respecting any radix prefix (e.g. 0x) the user > may have supplied. This patch implements _convert_scalar method for the Int > class so that we can pass the special radix base of zero to the int constructor > telling it to determine the radix from the prefix (if present). > --- > From: John Dennis > Subject: [PATCH] Reading INT parameter class should respect radix prefix > > > ipalib/parameters.py | 29 +++++++++++++++++++++++++++++ > tests/test_ipalib/test_parameters.py | 27 +++++++++++++++++++++++++++ > 2 files changed, 56 insertions(+), 0 deletions(-) > > diff --git a/ipalib/parameters.py b/ipalib/parameters.py > index 227757d..8a1aede 100644 > --- a/ipalib/parameters.py > +++ b/ipalib/parameters.py > @@ -909,6 +909,35 @@ class Int(Number): > self.nice, self.minvalue, self.maxvalue) > ) > > + def _convert_scalar(self, value, index=None): > + """ > + Convert a single scalar value. > + """ > + if type(value) in (int, long): > + return value > + if type(value) is unicode: > + # permit floating point strings > + if value.find(u'.') >= 0: > + try: > + return int(float(value)) > + except ValueError: > + pass > + else: > + try: > + # 2nd arg is radix base, 2nd arg only accepted for strings. > + # Zero means determine radix base from prefix (e.g. 0x for hex) > + return int(value, 0) > + except ValueError: > + pass > + if type(value) is float: > + try: > + return int(value) > + except ValueError: > + pass > + raise ConversionError(name=self.name, index=index, > + error=ugettext(self.type_error), > + ) > + > def _rule_minvalue(self, _, value): > """ > Check min constraint. > diff --git a/tests/test_ipalib/test_parameters.py b/tests/test_ipalib/test_parameters.py > index f6bc66f..d1c5f7f 100644 > --- a/tests/test_ipalib/test_parameters.py > +++ b/tests/test_ipalib/test_parameters.py > @@ -22,6 +22,7 @@ Test the `ipalib.parameters` module. > """ > > import re > +import sys > from types import NoneType > from inspect import isclass > from tests.util import raises, ClassChecker, read_only > @@ -1225,6 +1226,32 @@ class test_Int(ClassChecker): > assert dummy.called() is True > dummy.reset() > > + def test_convert_scalar(self): > + """ > + Test the `ipalib.parameters.Int._convert_scalar` method. > + Assure radix prefixes work, str objects fail, > + floats (native & string) are truncated, > + large magnitude values are promoted to long, > + empty strings & invalid numerical representations fail > + """ > + o = self.cls('my_number') > + # Assure invalid inputs raise error > + for bad in ['hello', u'hello', True, None, '10', u'', u'.']: > + e = raises(errors.ConversionError, o._convert_scalar, bad) > + assert e.name == 'my_number' > + assert e.index is None > + # Assure large magnatude values are handled correctly > + assert type(o._convert_scalar(sys.maxint*2)) == long > + assert o._convert_scalar(sys.maxint*2) == sys.maxint*2 > + assert o._convert_scalar(unicode(sys.maxint*2)) == sys.maxint*2 > + assert o._convert_scalar(long(16)) == 16 > + # Assure normal conversions produce expected result > + assert o._convert_scalar(u'16.99') == 16 > + assert o._convert_scalar(16.99) == 16 > + assert o._convert_scalar(u'16') == 16 > + assert o._convert_scalar(u'0x10') == 16 > + assert o._convert_scalar(u'020') == 16 > + > class test_Float(ClassChecker): > """ > Test the `ipalib.parameters.Float` class. > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel ack, pushed to master From rcritten at redhat.com Mon Nov 23 21:55:52 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 23 Nov 2009 16:55:52 -0500 Subject: [Freeipa-devel] [PATCH] Fix raising nonexistent errros.NotImplemented In-Reply-To: <4B06F426.8050009@redhat.com> References: <20091120192831.32511.85522.stgit@jdennis> <4B06F309.6000906@redhat.com> <4B06F426.8050009@redhat.com> Message-ID: <4B0B04E8.5070903@redhat.com> John Dennis wrote: > On 11/20/2009 02:50 PM, Rob Crittenden wrote: >> John Dennis wrote: >>> - raise errors.NotImplementedError('%s.check_request_status' % >>> self.name) >>> + raise NotImplementedError('%s.check_request_status' % self.name) >> >> This will go to the client as an Internal Error. Is this what we want? >> >> I must've forgotten to include errors.py when I created the original >> patch for this. I created a new error class for this but it is lost now. > > Good question, FWIW it seems like all the other uses of NotImplemented > use the built-in exception rather one derived from PublicError. > > In a way I could imagine this is actually an internal error however it > would really result from misconfiguration of the server. I think having > it be an internal error is O.K. but I don't have strong feelings either > way. > Yeah, I'm not sure what to do either. In this case the selfsign plugin will probably never implement these things. In this case should the plugin implement a barebones function that says "I don't do that" or should we raise a more friendly error for the client. It is probably cleaner to raise our own NotImplemented error for plugins. I think for the framework the existing NotImplemented is fine b/c the developer should/would catch those. rob From jderose at redhat.com Tue Nov 24 16:54:04 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Tue, 24 Nov 2009 09:54:04 -0700 Subject: [Freeipa-devel] [PATCH] Print only one line of docstrings in command listings. In-Reply-To: <4B055CD4.30804@redhat.com> References: <4B055CD4.30804@redhat.com> Message-ID: <1259081644.24813.1.camel@jgd-dsk> On Thu, 2009-11-19 at 15:57 +0100, Pavel Zuna wrote: > Full docstring is shown on `ipa help COMMAND` > > Pavel nack. There is already a Plugin.summary attribute containing the first line of the docstring. See ipalib/plugable.py line 170. From ssorce at redhat.com Tue Nov 24 17:08:45 2009 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 24 Nov 2009 12:08:45 -0500 Subject: [Freeipa-devel] [PATCH] handle kdb stash being a keytab In-Reply-To: <1248203321.29458.314.camel@localhost.localdomain> References: <20090720215328.GA10770@redhat.com> <1248203321.29458.314.camel@localhost.localdomain> Message-ID: <1259082525.19634.3014.camel@willson.li.ssimo.org> On Tue, 2009-07-21 at 15:08 -0400, Simo Sorce wrote: > On Mon, 2009-07-20 at 17:53 -0400, Nalin Dahyabhai wrote: > > In krb5 1.7 and later, the stash file > > (/var/kerberos/krb5kdc/.k5.$REALM > > on Fedora) is created in the regular keytab format instead of the > > older > > less-portable one. Based from comments and code in kt_file.c, > here's > > a > > change to try to recognize that case (the file starts with a magic > > number) and read the master key from Python. > > > > The KDC will still read either format, so I left the bits that set > > things up on replicas alone (advice appreciated). The patch works > as > > expected on my 64-bit box, both on RHEL5 (krb5 1.6.1 with a > > traditional > > stash file) and on Raw Hide (krb5 1.7 with a keytab). > > Ack This patch had been basically forgotten and never pushed. I have applied it and pushed to master. Simo. -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Tue Nov 24 20:02:01 2009 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 24 Nov 2009 15:02:01 -0500 Subject: [Freeipa-devel] patch for freeipa on Fedora 12 Message-ID: <1259092921.19634.3020.camel@willson.li.ssimo.org> Fedora 12 comes with a new version of Kerberos that now stored stash files as a keytab. This breaks installation for Freeipa 1.2.2 Attached a patch that solves this (you can apply it on a live system by patching krbinstance.py, see rpm -ql ipa-server to find where the file is located). I am going to make a new package for F-12 with this patch. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: Fix-install-with-krb-1.7.patch Type: text/x-patch Size: 4347 bytes Desc: not available URL: From rcritten at redhat.com Tue Nov 24 21:17:15 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 24 Nov 2009 16:17:15 -0500 Subject: [Freeipa-devel] [PATCH] 318 add PKCS#10 parser Message-ID: <4B0C4D5B.40103@redhat.com> The pyOpenSSL PKCS#10 parser doesn't provide a way to get to attributes so we can't get the subject alt names (or other interesting bits). This pyasn1-based parser adds that support. I'm also switching to the pyasn1 X509v3 support because older releases of pyOpenSSL lacked the get_components() method on subjects making it difficult to get a usable subject. This PKCS#10 parser cannot handle all possible attribute types. It should be robust enough to not blow up if it gets something it knows nothing about. If a subjectaltname extension is present in a CSR we: - require that the host(s) exist in IPA - If the requestor is a machine then the alt names must be present in the services managedBy attribute. This is so we can control what hosts(s) a machine can request a cert for. I'm working on a way to be able to set the service principal within the reuqest. Nalin's certmonger program will set it as an otherName in the GeneralNames attribute. We should be able to make principal an optional argument to cert-request and use the value from the CSR (and blow up if we get it neither way). rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-318-pkcs10.patch Type: application/mbox Size: 51534 bytes Desc: not available URL: From jderose at redhat.com Wed Nov 25 15:35:16 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Wed, 25 Nov 2009 08:35:16 -0700 Subject: [Freeipa-devel] [PATCH] jderose 027 Extensible return values Message-ID: <1259163317.24813.84.camel@jgd-dsk> This patch is big. There are some significant changes, but easily 50% of this patch is made up of small changes to plugins and their unit tests to port them to the lower-level API changes. All unit tests pass assuming you install without the --ca option. I didn't touch the cert/ra plugins as I know John has pending changes there. Here's a summary: 1. Our standard return value is now a dict, which gives us a nice way to extend the API without breaking clients/code. So in this respect, the return values are more deeply nested. Basically if you returned 'foo' before, now you return {'result': 'foo'}. 2. Previously nested data-structures have been flattened. Most importantly, these have been changed: Entry: [dn, entry] => {'result': entry} Entries: [entries, truncated] => {'result': entries, 'truncated': truncated} This was mostly done so that an entry/row/object/whatever can be consistently represented as a dict... whether it's from LDAP, or a database, or whatever else IPA might get glued into. Likewise, lists are now consistently used to represent lists of things, rather than also a struct-like pair, triple, etc. 3. We now have a way to specify a return value schema, and this schema is enforced. The top-level schema of the return value dict is described using the Param-like classes in ipalib/output.py. For example: from ipalib import output class user_add(LDAPCreate): has_output = ( output.Entry('result'), output.Output('primary_key', unicode, 'The user uid') ) This hypothetical `user_add` expects you to return something like this: dict( result=dict(givenname=u'Foo', sn=u'Bar', uid=u'fbar'), primary_key=u'fbar', ) A few commands have C clients whose API would break with these changes. If your command is one of them, you can disable the return value validation for now by setting the 'use_output_validation' attribute to False, like this: class join(Command): use_output_validation = False I already did this for join so the installer doesn't break. The plugin browser now shows the output schema. Just fire up the in-tree server like this: ./lite-server.py And then go to http://localhost:8888/ipa/ui/Command 4. I'm using the return values themselves to drive CLI and webUI output. As I've worked on extending meta-data for the webUI, I've realized a lot of it can be delivered in the return values themselves. This is good because we display the output more on what the commands actually return, less on what they merely say they will return... this keeps us honest and is another way to ensure we live up to our API contracts. My goal is to have a single base output_for_cli() implementation, and this patch get's us 90% there already. Not all the commands have been updated, but this will continue in additional patches (or be delegated). More webUI functionality has been enabled also, but not as much as I hoped for as work on the command plugins took so much time. The commands return a page now instead of JSON. 5. I stubbed out the remained missing gettext features we need. They aren't hooked up to gettext yet, but for plugin writers, the API is now complete. See ipalib/text.py for the implementation and ipalib/plugins/user.py for a good example of how they are used. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jderose-027-Extensible-return-values.patch Type: text/x-patch Size: 177821 bytes Desc: not available URL: From jderose at redhat.com Wed Nov 25 16:07:22 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Wed, 25 Nov 2009 09:07:22 -0700 Subject: [Freeipa-devel] [PATCH] 315 tab completion for ipa command In-Reply-To: <4B0AFCAE.3030005@redhat.com> References: <4AFDBE28.5070108@redhat.com> <4B0AFCAE.3030005@redhat.com> Message-ID: <1259165242.24813.94.camel@jgd-dsk> On Mon, 2009-11-23 at 16:20 -0500, Rob Crittenden wrote: > Rob Crittenden wrote: > > This adds bash tab completion for the ipa command. > > > > Note that this is sourced when you log in, so installing the package > > isn't enough to load this file. Alternatively you can do: > > > > % source /etc/bash_completion.d/ipa > > % ipa user > > user-add user-find user-mod user-unlock > > user-del user-lock user-show > > > > rob > > re-based patch should now apply cleanly. > > rob ack. pushed to master. Thanks, Rob, this is really nice. Two things I'd like to see going forward: 1. Enable it by default. Is it a Fedora policy to not have tab-completion enabled by default? What does a user need to do to enable this? 2. Improve performance. I'm to blame for this, but the start-up time of ipa is a bit too slow for this to feel right. As I've never profiled it, there is probably some low hanging fruit that could give us a decent boost. Otherwise we can prepare a static list of commands when the package is installed. From jderose at redhat.com Wed Nov 25 16:36:29 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Wed, 25 Nov 2009 09:36:29 -0700 Subject: [Freeipa-devel] [PATCH] 311 more integrated client install In-Reply-To: <4B059962.1000904@redhat.com> References: <4AFAE8D2.3050701@redhat.com> <1258627711.19095.9.camel@jgd-dsk> <4B059962.1000904@redhat.com> Message-ID: <1259166989.24813.96.camel@jgd-dsk> On Thu, 2009-11-19 at 14:15 -0500, Rob Crittenden wrote: > Jason Gerard DeRose wrote: > > On Wed, 2009-11-11 at 11:39 -0500, Rob Crittenden wrote: > >> This patch integrates ipa-join and ipa-rmkeytab into the client > >> installer. This will join a machine to the IPA realm and fetch a host > >> principal for /etc/krb5.keytab. > >> > >> On uninstall all principals for the realm will be removed from > >> /etc/krb5.keytab. > >> > >> By default installation will fail if a host principal cannot be > >> obtained. Use the --force option to continue anyway. > >> > >> rob > > > > nack. this is breaking the installer: > > > > [6/12]: creating a keytab for httpd > > [7/12]: Setting up ssl > > [8/12]: Setting up browser autoconfig > > [9/12]: publish CA cert > > [10/12]: configuring SELinux for httpd > > [11/12]: restarting httpd > > [12/12]: configuring httpd to start on boot > > done configuring httpd. > > Applying LDAP updates > > restarting the directory server > > restarting the KDC > > Sample zone file for bind has been created in /tmp/sample.zone.WUedsi.db > > Configuration of client side components failed! > > ipa-client-install returned: Command '/usr/sbin/ipa-client-install > > --on-master --unattended --domain example.com --server > > fedora11.example.com --realm EXAMPLE.COM' returned non-zero exit status > > 1 > > It shouldn't require a password/principal when on the master. New patch > attached. > > rob ack. pushed to master. From rcritten at redhat.com Wed Nov 25 16:37:41 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 25 Nov 2009 11:37:41 -0500 Subject: [Freeipa-devel] [PATCH] 319 add -s option to ipa-join Message-ID: <4B0D5D55.7050905@redhat.com> In ipa-client-install we do the ipa-join before creating any of the configuration files. I added a -s option to ipa-join to specify the IPA server since it won't be defined in /etc/ipa/default.conf yet. I discovered to my chagrin that previous testing of this worked because /etc/ipa/default.conf isn't owned by our packages. I'll fix this in a future patch. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-319-join.patch Type: application/mbox Size: 3636 bytes Desc: not available URL: From jderose at redhat.com Wed Nov 25 16:47:28 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Wed, 25 Nov 2009 09:47:28 -0700 Subject: [Freeipa-devel] [PATCH] 312 fix some ACI parsing bugs In-Reply-To: <4AFC5145.6080605@redhat.com> References: <4AFC5145.6080605@redhat.com> Message-ID: <1259167648.24813.97.camel@jgd-dsk> On Thu, 2009-11-12 at 13:17 -0500, Rob Crittenden wrote: > This fixes 2 bugs in the ACI parser: > > 1. When looking for the version section I wasn't specific enough. If the > aci had the attribute nsosversion in it this was found instead. I > switched it to look for version 3.0 instead. This removes a tiny amount > of flexibility but since we create all the ACIs ourselves I think it is ok. > > 2. In isequal() if one of the ACIs had no targetattrs it was trying to > create a set out of None which is illegal. > > rob ack. pushed to master. From jderose at redhat.com Wed Nov 25 16:53:53 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Wed, 25 Nov 2009 09:53:53 -0700 Subject: [Freeipa-devel] [PATCH] 313 fix aci plugin host helper In-Reply-To: <4AFC528D.40601@redhat.com> References: <4AFC528D.40601@redhat.com> Message-ID: <1259168033.24813.98.camel@jgd-dsk> On Thu, 2009-11-12 at 13:23 -0500, Rob Crittenden wrote: > When creating an aci to cover host objects the wrong attribute is used > in the DN. It should be using fqdn, not cn. > > rob ack. pushed to master. From jderose at redhat.com Wed Nov 25 17:10:01 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Wed, 25 Nov 2009 10:10:01 -0700 Subject: [Freeipa-devel] [PATCH] 317 fix location of CA In-Reply-To: <4B0574E6.60006@redhat.com> References: <4B0574E6.60006@redhat.com> Message-ID: <1259169001.24813.99.camel@jgd-dsk> On Thu, 2009-11-19 at 11:40 -0500, Rob Crittenden wrote: > The output of ipa-server-install pointed to the old location of the > self-signed database, the 389-DS instance. It is now stored in the > Apache NSS database. > > Also set a db password on the 389-DS NSS database. It was using a blank > password. > > rob ack. pushed to master. From pzuna at redhat.com Wed Nov 25 17:16:59 2009 From: pzuna at redhat.com (=?UTF-8?B?UGF2ZWwgWsWvbmE=?=) Date: Wed, 25 Nov 2009 18:16:59 +0100 Subject: [Freeipa-devel] [PATCH] 285 CRL publishing In-Reply-To: <4B030228.8010409@redhat.com> References: <4B030228.8010409@redhat.com> Message-ID: <4B0D668B.20104@redhat.com> Rob Crittenden wrote: > This enables CRL publishing by dogtag to a place where Apache can get > the files. > > I have to do a couple of tricks here because dogtag is an optional > component. This is why in the installer I first see if the dogtag > SELinux policy is installed and if not add it. Similarly the installer > will remove it upon uninstall. > > The policy itself just lets dogtag write to some Apache-labeled > directories. dogtag uses symlinks to mark the latest CRL hence the > permissions for links. > > rob > The patch looks fine, except that it doesn't apply on the current tree. Pavel From jderose at redhat.com Wed Nov 25 18:16:35 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Wed, 25 Nov 2009 11:16:35 -0700 Subject: [Freeipa-devel] [PATCH] jderose 027 Extensible return values In-Reply-To: <4B0D63D9.9010602@redhat.com> References: <1259163317.24813.84.camel@jgd-dsk> <4B0D63D9.9010602@redhat.com> Message-ID: <1259172995.24813.115.camel@jgd-dsk> On Wed, 2009-11-25 at 12:05 -0500, Rob Crittenden wrote: > > This is purely from reading the patch, I haven't applied and tested it yet. > > ipalib/output.py: > +primary_key = Output('primary_key', unicode, > + 'The primary key of the deleted entry' > +) > > This isn't only for deleted entries, right? Ah, yeah, that should be made more generic. This doc message is only used by developers, though. > This import doesn't seem to be used: > from inspect import getdoc > > What is dont_output_for_cli()? Is this an effort to make things work > while we're in transition? Yeah, I just renamed some methods so we can reference how they were implemented. Temporary. > You seem to have disabled the raw option in LDAPSearch, was that > intentional? Originally I got the impression we weren't going to keep both --raw and --all, but this can be changed. > Is cli_name being dropped for label? I'm ok with that but should we > remove it from all the plugins? No, here is how they work: `cli_name` is used for the optparse names and defaults to Param.name, like: --first `label` is a human readable, translatable string. It's used in the webUI, and to prompt show entries on cli, like: First name: John Doe `doc` is human readable help passed to optparse.make_option(help=doc). It default to the value of the label. It's used like this: --uid=INT UID (use this option to set it manually) In the above case the `label` is "UID" (not shown) but the `doc` is this longer string. The user plugins provide good examples of how I think these should be used. I'll submit a patch later documented these different string uses. > rob From jderose at redhat.com Wed Nov 25 19:23:54 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Wed, 25 Nov 2009 12:23:54 -0700 Subject: [Freeipa-devel] [PATCH] 285 CRL publishing In-Reply-To: <4B0D7B4D.6090407@redhat.com> References: <4B030228.8010409@redhat.com> <1259165972.24813.95.camel@jgd-dsk> <4B0D7B4D.6090407@redhat.com> Message-ID: <1259177035.24813.121.camel@jgd-dsk> On Wed, 2009-11-25 at 13:45 -0500, Rob Crittenden wrote: > Jason Gerard DeRose wrote: > > On Tue, 2009-11-17 at 15:06 -0500, Rob Crittenden wrote: > >> This enables CRL publishing by dogtag to a place where Apache can get > >> the files. > >> > >> I have to do a couple of tricks here because dogtag is an optional > >> component. This is why in the installer I first see if the dogtag > >> SELinux policy is installed and if not add it. Similarly the installer > >> will remove it upon uninstall. > >> > >> The policy itself just lets dogtag write to some Apache-labeled > >> directories. dogtag uses symlinks to mark the latest CRL hence the > >> permissions for links. > >> > >> rob > > > > can't get this to apply: > > > > Applying: Add SELinux policy for CRL file publishing. > > error: patch failed: ipa.spec.in:379 > > error: ipa.spec.in: patch does not apply > > error: patch failed: selinux/Makefile:1 > > error: selinux/Makefile: patch does not apply > > Patch failed at 0001 Add SELinux policy for CRL file publishing. > > When you have resolved this problem run "git am --resolved". > > If you would prefer to skip this patch, instead run "git am --skip". > > To restore the original branch and stop patching run "git am --abort". > > > > > > Rebased patch attached. > nack. This seems to be breaking the installer. This was a clean build and install: Failed to populate the realm structure in kerberos Command '/usr/kerberos/sbin/kdb5_ldap_util -D uid=kdc,cn=sysaccounts,cn=etc,dc=example,dc=com -w Xl"t%3j8}VX create -s -P >grbc"/F+Sh` -r EXAMPLE.COM -subtrees dc=example,dc=com -sscope sub' returned non-zero exit status 1 [6/13]: adding default keytypes root : CRITICAL Failed to load default-keytypes.ldif: Command '/usr/bin/ldapmodify -h 127.0.0.1 -xv -D cn=Directory Manager -y /tmp/tmpdRo9BD -f /tmp/tmpdls3uk' returned non-zero exit status 32 ipa: CRITICAL: Failed to load default-keytypes.ldif: Command '/usr/bin/ldapmodify -h 127.0.0.1 -xv -D cn=Directory Manager -y /tmp/tmpdRo9BD -f /tmp/tmpdls3uk' returned non-zero exit status 32 [7/13]: creating a keytab for the directory Unexpected error - see ipaserver-install.log for details: Command '/usr/kerberos/sbin/kadmin.local -q addprinc -randkey ldap/fedora11.example.com at EXAMPLE.COM' returned non-zero exit status 1 I attached the log. -------------- next part -------------- A non-text attachment was scrubbed... Name: ipaserver-install.log Type: text/x-log Size: 40402 bytes Desc: not available URL: From rcritten at redhat.com Wed Nov 25 20:09:37 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 25 Nov 2009 15:09:37 -0500 Subject: [Freeipa-devel] [PATCH] 285 CRL publishing In-Reply-To: <1259177035.24813.121.camel@jgd-dsk> References: <4B030228.8010409@redhat.com> <1259165972.24813.95.camel@jgd-dsk> <4B0D7B4D.6090407@redhat.com> <1259177035.24813.121.camel@jgd-dsk> Message-ID: <4B0D8F01.6040208@redhat.com> Jason Gerard DeRose wrote: > On Wed, 2009-11-25 at 13:45 -0500, Rob Crittenden wrote: >> Jason Gerard DeRose wrote: >>> On Tue, 2009-11-17 at 15:06 -0500, Rob Crittenden wrote: >>>> This enables CRL publishing by dogtag to a place where Apache can get >>>> the files. >>>> >>>> I have to do a couple of tricks here because dogtag is an optional >>>> component. This is why in the installer I first see if the dogtag >>>> SELinux policy is installed and if not add it. Similarly the installer >>>> will remove it upon uninstall. >>>> >>>> The policy itself just lets dogtag write to some Apache-labeled >>>> directories. dogtag uses symlinks to mark the latest CRL hence the >>>> permissions for links. >>>> >>>> rob >>> can't get this to apply: >>> >>> Applying: Add SELinux policy for CRL file publishing. >>> error: patch failed: ipa.spec.in:379 >>> error: ipa.spec.in: patch does not apply >>> error: patch failed: selinux/Makefile:1 >>> error: selinux/Makefile: patch does not apply >>> Patch failed at 0001 Add SELinux policy for CRL file publishing. >>> When you have resolved this problem run "git am --resolved". >>> If you would prefer to skip this patch, instead run "git am --skip". >>> To restore the original branch and stop patching run "git am --abort". >>> >>> >> Rebased patch attached. >> > > nack. This seems to be breaking the installer. This was a clean build > and install: > > Failed to populate the realm structure in kerberos Command > '/usr/kerberos/sbin/kdb5_ldap_util -D > uid=kdc,cn=sysaccounts,cn=etc,dc=example,dc=com -w Xl"t%3j8}VX create > -s -P >grbc"/F+Sh` -r EXAMPLE.COM -subtrees dc=example,dc=com -sscope > sub' returned non-zero exit status 1 > [6/13]: adding default keytypes > root : CRITICAL Failed to load default-keytypes.ldif: Command > '/usr/bin/ldapmodify -h 127.0.0.1 -xv -D cn=Directory Manager > -y /tmp/tmpdRo9BD -f /tmp/tmpdls3uk' returned non-zero exit status 32 > ipa: CRITICAL: Failed to load default-keytypes.ldif: Command > '/usr/bin/ldapmodify -h 127.0.0.1 -xv -D cn=Directory Manager > -y /tmp/tmpdRo9BD -f /tmp/tmpdls3uk' returned non-zero exit status 32 > [7/13]: creating a keytab for the directory > Unexpected error - see ipaserver-install.log for details: > Command '/usr/kerberos/sbin/kadmin.local -q addprinc -randkey > ldap/fedora11.example.com at EXAMPLE.COM' returned non-zero exit status 1 > > I attached the log. > > Very strange, I can't reproduce this. What release are you on? What version of krb5-server do you have installed? rob From rcritten at redhat.com Wed Nov 25 21:07:37 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 25 Nov 2009 16:07:37 -0500 Subject: [Freeipa-devel] [PATCH] jderose 027 Extensible return values In-Reply-To: <1259172995.24813.115.camel@jgd-dsk> References: <1259163317.24813.84.camel@jgd-dsk> <4B0D63D9.9010602@redhat.com> <1259172995.24813.115.camel@jgd-dsk> Message-ID: <4B0D9C99.5030607@redhat.com> Jason Gerard DeRose wrote: > On Wed, 2009-11-25 at 12:05 -0500, Rob Crittenden wrote: >> This is purely from reading the patch, I haven't applied and tested it yet. >> >> ipalib/output.py: >> +primary_key = Output('primary_key', unicode, >> + 'The primary key of the deleted entry' >> +) >> >> This isn't only for deleted entries, right? > > Ah, yeah, that should be made more generic. This doc message is only > used by developers, though. > >> This import doesn't seem to be used: >> from inspect import getdoc >> >> What is dont_output_for_cli()? Is this an effort to make things work >> while we're in transition? > > Yeah, I just renamed some methods so we can reference how they were > implemented. Temporary. > >> You seem to have disabled the raw option in LDAPSearch, was that >> intentional? > > Originally I got the impression we weren't going to keep both --raw and > --all, but this can be changed. > >> Is cli_name being dropped for label? I'm ok with that but should we >> remove it from all the plugins? > > No, here is how they work: > > `cli_name` is used for the optparse names and defaults to Param.name, > like: > > --first > > `label` is a human readable, translatable string. It's used in the > webUI, and to prompt show entries on cli, like: > > First name: John Doe > > `doc` is human readable help passed to optparse.make_option(help=doc). > It default to the value of the label. It's used like this: > > --uid=INT UID (use this option to set it manually) > > In the above case the `label` is "UID" (not shown) but the `doc` is this > longer string. > > The user plugins provide good examples of how I think these should be > used. > > I'll submit a patch later documented these different string uses. > >> rob > We'll also need to determine what we'll do about all the plugins. The cert plugin, for example, isn't ported to this new return value system and blows up in many places. There are also some labels missing, such as for fqdn in the host plugin. These are both quite easy to fix, I think we just need to coordinate things. Perhaps if Pavel and I split up the plugins and fix anything that needs fixing and commit all the patches at one time to avoid any period of breakage. rob From rcritten at redhat.com Wed Nov 25 22:43:45 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 25 Nov 2009 17:43:45 -0500 Subject: [Freeipa-devel] [PATCH] 320 remove /etc/ipa/ipa.conf Message-ID: <4B0DB321.1020104@redhat.com> The configuration file /etc/ipa/ipa.conf was used by the v1 clients and servers to manually set realm, domain and server(s). This has been renamed to /etc/ipa/default.conf in v2. Some old utilities still referenced this old file and we still created it. This patch should completely remove it. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-320-conf.patch Type: application/mbox Size: 9280 bytes Desc: not available URL: From jderose at redhat.com Thu Nov 26 07:38:32 2009 From: jderose at redhat.com (Jason Gerard DeRose) Date: Thu, 26 Nov 2009 00:38:32 -0700 Subject: [Freeipa-devel] [PATCH] 285 CRL publishing In-Reply-To: <4B0D8F01.6040208@redhat.com> References: <4B030228.8010409@redhat.com> <1259165972.24813.95.camel@jgd-dsk> <4B0D7B4D.6090407@redhat.com> <1259177035.24813.121.camel@jgd-dsk> <4B0D8F01.6040208@redhat.com> Message-ID: <1259221112.24813.123.camel@jgd-dsk> On Wed, 2009-11-25 at 15:09 -0500, Rob Crittenden wrote: > Jason Gerard DeRose wrote: > > On Wed, 2009-11-25 at 13:45 -0500, Rob Crittenden wrote: > >> Jason Gerard DeRose wrote: > >>> On Tue, 2009-11-17 at 15:06 -0500, Rob Crittenden wrote: > >>>> This enables CRL publishing by dogtag to a place where Apache can get > >>>> the files. > >>>> > >>>> I have to do a couple of tricks here because dogtag is an optional > >>>> component. This is why in the installer I first see if the dogtag > >>>> SELinux policy is installed and if not add it. Similarly the installer > >>>> will remove it upon uninstall. > >>>> > >>>> The policy itself just lets dogtag write to some Apache-labeled > >>>> directories. dogtag uses symlinks to mark the latest CRL hence the > >>>> permissions for links. > >>>> > >>>> rob > >>> can't get this to apply: > >>> > >>> Applying: Add SELinux policy for CRL file publishing. > >>> error: patch failed: ipa.spec.in:379 > >>> error: ipa.spec.in: patch does not apply > >>> error: patch failed: selinux/Makefile:1 > >>> error: selinux/Makefile: patch does not apply > >>> Patch failed at 0001 Add SELinux policy for CRL file publishing. > >>> When you have resolved this problem run "git am --resolved". > >>> If you would prefer to skip this patch, instead run "git am --skip". > >>> To restore the original branch and stop patching run "git am --abort". > >>> > >>> > >> Rebased patch attached. > >> > > > > nack. This seems to be breaking the installer. This was a clean build > > and install: > > > > Failed to populate the realm structure in kerberos Command > > '/usr/kerberos/sbin/kdb5_ldap_util -D > > uid=kdc,cn=sysaccounts,cn=etc,dc=example,dc=com -w Xl"t%3j8}VX create > > -s -P >grbc"/F+Sh` -r EXAMPLE.COM -subtrees dc=example,dc=com -sscope > > sub' returned non-zero exit status 1 > > [6/13]: adding default keytypes > > root : CRITICAL Failed to load default-keytypes.ldif: Command > > '/usr/bin/ldapmodify -h 127.0.0.1 -xv -D cn=Directory Manager > > -y /tmp/tmpdRo9BD -f /tmp/tmpdls3uk' returned non-zero exit status 32 > > ipa: CRITICAL: Failed to load default-keytypes.ldif: Command > > '/usr/bin/ldapmodify -h 127.0.0.1 -xv -D cn=Directory Manager > > -y /tmp/tmpdRo9BD -f /tmp/tmpdls3uk' returned non-zero exit status 32 > > [7/13]: creating a keytab for the directory > > Unexpected error - see ipaserver-install.log for details: > > Command '/usr/kerberos/sbin/kadmin.local -q addprinc -randkey > > ldap/fedora11.example.com at EXAMPLE.COM' returned non-zero exit status 1 > > > > I attached the log. > > > > > > Very strange, I can't reproduce this. What release are you on? What > version of krb5-server do you have installed? > > rob Hmm, I must have had something weird in my tree. I just did two clean build and installs without error. ack. pushed to master. From pzuna at redhat.com Thu Nov 26 15:28:48 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Thu, 26 Nov 2009 16:28:48 +0100 Subject: [Freeipa-devel] [PATCH] Make host objects aware of their membership and that l==localityName. Message-ID: <4B0E9EB0.3040402@redhat.com> host object's didn't know they can be member of hostgroups, netgroups and rolegroups and they also didn't know that 'l' and 'localityName' is the same attribute. Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Make-host-objects-aware-of-their-membership-and-that.patch Type: application/mbox Size: 1280 bytes Desc: not available URL: From pzuna at redhat.com Thu Nov 26 15:29:37 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Thu, 26 Nov 2009 16:29:37 +0100 Subject: [Freeipa-devel] [PATCH] Make host objects aware of their membership and that l==localityName. In-Reply-To: <4B0E9EB0.3040402@redhat.com> References: <4B0E9EB0.3040402@redhat.com> Message-ID: <4B0E9EE1.5030109@redhat.com> Pavel Zuna wrote: > host object's didn't know they can be member of hostgroups, netgroups > and rolegroups and they also didn't know that 'l' and 'localityName' is > the same attribute. > > Pavel And here's the patch. :) Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Make-host-objects-aware-of-their-membership-and-that.patch Type: application/mbox Size: 1280 bytes Desc: not available URL: From pzuna at redhat.com Thu Nov 26 15:30:35 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Thu, 26 Nov 2009 16:30:35 +0100 Subject: [Freeipa-devel] [PATCH] Make host objects aware of their membership and that l==localityName. In-Reply-To: <4B0E9EE1.5030109@redhat.com> References: <4B0E9EB0.3040402@redhat.com> <4B0E9EE1.5030109@redhat.com> Message-ID: <4B0E9F1B.3000102@redhat.com> Pavel Zuna wrote: > Pavel Zuna wrote: >> host object's didn't know they can be member of hostgroups, netgroups >> and rolegroups and they also didn't know that 'l' and 'localityName' >> is the same attribute. >> >> Pavel > And here's the patch. :) > > Pavel I suspect they put some drugs in our air conditioner, the patch was already there... Pavel From pzuna at redhat.com Thu Nov 26 15:31:47 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Thu, 26 Nov 2009 16:31:47 +0100 Subject: [Freeipa-devel] [PATCH] Print only one line of docstrings in command listings. In-Reply-To: <1259081644.24813.1.camel@jgd-dsk> References: <4B055CD4.30804@redhat.com> <1259081644.24813.1.camel@jgd-dsk> Message-ID: <4B0E9F63.1040407@redhat.com> Jason Gerard DeRose wrote: > On Thu, 2009-11-19 at 15:57 +0100, Pavel Zuna wrote: >> Full docstring is shown on `ipa help COMMAND` >> >> Pavel > > nack. There is already a Plugin.summary attribute containing the first > line of the docstring. See ipalib/plugable.py line 170. > Ok, I didn't know that. Updated patch attached. Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Print-only-one-line-of-docstrings-in-command-listing.patch Type: application/mbox Size: 1600 bytes Desc: not available URL: From pzuna at redhat.com Thu Nov 26 15:33:05 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Thu, 26 Nov 2009 16:33:05 +0100 Subject: [Freeipa-devel] [PATCH] Fix takes_options in automount plugin. Message-ID: <4B0E9FB1.9050405@redhat.com> Automount plugin was overwriting options from its base class instead of adding new ones. Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0003-Fix-takes_options-in-automount-plugin.patch Type: application/mbox Size: 755 bytes Desc: not available URL: From pzuna at redhat.com Thu Nov 26 15:38:58 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Thu, 26 Nov 2009 16:38:58 +0100 Subject: [Freeipa-devel] [PATCH] Add {user, host, sourcehost}Category to HBAC and make accessTime multivalue. In-Reply-To: <4B0AFDB8.5080407@redhat.com> References: <4B0425BB.3030700@redhat.com> <4B0AFDB8.5080407@redhat.com> Message-ID: <4B0EA112.5050202@redhat.com> Rob Crittenden wrote: > Pavel Zuna wrote: >> Due to the format of accessTime (it has commas and spaces in it), we >> can't use the List parameter type. I made it so that accessTime values >> have to be entered one by one using new commands. >> >> We also agreed, that we're going to rename GeneralizedTime parameter >> to AccessTime to prevent confusion with RFC 4517 standard. I attached >> a separate patch for clarity. >> >> Pavel > > A couple of questions: > > - Would it make sense to leave time in as an option that takes a > singular value? If someone wants multiple times they can use the new add > interface, right? It would and I think it's a good idea, updated patch attached. > - What are these new enums for? If there is only one choice do you > really have a choice? Well for now, we only have the 'all' in categories, but the list is expected to grow. At first I didn't include categories in the plugin, because of this, but Sumit wanted it to be complete. > - We still need some tests for GeneralizedTime/AccessTime. Ok, added to my TODO list. > > rob Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0004-Add-user-host-sourcehost-Category-to-HBAC-and-make.patch Type: application/mbox Size: 6311 bytes Desc: not available URL: From pzuna at redhat.com Thu Nov 26 15:40:07 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Thu, 26 Nov 2009 16:40:07 +0100 Subject: [Freeipa-devel] [PATCH] Change object_class of group object. Message-ID: <4B0EA157.90007@redhat.com> Some groups created by default don't have ipaUserGroup and won't show up in searches. Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0006-Change-object_class-on-group-object.patch Type: application/mbox Size: 908 bytes Desc: not available URL: From pzuna at redhat.com Thu Nov 26 15:42:19 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Thu, 26 Nov 2009 16:42:19 +0100 Subject: [Freeipa-devel] [PATCH] Fix Bool parameter type. It was impossible to set to FALSE. Message-ID: <4B0EA1DB.7010208@redhat.com> It was behaving just like Flag except it didn't have autofill=True by default, so it was impossible to set it to False from the CLI. I also made it ignore case (so 'FalSE' and 'tRue' are accepted). Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0007-Fix-Bool-parameter-type.-It-was-impossible-to-set-it.patch Type: application/mbox Size: 1799 bytes Desc: not available URL: From pzuna at redhat.com Mon Nov 30 09:49:38 2009 From: pzuna at redhat.com (Pavel Zuna) Date: Mon, 30 Nov 2009 10:49:38 +0100 Subject: [Freeipa-devel] [PATCH] Fix boolean attributes in DNS plugin. Message-ID: <4B139532.9070307@redhat.com> Sometimes they worked fine and sometimes DS rejected them as invalid. Now they should always work. Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-boolean-attributes-in-DNS-plugin.patch Type: application/mbox Size: 1929 bytes Desc: not available URL: From mnagy at redhat.com Mon Nov 30 10:15:53 2009 From: mnagy at redhat.com (Martin Nagy) Date: Mon, 30 Nov 2009 11:15:53 +0100 Subject: [Freeipa-devel] [PATCH] Fix boolean attributes in DNS plugin. In-Reply-To: <4B139532.9070307@redhat.com> References: <4B139532.9070307@redhat.com> Message-ID: <1259576153.3326.24.camel@wolverine.englab.brq.redhat.com> On Mon, 2009-11-30 at 10:49 +0100, Pavel Zuna wrote: > Sometimes they worked fine and sometimes DS rejected them as invalid. Now they > should always work. > > Pavel Tested and ack. Martin From rcritten at redhat.com Mon Nov 30 18:26:39 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 30 Nov 2009 13:26:39 -0500 Subject: [Freeipa-devel] [PATCH] Print only one line of docstrings in command listings. In-Reply-To: <4B0E9F63.1040407@redhat.com> References: <4B055CD4.30804@redhat.com> <1259081644.24813.1.camel@jgd-dsk> <4B0E9F63.1040407@redhat.com> Message-ID: <4B140E5F.9050307@redhat.com> Pavel Zuna wrote: > Jason Gerard DeRose wrote: >> On Thu, 2009-11-19 at 15:57 +0100, Pavel Zuna wrote: >>> Full docstring is shown on `ipa help COMMAND` >>> >>> Pavel >> >> nack. There is already a Plugin.summary attribute containing the first >> line of the docstring. See ipalib/plugable.py line 170. >> > Ok, I didn't know that. Updated patch attached. > > Pavel ack, pushed to master From rcritten at redhat.com Mon Nov 30 18:28:40 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 30 Nov 2009 13:28:40 -0500 Subject: [Freeipa-devel] [PATCH] Fix takes_options in automount plugin. In-Reply-To: <4B0E9FB1.9050405@redhat.com> References: <4B0E9FB1.9050405@redhat.com> Message-ID: <4B140ED8.9050201@redhat.com> Pavel Zuna wrote: > Automount plugin was overwriting options from its base class instead of > adding new ones. > > Pavel ack, pushed to master From rcritten at redhat.com Mon Nov 30 18:34:39 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 30 Nov 2009 13:34:39 -0500 Subject: [Freeipa-devel] [PATCH] Add {user, host, sourcehost}Category to HBAC and make accessTime multivalue. In-Reply-To: <4B0EA112.5050202@redhat.com> References: <4B0425BB.3030700@redhat.com> <4B0AFDB8.5080407@redhat.com> <4B0EA112.5050202@redhat.com> Message-ID: <4B14103F.6040303@redhat.com> Pavel Zuna wrote: > Rob Crittenden wrote: >> Pavel Zuna wrote: >>> Due to the format of accessTime (it has commas and spaces in it), we >>> can't use the List parameter type. I made it so that accessTime >>> values have to be entered one by one using new commands. >>> >>> We also agreed, that we're going to rename GeneralizedTime parameter >>> to AccessTime to prevent confusion with RFC 4517 standard. I attached >>> a separate patch for clarity. >>> >>> Pavel >> >> A couple of questions: >> >> - Would it make sense to leave time in as an option that takes a >> singular value? If someone wants multiple times they can use the new >> add interface, right? > It would and I think it's a good idea, updated patch attached. > >> - What are these new enums for? If there is only one choice do you >> really have a choice? > Well for now, we only have the 'all' in categories, but the list is > expected to grow. At first I didn't include categories in the plugin, > because of this, but Sumit wanted it to be complete. > >> - We still need some tests for GeneralizedTime/AccessTime. > Ok, added to my TODO list. The patch isn't applying for me: $ patch -p1 --dry-run < 0003-Fix-takes_options-in-automount-plugin.patch patching file ipalib/plugins/hbac.py patching file tests/test_xmlrpc/test_hbac_plugin.py Hunk #1 FAILED at 52. Hunk #2 FAILED at 84. 2 out of 3 hunks FAILED -- saving rejects to file tests/test_xmlrpc/test_hbac_plugin.py.rej Since you have to mess with this anyway, can you: - add another test to also test adding the access time on the add. You added back the capability but the tests are still removed AFAICT. - add a FUTURE or FIXME comment indicating that the enumerators are future-proofing things by making them a 1-option enumerator for now? rob From rcritten at redhat.com Mon Nov 30 18:35:51 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 30 Nov 2009 13:35:51 -0500 Subject: [Freeipa-devel] [PATCH] Change object_class of group object. In-Reply-To: <4B0EA157.90007@redhat.com> References: <4B0EA157.90007@redhat.com> Message-ID: <4B141087.5020106@redhat.com> Pavel Zuna wrote: > Some groups created by default don't have ipaUserGroup and won't show up > in searches. > > Pavel > nack, isn't the better approach to fix up the groups that are created by default without the ipaUserGroup objectclass? rob From rcritten at redhat.com Mon Nov 30 18:38:45 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 30 Nov 2009 13:38:45 -0500 Subject: [Freeipa-devel] [PATCH] Fix Bool parameter type. It was impossible to set to FALSE. In-Reply-To: <4B0EA1DB.7010208@redhat.com> References: <4B0EA1DB.7010208@redhat.com> Message-ID: <4B141135.1060704@redhat.com> Pavel Zuna wrote: > It was behaving just like Flag except it didn't have autofill=True by > default, so it was impossible to set it to False from the CLI. I also > made it ignore case (so 'FalSE' and 'tRue' are accepted). > > Pavel ack, pushed to master From rcritten at redhat.com Mon Nov 30 18:39:59 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 30 Nov 2009 13:39:59 -0500 Subject: [Freeipa-devel] [PATCH] Fix boolean attributes in DNS plugin. In-Reply-To: <1259576153.3326.24.camel@wolverine.englab.brq.redhat.com> References: <4B139532.9070307@redhat.com> <1259576153.3326.24.camel@wolverine.englab.brq.redhat.com> Message-ID: <4B14117F.60307@redhat.com> Martin Nagy wrote: > On Mon, 2009-11-30 at 10:49 +0100, Pavel Zuna wrote: >> Sometimes they worked fine and sometimes DS rejected them as invalid. Now they >> should always work. >> >> Pavel > > Tested and ack. > Martin pushed to master From rcritten at redhat.com Mon Nov 30 20:29:22 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 30 Nov 2009 15:29:22 -0500 Subject: [Freeipa-devel] [PATCH] 309 make exception from ipautil.run() optional In-Reply-To: <4B03007C.6050109@redhat.com> References: <4AFAE737.7090507@redhat.com> <4AFAE928.1020801@redhat.com> <1258477421.15021.2.camel@jgd-dsk> <4B03007C.6050109@redhat.com> Message-ID: <4B142B22.7060308@redhat.com> Rob Crittenden wrote: > Jason Gerard DeRose wrote: >> On Wed, 2009-11-11 at 11:41 -0500, Rob Crittenden wrote: >>> Rob Crittenden wrote: >>>> There are probably occasions where a caller will want more control >>>> over what happens when running a command fails. I've added an >>>> optional argument to run where it will not raise an exception on >>>> errors. >>>> >>>> I've also added returncode to the tuple of things returned. >>>> >>>> rob >>> I forgot to include this additional change in the patch. When acked >>> I'll add this bit too and commit it. >>> >>> --- a/ipaserver/install/httpinstance.py >>> +++ b/ipaserver/install/httpinstance.py >>> @@ -100,7 +100,7 @@ class HTTPInstance(service.Service): >>> if selinux: >>> try: >>> # returns e.g. "httpd_can_network_connect --> off" >>> - (stdout, stderr) = ipautil.run(["/usr/sbin/getsebool", >>> + (stdout, stderr, returncode) = >>> ipautil.run(["/usr/sbin/getsebool", >>> >>> "httpd_can_network_connect"]) >>> self.backup_state("httpd_can_network_connect", >>> stdout.split()[2]) >>> except: >> >> ack. It all looks fine to me, although I can't get this patch to apply. >> > > Ok, it is failing because it relies on a patch I didn't submit yet. I'll > send that one out shortly and hold onto this one for now. > > rob Ok, the missing patch (285 CRL publishing) was acked and pushed. I pushed this one to master. rob From rcritten at redhat.com Mon Nov 30 20:50:02 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 30 Nov 2009 15:50:02 -0500 Subject: [Freeipa-devel] [PATCH] 321 better LDAP error handling in client Message-ID: <4B142FFA.3040104@redhat.com> This improves the LDAP exception handling in the client. The existing code spit out a slew of deprecation errors because of the use of the message attribute. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-321-client.patch Type: application/mbox Size: 1316 bytes Desc: not available URL: From rcritten at redhat.com Mon Nov 30 21:29:51 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 30 Nov 2009 16:29:51 -0500 Subject: [Freeipa-devel] [PATCH] Require current versions of python-nss & python-lxml In-Reply-To: <20091119233505.20374.89637.stgit@jdennis> References: <20091119233505.20374.89637.stgit@jdennis> Message-ID: <4B14394F.2080200@redhat.com> John Dennis wrote: > This adds a new dependency on python-lxml. It also requires that > python-nss be the current version (0.8). This is needed for the http > authentication work I did for Jason which was added in a recent > version of python-nss. This patch also fixes a deprecation warning > concerning nssinit() which is dependent on a newer version of > python-nss. > > To the best of my knowledge both python-lxml and python-nss.0-8 are > currently available in the RHEL5 IPA compose, they are both > definately available in fedora F-11, F-12, rawhide. > --- > From: John Dennis > Subject: [PATCH] Require current versions of python-nss & python-lxml > > > ipa.spec.in | 3 ++- > ipapython/nsslib.py | 2 +- > 2 files changed, 3 insertions(+), 2 deletions(-) > > diff --git a/ipa.spec.in b/ipa.spec.in > index ce2c0a6..659a76a 100644 > --- a/ipa.spec.in > +++ b/ipa.spec.in > @@ -173,7 +173,8 @@ Requires: python-kerberos >= 1.1-3 > Requires: authconfig > Requires: gnupg > Requires: pyOpenSSL > -Requires: python-nss > +Requires: python-nss >= 0.8 > +Requires: python-lxml > > %description python > IPA is an integrated solution to provide centrally managed Identity (machine, > diff --git a/ipapython/nsslib.py b/ipapython/nsslib.py > index 05b45c1..2052843 100644 > --- a/ipapython/nsslib.py > +++ b/ipapython/nsslib.py > @@ -95,7 +95,7 @@ class NSSConnection(httplib.HTTPConnection): > if not dbdir: > raise RuntimeError("dbdir is required") > > - ssl.nssinit(dbdir) > + nss.nss_init(dbdir) > ssl.set_domestic_policy() > nss.set_password_callback(self.password_callback) > I guess I forgot to send an e-mail about this. I acked and pushed it on Nov 23rd. rob From rcritten at redhat.com Mon Nov 30 22:31:44 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 30 Nov 2009 17:31:44 -0500 Subject: [Freeipa-devel] remove config files on uninstall? Message-ID: <4B1447D0.7090808@redhat.com> When one does an IPA server uninstall we currently leave the configuration files in /etc/ipa and the Apache configuration in /etc/httpd/conf.d/. I'm thinking we should remove these files when we do an uninstall. Any objections? rob From jdennis at redhat.com Mon Nov 30 22:34:44 2009 From: jdennis at redhat.com (John Dennis) Date: Mon, 30 Nov 2009 17:34:44 -0500 Subject: [Freeipa-devel] remove config files on uninstall? In-Reply-To: <4B1447D0.7090808@redhat.com> References: <4B1447D0.7090808@redhat.com> Message-ID: <4B144884.2000705@redhat.com> On 11/30/2009 05:31 PM, Rob Crittenden wrote: > When one does an IPA server uninstall we currently leave the > configuration files in /etc/ipa and the Apache configuration in > /etc/httpd/conf.d/. I'm thinking we should remove these files when we do > an uninstall. Any objections? I agree these should be removed. But shouldn't this be handled by rpm? In other words shouldn't the files be %ghost and marked as %config? -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Mon Nov 30 22:41:00 2009 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 30 Nov 2009 17:41:00 -0500 Subject: [Freeipa-devel] remove config files on uninstall? In-Reply-To: <4B144884.2000705@redhat.com> References: <4B1447D0.7090808@redhat.com> <4B144884.2000705@redhat.com> Message-ID: <4B1449FC.4020700@redhat.com> John Dennis wrote: > On 11/30/2009 05:31 PM, Rob Crittenden wrote: >> When one does an IPA server uninstall we currently leave the >> configuration files in /etc/ipa and the Apache configuration in >> /etc/httpd/conf.d/. I'm thinking we should remove these files when we do >> an uninstall. Any objections? > > I agree these should be removed. But shouldn't this be handled by rpm? > In other words shouldn't the files be %ghost and marked as %config? > They are currently ghosted but that is more for protecting them between rpm upgrades on an existing server than for removing them when the package is uninstalled. In fact, they aren't removed when you remove the package (to preserve any custom changes). rob