From edewata at redhat.com Thu Sep 1 15:04:08 2016 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 1 Sep 2016 11:04:08 -0400 (EDT) Subject: [Pki-devel] [PATCH] 827 Added support to create system certificates in different tokens. In-Reply-To: <93be7cad-3aa8-d858-1bbc-ac618219ea75@redhat.com> References: <9d73b560-17e9-6421-9b92-b54d80851173@redhat.com> <93be7cad-3aa8-d858-1bbc-ac618219ea75@redhat.com> Message-ID: <1916132391.7605255.1472742248458.JavaMail.zimbra@redhat.com> I think if you search for the usage of CryptoToken.login(), it's done at the beginning of configuration servlet (SystemConfigService) and in the beginning of server startup (JssSubsystem). TomcatJSS might also do the same thing (I haven't checked the code). This patch doesn't change the order of token login and the certificate creations. It mainly changes how the tokenname parameters are initialized. Previously it's done by the configuration servlet but with the wrong value. Now it's done in pkispawn with the correct value. -- Endi S. Dewata ----- Original Message ----- > > > I'm less familiar with the area, so I'm just going to ask a question. Where > in the new code does it handle taking in passwords and logging into the > extra token(s)? > > > thanks, > > Christina > > On 08/31/2016 12:35 PM, Endi Sukma Dewata wrote: > > > Previously all system certificates were always created in the same > token specified in the pki_token_name parameter. > > To allow creating system certificates in different tokens, the > configuration.py has been modified to store the system certificate > token names specified in pki__token parameters into the > CS.cfg before the server is started. > > After the server is started, the configuration servlet will read > the token names from the CS.cfg and create the certificates in the > appropriate token. > > https://fedorahosted.org/pki/ticket/2449 > > > > _______________________________________________ > Pki-devel mailing list Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel > > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From alee at redhat.com Fri Sep 2 15:12:47 2016 From: alee at redhat.com (Ade Lee) Date: Fri, 02 Sep 2016 11:12:47 -0400 Subject: [Pki-devel] [PATCH] 827 Added support to create system certificates in different tokens. In-Reply-To: <1916132391.7605255.1472742248458.JavaMail.zimbra@redhat.com> References: <9d73b560-17e9-6421-9b92-b54d80851173@redhat.com> <93be7cad-3aa8-d858-1bbc-ac618219ea75@redhat.com> <1916132391.7605255.1472742248458.JavaMail.zimbra@redhat.com> Message-ID: <1472829167.3934.4.camel@redhat.com> This looks OK to me. I will merge it Ade On Thu, 2016-09-01 at 11:04 -0400, Endi Sukma Dewata wrote: > I think if you search for the usage of CryptoToken.login(), it's done > at the beginning of configuration servlet (SystemConfigService) and > in the beginning of server startup (JssSubsystem). TomcatJSS might > also do the same thing (I haven't checked the code). > > This patch doesn't change the order of token login and the > certificate creations. It mainly changes how the tokenname parameters > are initialized. Previously it's done by the configuration servlet > but with the wrong value. Now it's done in pkispawn with the correct > value. > > -- > Endi S. Dewata > > ----- Original Message ----- > > > > > > I'm less familiar with the area, so I'm just going to ask a > > question. Where > > in the new code does it handle taking in passwords and logging into > > the > > extra token(s)? > > > > > > thanks, > > > > Christina > > > > On 08/31/2016 12:35 PM, Endi Sukma Dewata wrote: > > > > > > Previously all system certificates were always created in the same > > token specified in the pki_token_name parameter. > > > > To allow creating system certificates in different tokens, the > > configuration.py has been modified to store the system certificate > > token names specified in pki__token parameters into the > > CS.cfg before the server is started. > > > > After the server is started, the configuration servlet will read > > the token names from the CS.cfg and create the certificates in the > > appropriate token. > > > > https://fedorahosted.org/pki/ticket/2449 > > > > > > > > _______________________________________________ > > Pki-devel mailing list Pki-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/pki-devel > > > > > > _______________________________________________ > > Pki-devel mailing list > > Pki-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/pki-devel > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From alee at redhat.com Fri Sep 2 20:14:52 2016 From: alee at redhat.com (Ade Lee) Date: Fri, 02 Sep 2016 16:14:52 -0400 Subject: [Pki-devel] [PATCH] 233 - fix incrorrect URLs in CertRequestInfos Message-ID: <1472847292.3934.7.camel@redhat.com> Fix CertRequestInfo URLs The URLs were generated by a UriBuilder that referred to the resource's annotated path. This top-level path changed though, even if the underlying paths did not. Replace this with a reference to the getX methods instead. Also fixed a few eclipse flagged warnings (unused imports etc). Ticket 2447 Please review .. -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-vakwetu-0330-Fix-CertRequestInfo-URLs.patch Type: text/x-patch Size: 10402 bytes Desc: not available URL: From alee at redhat.com Fri Sep 2 20:22:04 2016 From: alee at redhat.com (Ade Lee) Date: Fri, 02 Sep 2016 16:22:04 -0400 Subject: [Pki-devel] [PATCH] 233 - fix incrorrect URLs in CertRequestInfos In-Reply-To: <1472847292.3934.7.camel@redhat.com> References: <1472847292.3934.7.camel@redhat.com> Message-ID: <1472847724.3934.8.camel@redhat.com> Pushed to master on basis of trivial rule. Ade On Fri, 2016-09-02 at 16:14 -0400, Ade Lee wrote: > Fix CertRequestInfo URLs > > The URLs were generated by a UriBuilder that referred to the > resource's > annotated path. This top-level path changed though, even if the > underlying > paths did not. Replace this with a reference to the getX methods > instead. > > Also fixed a few eclipse flagged warnings (unused imports etc). > > Ticket 2447 > > Please review .. > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From abokovoy at redhat.com Sat Sep 3 22:04:33 2016 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Sun, 4 Sep 2016 01:04:33 +0300 Subject: [Pki-devel] [Freeipa-devel] Karma Requests for pki-core-10.3.5-4 In-Reply-To: References: Message-ID: <20160903220239.izvulsplt3rlkhya@redhat.com> On Tue, 30 Aug 2016, Matthew Harmsen wrote: >*The following updated candidate builds of pki-core 10.3.5 on Fedora >24, 25, and 26 (rawhide) consist of the following: >* > > * *Fedora 24* > o *pki-core-10.3.5-4.fc24 > > * > * *Fedora 25* > o *pki-core-10.3.5-4.fc25 > > * > * *Fedora 26* > o *pki-core-10.3.5-4.fc26 > > * > Unfortunately, upgrade in Fedora 24 does not work for existing FreeIPA deployments due to lack upgrade for dangling symlinks of jaxrs-api.jar. I filed a ticket https://fedorahosted.org/pki/ticket/2452. Please fix it ASAP because we already have users in Fedora 24 complaining about broken deployments after a mere 'dnf update'. -- / Alexander Bokovoy From edewata at redhat.com Tue Sep 6 18:49:07 2016 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 6 Sep 2016 13:49:07 -0500 Subject: [Pki-devel] [PATCH] 0133 Revoke lightweight CA certificate on deletion In-Reply-To: <20160825041840.GL3877@dhcp-40-8.bne.redhat.com> References: <20160825041840.GL3877@dhcp-40-8.bne.redhat.com> Message-ID: On 8/24/2016 11:18 PM, Fraser Tweedale wrote: > Hi team, > > The attached patch implements cert revocation on LWCA deletion. The > TODO for parametrising over revocation reason and invalid date is > intentional - I just want to get the minimal viable solution into > 10.3.x ASAP and we can look at what more is wanted/needed later. > > Thanks, > Fraser I was comparing this patch to the existing code in CertService, it looks like some methods of RevocationProcessor are not called by this patch: * setStartTime() * setInitiative() * setRequestType() * setComments() * validateNonce() * validateCertificateToRevoke() They seem to be related to audit or validation, so maybe that can be added later. Also there probably should be try-catch blocks to audit failed operations. I pushed the patch as is to master (10.4), but feel free to post additional patches to address the above issue. -- Endi S. Dewata From edewata at redhat.com Tue Sep 6 19:56:14 2016 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 6 Sep 2016 14:56:14 -0500 Subject: [Pki-devel] [PATCH] 827 Added support to create system certificates in different tokens. In-Reply-To: <1472829167.3934.4.camel@redhat.com> References: <9d73b560-17e9-6421-9b92-b54d80851173@redhat.com> <93be7cad-3aa8-d858-1bbc-ac618219ea75@redhat.com> <1916132391.7605255.1472742248458.JavaMail.zimbra@redhat.com> <1472829167.3934.4.camel@redhat.com> Message-ID: On 9/2/2016 10:12 AM, Ade Lee wrote: > This looks OK to me. I will merge it > > Ade Thanks! -- Endi S. Dewata From alee at redhat.com Tue Sep 6 21:17:23 2016 From: alee at redhat.com (Ade Lee) Date: Tue, 06 Sep 2016 17:17:23 -0400 Subject: [Pki-devel] [PATCH] 0131..0132 Fix LWCA entryUSN handling In-Reply-To: <20160824053635.GH3877@dhcp-40-8.bne.redhat.com> References: <20160824053635.GH3877@dhcp-40-8.bne.redhat.com> Message-ID: <1473196643.22020.2.camel@redhat.com> We still dont know how this state happened, but .. ack. Ade On Wed, 2016-08-24 at 15:36 +1000, Fraser Tweedale wrote: > The attached patches address a couple of issues related to handling > entryUSN attribute when reading lightweight CA entries. > > https://fedorahosted.org/pki/ticket/2444 > > Thanks, > Fraser > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From alee at redhat.com Tue Sep 6 21:17:39 2016 From: alee at redhat.com (Ade Lee) Date: Tue, 06 Sep 2016 17:17:39 -0400 Subject: [Pki-devel] [PATCH] 0130 Prevent deletion of host CA cert and key from NSSDB In-Reply-To: <20160824053449.GG3877@dhcp-40-8.bne.redhat.com> References: <20160824053449.GG3877@dhcp-40-8.bne.redhat.com> Message-ID: <1473196659.22020.3.camel@redhat.com> ack On Wed, 2016-08-24 at 15:34 +1000, Fraser Tweedale wrote: > Hi, > > Attached patch fixes https://fedorahosted.org/pki/ticket/2443. > > Thanks, > Fraser > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From alee at redhat.com Tue Sep 6 21:26:19 2016 From: alee at redhat.com (Ade Lee) Date: Tue, 06 Sep 2016 17:26:19 -0400 Subject: [Pki-devel] [PATCH] 0131..0132 Fix LWCA entryUSN handling In-Reply-To: <1473196643.22020.2.camel@redhat.com> References: <20160824053635.GH3877@dhcp-40-8.bne.redhat.com> <1473196643.22020.2.camel@redhat.com> Message-ID: <1473197179.22020.4.camel@redhat.com> pushed to master On Tue, 2016-09-06 at 17:17 -0400, Ade Lee wrote: > We still dont know how this state happened, but .. ack. > > Ade > On Wed, 2016-08-24 at 15:36 +1000, Fraser Tweedale wrote: > > The attached patches address a couple of issues related to handling > > entryUSN attribute when reading lightweight CA entries. > > > > https://fedorahosted.org/pki/ticket/2444 > > > > Thanks, > > Fraser > > _______________________________________________ > > Pki-devel mailing list > > Pki-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/pki-devel > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From alee at redhat.com Tue Sep 6 21:26:43 2016 From: alee at redhat.com (Ade Lee) Date: Tue, 06 Sep 2016 17:26:43 -0400 Subject: [Pki-devel] [PATCH] 0130 Prevent deletion of host CA cert and key from NSSDB In-Reply-To: <1473196659.22020.3.camel@redhat.com> References: <20160824053449.GG3877@dhcp-40-8.bne.redhat.com> <1473196659.22020.3.camel@redhat.com> Message-ID: <1473197203.22020.5.camel@redhat.com> pushed to master. On Tue, 2016-09-06 at 17:17 -0400, Ade Lee wrote: > ack > > On Wed, 2016-08-24 at 15:34 +1000, Fraser Tweedale wrote: > > Hi, > > > > Attached patch fixes https://fedorahosted.org/pki/ticket/2443. > > > > Thanks, > > Fraser > > _______________________________________________ > > Pki-devel mailing list > > Pki-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/pki-devel > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From edewata at redhat.com Tue Sep 6 22:55:14 2016 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 6 Sep 2016 17:55:14 -0500 Subject: [Pki-devel] [PATCH] 828-829 Removed FixSELinuxContexts upgrade script. Message-ID: The FixSELinuxContexts upgrade script has been removed temporarily due to a problem importing selinux library during RPM upgrade. The FixDeploymentDescriptor script number has been changed accordingly. The code in the RPM spec that moves the upgrade scripts has been updated to reflect the FixSELinuxContexts deletion. The libselinux-python is used by deployment and upgrade scripts to set the SELinux contexts, so a direct runtime dependency has been added to the RPM spec file. The duplicate python-ldap and python-lxml dependencies have been removed. https://fedorahosted.org/pki/ticket/2452 -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-edewata-0828-Removed-FixSELinuxContexts-upgrade-script.patch Type: text/x-patch Size: 2705 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-edewata-0829-Updated-RPM-spec.patch Type: text/x-patch Size: 2333 bytes Desc: not available URL: From edewata at redhat.com Tue Sep 6 23:00:28 2016 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 6 Sep 2016 18:00:28 -0500 Subject: [Pki-devel] [PATCH] 828-829 Removed FixSELinuxContexts upgrade script. In-Reply-To: References: Message-ID: <0cc0160d-d21b-fe59-061e-0ce25f7ee206@redhat.com> On 9/6/2016 5:55 PM, Endi Sukma Dewata wrote: > The FixSELinuxContexts upgrade script has been removed temporarily > due to a problem importing selinux library during RPM upgrade. > > The FixDeploymentDescriptor script number has been changed > accordingly. > > The code in the RPM spec that moves the upgrade scripts has been > updated to reflect the FixSELinuxContexts deletion. > > The libselinux-python is used by deployment and upgrade scripts > to set the SELinux contexts, so a direct runtime dependency has > been added to the RPM spec file. > > The duplicate python-ldap and python-lxml dependencies have been > removed. > > https://fedorahosted.org/pki/ticket/2452 Pushed to master (10.4) under one-liner/trivial rule. -- Endi S. Dewata From emaldona at redhat.com Thu Sep 8 00:30:36 2016 From: emaldona at redhat.com (Elio Maldonado) Date: Wed, 7 Sep 2016 20:30:36 -0400 (EDT) Subject: [Pki-devel] Fedora PKI_Documentation page has some broken links In-Reply-To: <1735242686.14649107.1473294336366.JavaMail.zimbra@redhat.com> Message-ID: <495394400.14650140.1473294636924.JavaMail.zimbra@redhat.com> On http://pki.fedoraproject.org/wiki/PKI_DocumentationtThe Quick Links area has these 3 broken links http://directory.fedoraproject.org/wiki/Fortitude not found, haven't found a replacement, there probably is none at all http://directory.fedoraproject.org/wiki/CoolKey not found and should be changed to http://pki.fedoraproject.org/wiki/CoolKey http://directory.fedoraproject.org/wiki/Windows_Certificate_Auto_Enrollment not found, found this one: https://access.redhat.com/documentation/en-US/Red_Hat_Certificate_System/8.0/html/Admin_Guide/autoenrollemnt-proxy.html not sure if it's a good one given that's for CS 8.0 -Elio From mharmsen at redhat.com Thu Sep 8 00:33:06 2016 From: mharmsen at redhat.com (Matthew Harmsen) Date: Wed, 7 Sep 2016 18:33:06 -0600 Subject: [Pki-devel] Karma Requests for pki-core-10.3.5-5 Message-ID: <09cf941b-4dfa-f376-b775-ba5954f403dd@redhat.com> *The following updated candidate builds of pki-core 10.3.5 on Fedora 24, 25, and 26 (rawhide) consist of the following: * * *Fedora 24* o *pki-core-10.3.5-5.fc24 * * *Fedora 25* o *pki-core-10.3.5-5.fc25 * * *Fedora 26* o *pki-core-10.3.5-5.fc26 * *Additionally, the CentOS 7 COPR EPEL Builds of Dogtag 10.3.3 were also updated:* * *https://copr.fedorainfracloud.org/coprs/g/pki/10.3.3/repo/epel-7/group_pki-10.3.3-epel-7.repo* [group_pki-10.3.3] name=Copr repo for 10.3.3 owned by @pki baseurl=https://copr-be.cloud.fedoraproject.org/results/@pki/10.3.3/epel-7-$basearch/ skip_if_unavailable=True gpgcheck=1 gpgkey=https://copr-be.cloud.fedoraproject.org/results/@pki/10.3.3/pubkey.gpg enabled=1 enabled_metadata=1 *These builds address the following PKI tickets: * * PKI TRAC Ticket #1638 - Lightweight CAs: revoke certificate on CA deletion * PKI TRAC Ticket #2346 - Dogtag 10.3.6: Miscellaneous Enhancements * PKI TRAC Ticket #2443 - Prevent deletion of host CA's keys if LWCA entry deleted * PKI TRAC Ticket #2444 - Authority entry without entryUSN is skipped even if USN plugin enabled * PKI TRAC Ticket #2446 - pkispawn: make subject_dn defaults unique per instance name (for shared HSM) * PKI TRAC Ticket #2447 - CertRequestInfo has incorrect URLs * PKI TRAC Ticket #2449 - Unable to create system certificates in different tokens *Please provide Karma for the following builds: * * *Fedora 24* o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-994f943797 pki-core-10.3.5-5.fc24* * *Fedora 25* o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-d363d36e22 pki-core-10.3.5-5.fc25 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From edewata at redhat.com Thu Sep 8 19:13:06 2016 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 8 Sep 2016 14:13:06 -0500 Subject: [Pki-devel] [PATCH] 830 Removed support for creating system certificates in different tokens. Message-ID: The patch that added the support for creating system certificates in different tokens causes issues in certain cases, so for now it has been reverted. https://fedorahosted.org/pki/ticket/2449 -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-edewata-0830-Removed-support-for-creating-system-certificates-in-.patch Type: text/x-patch Size: 11354 bytes Desc: not available URL: From edewata at redhat.com Thu Sep 8 19:16:19 2016 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 8 Sep 2016 14:16:19 -0500 Subject: [Pki-devel] [PATCH] 830 Removed support for creating system certificates in different tokens. In-Reply-To: References: Message-ID: <1533f210-d143-9a30-03cf-7984e69d13a9@redhat.com> On 9/8/2016 2:13 PM, Endi Sukma Dewata wrote: > The patch that added the support for creating system certificates > in different tokens causes issues in certain cases, so for now it > has been reverted. > > https://fedorahosted.org/pki/ticket/2449 Pushed to master (10.4) under one-liner/trivial rule. -- Endi S. Dewata From mharmsen at redhat.com Tue Sep 13 04:09:30 2016 From: mharmsen at redhat.com (Matthew Harmsen) Date: Mon, 12 Sep 2016 22:09:30 -0600 Subject: [Pki-devel] Karma Requests for pki-core-10.3.5-6 In-Reply-To: <09cf941b-4dfa-f376-b775-ba5954f403dd@redhat.com> References: <09cf941b-4dfa-f376-b775-ba5954f403dd@redhat.com> Message-ID: <314a3af2-64b7-1160-f46c-7fa4488eac64@redhat.com> > *The following updated candidate builds of pki-core 10.3.5 on Fedora > 24, 25, and 26 (rawhide) consist of the following: > * > > * *Fedora 24* > o *pki-core-10.3.5-5.fc24 > > * > * *pki-core-10.3.5-6.fc24 * > o ** > * *Fedora 25* > o *pki-core-10.3.5-5.fc25 > * > o *pki-core-10.3.5-6.fc25 * > o ** > * *Fedora 26* > o *pki-core-10.3.5-5.fc26 > * > o *pki-core-10.3.5-6.fc26 ** * > *Additionally, the CentOS 7 COPR EPEL Builds of Dogtag 10.3.3 were > also updated:* > > * *https://copr.fedorainfracloud.org/coprs/g/pki/10.3.3/repo/epel-7/group_pki-10.3.3-epel-7.repo* > > > [group_pki-10.3.3] > name=Copr repo for 10.3.3 owned by @pki > baseurl=https://copr-be.cloud.fedoraproject.org/results/@pki/10.3.3/epel-7-$basearch/ > skip_if_unavailable=True > gpgcheck=1 > gpgkey=https://copr-be.cloud.fedoraproject.org/results/@pki/10.3.3/pubkey.gpg > enabled=1 > enabled_metadata=1 > > *These builds address the following PKI tickets: > * > > * PKI TRAC Ticket #1638 - Lightweight CAs: revoke certificate on CA > deletion > * PKI TRAC Ticket #2346 - Dogtag 10.3.6: Miscellaneous Enhancements > > * PKI TRAC Ticket #2443 - Prevent deletion of host CA's keys if LWCA > entry deleted > * PKI TRAC Ticket #2444 - Authority entry without entryUSN is > skipped even if USN plugin enabled > > * PKI TRAC Ticket #2446 - pkispawn: make subject_dn defaults unique > per instance name (for shared HSM) > > * PKI TRAC Ticket #2447 - CertRequestInfo has incorrect URLs > > * PKI TRAC Ticket #2449 - Unable to create system certificates in > different tokens > * *REVOKES PATCH FOR **PKI TRAC Ticket #2449 - Unable to create system certificates in different tokens * > *Please provide Karma for the following builds: > * > > * *Fedora 24* > o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-994f943797pki-core-10.3.5-5.fc24 > * > o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-7b06393ae4**pki-core-10.3.5-6.fc24* > * *Fedora 25* > o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-d363d36e22pki-core-10.3.5-5.fc25 > * > o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-734ba29899**pki-core-10.3.5-6.fc25** * -------------- next part -------------- An HTML attachment was scrubbed... URL: From ftweedal at redhat.com Wed Sep 14 12:14:41 2016 From: ftweedal at redhat.com (Fraser Tweedale) Date: Wed, 14 Sep 2016 22:14:41 +1000 Subject: [Pki-devel] [PATCH] 0134 Block reads during reload of LDAP-based profiles Message-ID: <20160914121441.GQ11489@dhcp-40-8.bne.redhat.com> Hi team, The attached patch fixes (yet another) race condition in LDAPProfileSubsystem. https://fedorahosted.org/pki/ticket/2453 Additional context: https://fedorahosted.org/freeipa/ticket/6274 Thanks, Fraser -------------- next part -------------- From 24a5ad6f84387055468e0125df90fea6635da484 Mon Sep 17 00:00:00 2001 From: Fraser Tweedale Date: Wed, 14 Sep 2016 19:39:36 +1000 Subject: [PATCH] Block reads during reload of LDAP-based profiles LDAP disconnect (e.g. due to DS restart) causes LDAPProfileSubsystem to drop all its profiles and reload them. If a profile is read during this time, e.g. to issue a certificate, it might not have been reloaded thus causing the operation to fail. Introduce the AsyncLoader class which allows a consumer to await the completion of a (re)load, if one is happening. Update the getProfile and getProfileIds method to use it. The existing 'initialLoadDone' CountDownLatch for blocking LDAPProfileSubsystem init until the inital load of profiles is completed was subsumed by AsyncLoader. Fixes: https://fedorahosted.org/pki/ticket/2453 --- .../src/com/netscape/certsrv/util/AsyncLoader.java | 86 ++++++++++++++++++++++ .../cmscore/profile/LDAPProfileSubsystem.java | 59 ++++++++++----- 2 files changed, 127 insertions(+), 18 deletions(-) create mode 100644 base/common/src/com/netscape/certsrv/util/AsyncLoader.java diff --git a/base/common/src/com/netscape/certsrv/util/AsyncLoader.java b/base/common/src/com/netscape/certsrv/util/AsyncLoader.java new file mode 100644 index 0000000000000000000000000000000000000000..39f8efd3272607ed6ac219b1b42bf9a4cb076a80 --- /dev/null +++ b/base/common/src/com/netscape/certsrv/util/AsyncLoader.java @@ -0,0 +1,86 @@ +// --- BEGIN COPYRIGHT BLOCK --- +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; version 2 of the License. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License along +// with this program; if not, write to the Free Software Foundation, Inc., +// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +// +// (C) 2016 Red Hat, Inc. +// All rights reserved. +// --- END COPYRIGHT BLOCK --- + +package com.netscape.certsrv.util; + +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.locks.ReentrantLock; + +/** A locking mechanism for loading or reloading an initially + * unknown number of items. + * + * The "producer" is the thread that loads items, informing the + * Loader when each item is loaded and how many items there are + * (when that fact becomes known). + * + * Other threads can await the completion of a (re)loading + * process. + */ +public class AsyncLoader { + private CountDownLatch producerInitialised = new CountDownLatch(1); + private ReentrantLock loadingLock = new ReentrantLock(); + private Integer numItems = null; + private int numItemsLoaded = 0; + + /** + * Acquire the lock as a producer. + */ + public void startLoading() { + numItems = null; + numItemsLoaded = 0; + loadingLock.lock(); + producerInitialised.countDown(); + } + + /** + * Increment the number of items loaded by 1. If the number + * of items is known and that many items have been loaded, + * unlock the loader. + */ + public void increment() { + numItemsLoaded += 1; + checkLoadDone(); + } + + /** + * Set the number of items. If the number of items already + * loaded is equal to or greater than the number, unlock the + * loader. + */ + public void setNumItems(Integer n) { + numItems = n; + checkLoadDone(); + } + + private void checkLoadDone() { + if (numItems != null && numItemsLoaded >= numItems) { + while (loadingLock.isHeldByCurrentThread()) + loadingLock.unlock(); + } + } + + public void awaitLoadDone() throws InterruptedException { + /* A consumer may await upon the Loader immediately after + * starting the producer. To ensure that the producer + * has time to acquire the lock, we use a CountDownLatch. + */ + producerInitialised.await(); + loadingLock.lock(); + loadingLock.unlock(); + } +} diff --git a/base/server/cmscore/src/com/netscape/cmscore/profile/LDAPProfileSubsystem.java b/base/server/cmscore/src/com/netscape/cmscore/profile/LDAPProfileSubsystem.java index 6dea1a0d88beaefeea489ea58ad9ad13d2da8bd7..fd5aa64eed8385ad18a307b6addaee6222d9f9cf 100644 --- a/base/server/cmscore/src/com/netscape/cmscore/profile/LDAPProfileSubsystem.java +++ b/base/server/cmscore/src/com/netscape/cmscore/profile/LDAPProfileSubsystem.java @@ -20,11 +20,11 @@ package com.netscape.cmscore.profile; import java.io.ByteArrayInputStream; import java.io.InputStream; import java.util.Arrays; +import java.util.Enumeration; import java.util.Hashtable; import java.util.LinkedHashMap; import java.util.TreeMap; import java.util.TreeSet; -import java.util.concurrent.CountDownLatch; import netscape.ldap.LDAPAttribute; import netscape.ldap.LDAPAttributeSet; @@ -49,6 +49,7 @@ import com.netscape.certsrv.profile.IProfile; import com.netscape.certsrv.profile.IProfileSubsystem; import com.netscape.certsrv.registry.IPluginInfo; import com.netscape.certsrv.registry.IPluginRegistry; +import com.netscape.certsrv.util.AsyncLoader; import com.netscape.cmscore.base.LDAPConfigStore; import com.netscape.cmsutil.ldap.LDAPUtil; @@ -71,10 +72,7 @@ public class LDAPProfileSubsystem /* Set of nsUniqueIds of deleted entries */ private TreeSet deletedNsUniqueIds; - /* Variables to track initial loading of profiles */ - private Integer initialNumProfiles = null; - private int numProfilesLoaded = 0; - private CountDownLatch initialLoadDone = new CountDownLatch(1); + private AsyncLoader loader = new AsyncLoader(); /** * Initializes this subsystem with the given configuration @@ -118,7 +116,7 @@ public class LDAPProfileSubsystem monitor = new Thread(this, "profileChangeMonitor"); monitor.start(); try { - initialLoadDone.await(); + loader.awaitLoadDone(); } catch (InterruptedException e) { CMS.debug("LDAPProfileSubsystem: caught InterruptedException " + "while waiting for initial load of profiles."); @@ -126,6 +124,27 @@ public class LDAPProfileSubsystem CMS.debug("LDAPProfileSubsystem: finished init"); } + public IProfile getProfile(String id) + throws EProfileException { + try { + loader.awaitLoadDone(); + } catch (InterruptedException e) { + CMS.debug("LDAPProfileSubsystem.getProfile: caught InterruptedException " + + "while waiting for profiles to be loaded."); + } + return super.getProfile(id); + } + + public Enumeration getProfileIds() { + try { + loader.awaitLoadDone(); + } catch (InterruptedException e) { + CMS.debug("LDAPProfileSubsystem.getProfile: caught InterruptedException " + + "while waiting for profiles to be loaded."); + } + return super.getProfileIds(); + } + /** * Read the given LDAPEntry into the profile subsystem. */ @@ -395,12 +414,6 @@ public class LDAPProfileSubsystem return "cn=" + id + "," + dn; } - private void checkInitialLoadDone() { - if (initialNumProfiles != null - && numProfilesLoaded >= initialNumProfiles) - initialLoadDone.countDown(); - } - private void ensureProfilesOU(LDAPConnection conn) throws LDAPException { try { conn.search(dn, LDAPConnection.SCOPE_BASE, "(objectclass=*)", null, false); @@ -431,7 +444,6 @@ public class LDAPProfileSubsystem CMS.debug("Profile change monitor: starting."); while (!stopped) { - forgetAllProfiles(); try { conn = dbFactory.getConn(); ensureProfilesOU(conn); @@ -443,16 +455,28 @@ public class LDAPProfileSubsystem LDAPSearchResults results = conn.search( dn, LDAPConnection.SCOPE_SUB, "(objectclass=*)", attrs, false, cons); + + /* Wait until the last possible moment before taking + * the load lock and dropping all profiles, so that + * we can continue to service requests while LDAP is + * down. + * + * Once we reconnect, we need to forget all profiles + * and reload in case some were removed in the + * interim. + */ + loader.startLoading(); + forgetAllProfiles(); + while (!stopped && results.hasMoreElements()) { LDAPEntry entry = results.next(); String[] objectClasses = entry.getAttribute("objectClass").getStringValueArray(); if (Arrays.asList(objectClasses).contains("organizationalUnit")) { - initialNumProfiles = new Integer( + loader.setNumItems(new Integer( entry.getAttribute("numSubordinates") - .getStringValueArray()[0]); - checkInitialLoadDone(); + .getStringValueArray()[0])); continue; } @@ -486,8 +510,7 @@ public class LDAPProfileSubsystem } else { CMS.debug("Profile change monitor: immediate result"); readProfile(entry); - numProfilesLoaded += 1; - checkInitialLoadDone(); + loader.increment(); } } } catch (ELdapException e) { -- 2.5.5 From edewata at redhat.com Thu Sep 15 00:16:32 2016 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 14 Sep 2016 19:16:32 -0500 Subject: [Pki-devel] [PATCH] 0134 Block reads during reload of LDAP-based profiles In-Reply-To: <20160914121441.GQ11489@dhcp-40-8.bne.redhat.com> References: <20160914121441.GQ11489@dhcp-40-8.bne.redhat.com> Message-ID: On 9/14/2016 7:14 AM, Fraser Tweedale wrote: > Hi team, > > The attached patch fixes (yet another) race condition in > LDAPProfileSubsystem. > > https://fedorahosted.org/pki/ticket/2453 > > Additional context: https://fedorahosted.org/freeipa/ticket/6274 > > Thanks, > Fraser The patch looks fine, but probably it can be simplified like this: class LDAPProfileSubsystem { void init() { // load initial profiles repository = new LDAPProfileRepository(); repository.load(); // monitor profile changes in the background monitor = new Thread(repository); monitor.start(); } IProfile getProfile(id) { return repository.getProfile(id); } } class LDAPProfileRepository { LinkedHashMap profiles = ... void synchronized load() { // create persistent search conn = dbFactory.getConn(); results = conn.search(...); // get number of profiles entry = results.next(); numProfiles = entry.getAttribute("numSubordinates"); for (i=0; i References: <20160914121441.GQ11489@dhcp-40-8.bne.redhat.com> Message-ID: <20160915053939.GW11489@dhcp-40-8.bne.redhat.com> On Wed, Sep 14, 2016 at 07:16:32PM -0500, Endi Sukma Dewata wrote: > On 9/14/2016 7:14 AM, Fraser Tweedale wrote: > > Hi team, > > > > The attached patch fixes (yet another) race condition in > > LDAPProfileSubsystem. > > > > https://fedorahosted.org/pki/ticket/2453 > > > > Additional context: https://fedorahosted.org/freeipa/ticket/6274 > > > > Thanks, > > Fraser > > The patch looks fine, but probably it can be simplified like this: > > class LDAPProfileSubsystem { > > void init() { > > // load initial profiles > repository = new LDAPProfileRepository(); > repository.load(); > > // monitor profile changes in the background > monitor = new Thread(repository); > monitor.start(); > } > > IProfile getProfile(id) { > return repository.getProfile(id); > } > } > > class LDAPProfileRepository { > > LinkedHashMap profiles = ... > > void synchronized load() { > > // create persistent search > conn = dbFactory.getConn(); > results = conn.search(...); > > // get number of profiles > entry = results.next(); > numProfiles = entry.getAttribute("numSubordinates"); > > for (i=0; i // read profile > entry = results.next(); > readProfile(entry); > } > } > > void synchronized readProfile() { > ... > } > > IProfile synchronized getProfile(id) { > return profiles.get(id); > } > > void run() { > > while (true) { > try { > // process profile changes > while (results.hasMoreElements()) { > entry = results.next(); > ... > } > } catch (...) { > // reconnect > load(); > } > } > } > } > > So the load() will block during initialization and will also block readers > during reload after reconnect. We probably can replace "synchronized" with > ReadWriteLock to allow concurrent readers. > Yep, that's a good approach. > Feel free to push the patch as is (assuming it's well tested). We can make > further improvements later on. > > One thing though, I highly suggest that we fix this issue on both Fedora and > RHEL/CentOS platforms. The patch is non-trivial, so the behavior could be > different if not applied consistently. Since PKI is developed mainly on > Fedora but used on different platforms, it would be much easier to > troubleshoot issues by keeping the behavior consistent across platforms, > especially on anything related to concurrency. > > We don't need to create new builds for all platforms at the same time, but > we should at least push this patch to all 10.3 branches so it can be picked > up in the next 10.3 build of the corresponding platform. > The patch is (at this stage) not destined for 10.3 at all. I'd prefer to push it to master to be included in Fedora when 10.4 gets released, and other platforms' builds whenever they rebase. I might go ahead and implement your suggested change before merging, too, although probably as a second patch. Thanks for reviewing! Cheers, Fraser From edewata at redhat.com Tue Sep 20 00:30:54 2016 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 19 Sep 2016 19:30:54 -0500 Subject: [Pki-devel] [PATCH] 831 Troubleshooting improvements for SigningUnit. Message-ID: <0e23b9ac-eee6-f975-21e7-73045185c3c9@redhat.com> To help troubleshooting the SigningUnit for CA and OCSP have been modified to chain the original exceptions. https://fedorahosted.org/pki/ticket/2463 Pushed master (10.4) under one-line/trivial rule. -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-edewata-0831-Troubleshooting-improvements-for-SigningUnit.patch Type: text/x-patch Size: 13655 bytes Desc: not available URL: From edewata at redhat.com Tue Sep 20 00:31:01 2016 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 19 Sep 2016 19:31:01 -0500 Subject: [Pki-devel] [PATCH] 832 Troubleshooting improvements for ConfigurationUtils. Message-ID: <79469084-5f8b-1e99-95b1-b8423955511c@redhat.com> To help troubleshooting the ConfigurationUtils has been modified to chain the original exceptions and to show additional log messages. https://fedorahosted.org/pki/ticket/2463 Pushed to master (10.4) under one-line/trivial rule. -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-edewata-0832-Troubleshooting-improvements-for-ConfigurationUtils.patch Type: text/x-patch Size: 4241 bytes Desc: not available URL: From edewata at redhat.com Wed Sep 21 15:26:48 2016 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 21 Sep 2016 10:26:48 -0500 Subject: [Pki-devel] [PATCH] 833 Additional improvements for SigningUnit. Message-ID: <8cebbbd2-902d-c447-7f9e-5aa2e187e11f@redhat.com> To help troubleshooting the SigningUnit for CA have been modified to show additional log messages. https://fedorahosted.org/pki/ticket/2463 Pushed to master (10.4) under one-liner/trivial rule. -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-edewata-0833-Additional-improvements-for-SigningUnit.patch Type: text/x-patch Size: 1711 bytes Desc: not available URL: From edewata at redhat.com Wed Sep 21 15:37:35 2016 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 21 Sep 2016 10:37:35 -0500 Subject: [Pki-devel] [PATCH] 834 Updated PKI server logging service. Message-ID: The PKI server logging service has been modified to utilize Java Logging API. This will allow PKI code to be refactored into smaller modules which can run outside the server (e.g. for unit testing). The common logging.properties file has been updated to define the debug log handlers for each subsystem. New logging.properties files have been added for each subsystem to specify the PKI packages to be logged. The deployment tool has been modified to create a link to the default logging.properties instead of creating a copy. The unused log4j.properties has been removed. The pki.policy has been modified to allow Tomcat to read the default logging.properties files in /usr/share/pki and to generate debug logs in subfolders under /logs. https://fedorahosted.org/pki/ticket/195 -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-edewata-0834-Updated-PKI-server-logging-service.patch Type: text/x-patch Size: 22931 bytes Desc: not available URL: From ftweedal at redhat.com Thu Sep 22 02:13:48 2016 From: ftweedal at redhat.com (Fraser Tweedale) Date: Thu, 22 Sep 2016 12:13:48 +1000 Subject: [Pki-devel] [PATCH] 0135 Do not attempt LWCA key retrieval for host authority Message-ID: <20160922021347.GU11489@dhcp-40-8.bne.redhat.com> Hi team, Please review the attached patch which fixes a regression in two-step externally-signed CA installation. It is destined for 10.3 branch as well as master. https://fedorahosted.org/pki/ticket/2466 Cheers, Fraser -------------- next part -------------- From fca5fd053434d112998c814bc6d9424b6a5bac98 Mon Sep 17 00:00:00 2001 From: Fraser Tweedale Date: Wed, 21 Sep 2016 20:18:37 +1000 Subject: [PATCH] Do not attempt LWCA key retrieval for host authority During two-step installation of externally-signed CA, installation can fail because host authority's private key cannot be located (a temporary condition), causing LWCA key replication to fire, which throws NullPointerException because the host authority's AuthorityID has not been set yet. Do not start key retrieval if the CA's AuthorityID is null (a condition which implies that the CA is the host authority). Fixes: https://fedorahosted.org/pki/ticket/2466 --- base/ca/src/com/netscape/ca/CertificateAuthority.java | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/base/ca/src/com/netscape/ca/CertificateAuthority.java b/base/ca/src/com/netscape/ca/CertificateAuthority.java index 1f77fd81fc850af9996329dbec7d6a973ba62942..a4f102435ae7a1f2ab1e27814a52b5689639d0f7 100644 --- a/base/ca/src/com/netscape/ca/CertificateAuthority.java +++ b/base/ca/src/com/netscape/ca/CertificateAuthority.java @@ -1569,7 +1569,12 @@ public class CertificateAuthority CMS.debug("CA signing key and cert not (yet) present in NSSDB"); signingUnitException = e; if (retrieveKeys == true) { - if (!keyRetrieverThreads.containsKey(authorityID)) { + if (authorityID == null) { + // Only the host authority should ever see a + // null authorityID, e.g. during two-step + // installation of externally-signed CA. + CMS.debug("null authorityID -> host authority; not starting KeyRetriever"); + } else if (!keyRetrieverThreads.containsKey(authorityID)) { CMS.debug("Starting KeyRetrieverRunner thread"); Thread t = new Thread( new KeyRetrieverRunner(authorityID, mNickname, authorityKeyHosts), -- 2.5.5 From ftweedal at redhat.com Thu Sep 22 02:20:19 2016 From: ftweedal at redhat.com (Fraser Tweedale) Date: Thu, 22 Sep 2016 12:20:19 +1000 Subject: [Pki-devel] [PATCH] 0136 Compare serialised DNs in host authority check Message-ID: <20160922022019.GV11489@dhcp-40-8.bne.redhat.com> Hi team, The attached patch fixes a bug in lightweight CAs' host authority detection, when CA cert Subject DN contains PrintableString-encoded attributes. https://fedorahosted.org/pki/ticket/2475 Thanks, Fraser -------------- next part -------------- From 6afdc9944cc147f9d4aab2d5274eaa4dd3fe9243 Mon Sep 17 00:00:00 2001 From: Fraser Tweedale Date: Thu, 22 Sep 2016 12:00:35 +1000 Subject: [PATCH] Compare serialised DNs in host authority check CA startup creates an LWCA entry for the host authority if it determines that one has not already been created. It determines if an LWCA entry corresponds to the host CA by comparing the DN from LDAP with the DN from the host authority's certificate. If the DN from the host authority's certificate contains values encoded as PrintableString, it will compare unequal to the DN from LDAP, which parses to UTF8String AVA values. This causes the addition of a spurious host authority entry every time the server starts. Serialise DNs before comparing, to avoid these false negatives. Fixes: https://fedorahosted.org/pki/ticket/2475 --- base/ca/src/com/netscape/ca/CertificateAuthority.java | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/base/ca/src/com/netscape/ca/CertificateAuthority.java b/base/ca/src/com/netscape/ca/CertificateAuthority.java index 1f77fd81fc850af9996329dbec7d6a973ba62942..6b504f58c142f416392c190a3b9574854280fcfe 100644 --- a/base/ca/src/com/netscape/ca/CertificateAuthority.java +++ b/base/ca/src/com/netscape/ca/CertificateAuthority.java @@ -3251,7 +3251,12 @@ public class CertificateAuthority if (descAttr != null) desc = (String) descAttr.getStringValues().nextElement(); - if (dn.equals(mName)) { + /* Determine if it is the host authority's entry, by + * comparing DNs. DNs must be serialised in case different + * encodings are used for AVA values, e.g. PrintableString + * from LDAP vs UTF8String in certificate. + */ + if (dn.toString().equals(mName.toString())) { CMS.debug("Found host authority"); foundHostAuthority = true; this.authorityID = aid; -- 2.5.5 From alee at redhat.com Thu Sep 22 04:12:05 2016 From: alee at redhat.com (Ade Lee) Date: Thu, 22 Sep 2016 00:12:05 -0400 Subject: [Pki-devel] [PATCH] 0135 Do not attempt LWCA key retrieval for host authority In-Reply-To: <20160922021347.GU11489@dhcp-40-8.bne.redhat.com> References: <20160922021347.GU11489@dhcp-40-8.bne.redhat.com> Message-ID: <1474517525.1328.48.camel@redhat.com> ACK On Thu, 2016-09-22 at 12:13 +1000, Fraser Tweedale wrote: > Hi team, > > Please review the attached patch which fixes a regression in > two-step externally-signed CA installation. It is destined for 10.3 > branch as well as master. > > https://fedorahosted.org/pki/ticket/2466 > > Cheers, > Fraser > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From alee at redhat.com Thu Sep 22 04:13:09 2016 From: alee at redhat.com (Ade Lee) Date: Thu, 22 Sep 2016 00:13:09 -0400 Subject: [Pki-devel] [PATCH] 0136 Compare serialised DNs in host authority check In-Reply-To: <20160922022019.GV11489@dhcp-40-8.bne.redhat.com> References: <20160922022019.GV11489@dhcp-40-8.bne.redhat.com> Message-ID: <1474517589.1328.49.camel@redhat.com> ACK On Thu, 2016-09-22 at 12:20 +1000, Fraser Tweedale wrote: > Hi team, > > The attached patch fixes a bug in lightweight CAs' host authority > detection, when CA cert Subject DN contains PrintableString-encoded > attributes. > > https://fedorahosted.org/pki/ticket/2475 > > Thanks, > Fraser > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From mharmsen at redhat.com Thu Sep 22 21:07:24 2016 From: mharmsen at redhat.com (Matthew Harmsen) Date: Thu, 22 Sep 2016 15:07:24 -0600 Subject: [Pki-devel] Announcing: External COPR Builds of CentOS 7 PKI EPEL Packages Message-ID: Everyone, The Dogtag PKI team is proud to announce the availability of new external COPR builds of PKI EPEL packages for the following two platforms: * CentOS 7.2: o https://copr.fedorainfracloud.org/coprs/g/pki/epel-7.2/repo/epel-7/group_pki-epel-7.2-epel-7.repo * CentOS 7.3: o https://copr.fedorainfracloud.org/coprs/g/pki/epel-7.3/repo/epel-7/group_pki-epel-7.3-epel-7.repo The CentOS 7.2 builds are based upon the Dogtag 10.2.6 release, while the CentOS 7.3 builds are based upon the Dogtag 10.3.3 release. Details on obtaining and using these new builds are available on the following Dogtag Wiki page: * External COPR Builds of CentOS PKI EPEL Packages Enjoy, Dogtag Team -------------- next part -------------- An HTML attachment was scrubbed... URL: From ftweedal at redhat.com Fri Sep 23 03:39:33 2016 From: ftweedal at redhat.com (Fraser Tweedale) Date: Fri, 23 Sep 2016 13:39:33 +1000 Subject: [Pki-devel] [PATCH] 0135 Do not attempt LWCA key retrieval for host authority In-Reply-To: <1474517525.1328.48.camel@redhat.com> References: <20160922021347.GU11489@dhcp-40-8.bne.redhat.com> <1474517525.1328.48.camel@redhat.com> Message-ID: <20160923033933.GA11489@dhcp-40-8.bne.redhat.com> On Thu, Sep 22, 2016 at 12:12:05AM -0400, Ade Lee wrote: > > ACK > Thanks! Pushed to... - master (3ea93c9b4bc03f3d79550d8bdfd1447ffa25238d) - DOGTAG_10_3_BRANCH (fca5fd053434d112998c814bc6d9424b6a5bac98) Cheers, Fraser > On Thu, 2016-09-22 at 12:13 +1000, Fraser Tweedale wrote: > > Hi team, > > > > Please review the attached patch which fixes a regression in > > two-step externally-signed CA installation. It is destined for 10.3 > > branch as well as master. > > > > https://fedorahosted.org/pki/ticket/2466 > > > > Cheers, > > Fraser > > _______________________________________________ > > Pki-devel mailing list > > Pki-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/pki-devel From ftweedal at redhat.com Fri Sep 23 03:40:09 2016 From: ftweedal at redhat.com (Fraser Tweedale) Date: Fri, 23 Sep 2016 13:40:09 +1000 Subject: [Pki-devel] [PATCH] 0136 Compare serialised DNs in host authority check In-Reply-To: <1474517589.1328.49.camel@redhat.com> References: <20160922022019.GV11489@dhcp-40-8.bne.redhat.com> <1474517589.1328.49.camel@redhat.com> Message-ID: <20160923034009.GB11489@dhcp-40-8.bne.redhat.com> On Thu, Sep 22, 2016 at 12:13:09AM -0400, Ade Lee wrote: > ACK > Thanks! Pushed to... - master (9043a08bef3723ca218ad7e5dd82be61166b5a1d) - DOGTAG_10_3_BRANCH (84606cc69390187b7f0f11fff41a372fd96f8f93) Cheers, Fraser > On Thu, 2016-09-22 at 12:20 +1000, Fraser Tweedale wrote: > > Hi team, > > > > The attached patch fixes a bug in lightweight CAs' host authority > > detection, when CA cert Subject DN contains PrintableString-encoded > > attributes. > > > > https://fedorahosted.org/pki/ticket/2475 > > > > Thanks, > > Fraser > > _______________________________________________ > > Pki-devel mailing list > > Pki-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/pki-devel From ftweedal at redhat.com Fri Sep 23 03:49:48 2016 From: ftweedal at redhat.com (Fraser Tweedale) Date: Fri, 23 Sep 2016 13:49:48 +1000 Subject: [Pki-devel] [PATCH] 0134 Block reads during reload of LDAP-based profiles In-Reply-To: <20160915053939.GW11489@dhcp-40-8.bne.redhat.com> References: <20160914121441.GQ11489@dhcp-40-8.bne.redhat.com> <20160915053939.GW11489@dhcp-40-8.bne.redhat.com> Message-ID: <20160923034948.GC11489@dhcp-40-8.bne.redhat.com> On Thu, Sep 15, 2016 at 03:39:39PM +1000, Fraser Tweedale wrote: > On Wed, Sep 14, 2016 at 07:16:32PM -0500, Endi Sukma Dewata wrote: > > On 9/14/2016 7:14 AM, Fraser Tweedale wrote: > > > Hi team, > > > > > > The attached patch fixes (yet another) race condition in > > > LDAPProfileSubsystem. > > > > > > https://fedorahosted.org/pki/ticket/2453 > > > > > > Additional context: https://fedorahosted.org/freeipa/ticket/6274 > > > > > > Thanks, > > > Fraser > > > > The patch looks fine, but probably it can be simplified like this: > > > > class LDAPProfileSubsystem { > > > > void init() { > > > > // load initial profiles > > repository = new LDAPProfileRepository(); > > repository.load(); > > > > // monitor profile changes in the background > > monitor = new Thread(repository); > > monitor.start(); > > } > > > > IProfile getProfile(id) { > > return repository.getProfile(id); > > } > > } > > > > class LDAPProfileRepository { > > > > LinkedHashMap profiles = ... > > > > void synchronized load() { > > > > // create persistent search > > conn = dbFactory.getConn(); > > results = conn.search(...); > > > > // get number of profiles > > entry = results.next(); > > numProfiles = entry.getAttribute("numSubordinates"); > > > > for (i=0; i > // read profile > > entry = results.next(); > > readProfile(entry); > > } > > } > > > > void synchronized readProfile() { > > ... > > } > > > > IProfile synchronized getProfile(id) { > > return profiles.get(id); > > } > > > > void run() { > > > > while (true) { > > try { > > // process profile changes > > while (results.hasMoreElements()) { > > entry = results.next(); > > ... > > } > > } catch (...) { > > // reconnect > > load(); > > } > > } > > } > > } > > > > So the load() will block during initialization and will also block readers > > during reload after reconnect. We probably can replace "synchronized" with > > ReadWriteLock to allow concurrent readers. > > > Yep, that's a good approach. > > > Feel free to push the patch as is (assuming it's well tested). We can make > > further improvements later on. > > > > One thing though, I highly suggest that we fix this issue on both Fedora and > > RHEL/CentOS platforms. The patch is non-trivial, so the behavior could be > > different if not applied consistently. Since PKI is developed mainly on > > Fedora but used on different platforms, it would be much easier to > > troubleshoot issues by keeping the behavior consistent across platforms, > > especially on anything related to concurrency. > > > > We don't need to create new builds for all platforms at the same time, but > > we should at least push this patch to all 10.3 branches so it can be picked > > up in the next 10.3 build of the corresponding platform. > > > The patch is (at this stage) not destined for 10.3 at all. I'd > prefer to push it to master to be included in Fedora when 10.4 gets > released, and other platforms' builds whenever they rebase. > > I might go ahead and implement your suggested change before merging, > too, although probably as a second patch. > On further investigation, the suggested approach will require either moving a lot of logic out of the AbstractProfileSubsystem base class or signficiant rework to make all profile subsystem implementations (LDAP and File) use a 'ProfileRepository' concept. Might be a good refactoring candidate for future. Original patch pushed to master (ced5cb71c1963d5234c2360d1f2ac11d4a452d9d) Cheers, Fraser From edewata at redhat.com Sat Sep 24 03:02:28 2016 From: edewata at redhat.com (Endi Sukma Dewata) Date: Fri, 23 Sep 2016 22:02:28 -0500 Subject: [Pki-devel] Replacing PKI logging framework Message-ID: <578c948e-80ca-6a93-ce1d-9bf36c52cb9c@redhat.com> Hi, This is a preliminary work for PKI 10.4. In ticket #195 we are planning to replace the current logging framework. This is needed to simplify writing unit tests and also provide debug log rotation. Here is the initial investigation to determine which framework to use: http://pki.fedoraproject.org/wiki/Logging_Frameworks I've also posted patch #834 that shows how it will be implemented. Please let me know if you have any comment/question. Thanks. -- Endi S. Dewata