[Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18
Ade Lee
alee at redhat.com
Wed Sep 12 02:42:26 UTC 2012
On Tue, 2012-09-11 at 14:45 -0400, Rob Crittenden wrote:
> Petr Viktorin wrote:
> > On 09/11/2012 04:38 PM, Rob Crittenden wrote:
> >> Ade Lee wrote:
> >>> On Tue, 2012-09-11 at 08:59 -0400, Rob Crittenden wrote:
> >>>> Petr Viktorin wrote:
> >>>>> On 09/11/2012 04:04 AM, Ade Lee wrote:
> >>>>>> On Mon, 2012-09-10 at 16:58 -0400, Rob Crittenden wrote:
> >>>>>>> Petr Viktorin wrote:
> >>>>>>>> Attaching rebased and squashed patches. I've done some testing with
> >>>>>>>> them
> >>>>>>>> but please test some more.
> >>>>>>>>
> >>>>>>>
> >>>>>>> Most of these aren't IPA issues, but dogtag issues. I'll try to
> >>>>>>> split
> >>>>>>> them out.
> >>>>>>>
> >>>>>>> IPA:
> >>>>>>>
> >>>>>>> For the configuration files in install/conf to be updated at rpm
> >>>>>>> update
> >>>>>>> time the VERSION needs to be incremented.
> >>>>>
> >>>>> These files should stay the same since on upgrade we're still using a
> >>>>> Dogtag 9 style instance. The Dogtag 10 ports are only used in new
> >>>>> installs.
> >>>>>
> >>>>>>> The ipa package lacks any updated dogtag dependencies, so I abused
> >>>>>>> it.
> >>>>>
> >>>>> What should the updated dependencies be? Since it should work with
> >>>>> both
> >>>>> dogtag 9 and 10, I don't see how they should change.
> >>>>
> >>>> I don't know either, but we need to prevent people from installing
> >>>> incompatible package combinations.
> >>>>
> >>> Would'nt the Conflicts: ipa < 3.0 in pki-ca mentioned below satisfy this
> >>> requirement? The main concern is that you must have ipa 3.0 if you have
> >>> dogtag 10.
> >>>
> >>> Given that dogtag is consumed by IPA though, it makes more sense to put
> >>> the relevant conflicts in IPA rather than in dogtag. So in this case,
> >>> that would mean putting Conflicts: pki-ca >= 10.0 in IPA 2.x.
> >>> Recall that dogtag 10 will only be officially available in f18+.
> >>
> >> That isn't enough. If a F-17 user with IPA 2.2 installed upgrades to
> >> F-18 they would be able to install dogtag 10 and blow up their IPA
> >> server.
> >>
We can add the Conflicts: freeipa-server < 3.0 to the dogtag packages
(likely in pki-base).
But we should also add explicit dependencies to ipa.
For ipa 2.2, Conflicts: pki-ca >= 10.0, Requires: pki-ca >= 9.x
For ipa 3, Requires: pki-ca >= 10.0
This is of course assumes that ipa 3 is only officially released on f18
(which is what will happen for dogtag 10). Just because we can support
d9 on ipa 3 does not mean we should.
As it is, in this case, we will have to support IPA 3 + d10, IPA 3 + d10
+ d9-style instance, IPA 2.x + d9.
> >>>
> >>>>>>> I installed IPA with dogtag 9 and created a replica.
> >>>>>>>
> >>>>>>> I updated the IPA bits, that worked fine.
> >>>>>>>
> >>>>>>> I updated to dogtag 10 and now the CA doesn't work on the master,
> >>>>>>> including starting the dogtag instance. Note that the rpm update
> >>>>>>> process
> >>>>>>> worked, no notice that the CA service didn't restart.
> >>>>>>>
> >>>>>> Did you try to restart the CA with selinux in permissive mode?
> >>>>>> This is
> >>>>>> still required right now until I get the selinux policy all
> >>>>>> straightened
> >>>>>> out.
> >>>>>>
> >>>>>> There is also a separate dogtag ticket (which is currently being
> >>>>>> worked
> >>>>>> on) to restart existing dogtag instances when dogtag is upgraded from
> >>>>>> 9->10.
> >>>>>
> >>>>> In permissive mode, this upgrade works for me.
> >>>>
> >>>> I was in enforcing mode but saw no AVCs. What is the ETA on fixing
> >>>> this?
> >>>>
> >>>
> >>> Within the next week or two, I need to finish the IPA merge database
> >>> patch first, and then co-ordinate changes with the selinux guys.
> >>>
> >>>>>
> >>>>>
> >>>>> Sometimes I do get this error intermittently:
> >>>>>
> >>>>> ipa: ERROR: Certificate operation cannot be completed: Unable to
> >>>>> communicate with CMS (Service Temporarily Unavailable)
> >>>>>
> >>>>> Usually, waiting a couple of minutes clears this up. Perhaps we are
> >>>>> not
> >>>>> doing startup detection correctly. Ade mentioned that waiting for
> >>>>> ports
> >>>>> may not be ideal. How do we know if Dogtag is initialized?
> >>>>
> >>>> Years ago we had discussed with Andrew and Matt creating a URI that can
> >>>> be queried to determine dogtag status. I don't think that ever went
> >>>> anywhere.
> >>>>
> >>> Petr, this happens only on reboot, right? And not on regular "service
> >>> ipa restart"?
> >
> > I've now seen it happen right after a 9 → 10 upgrade.
> >
> >>> Yeah, I remember this conversation - and even created a bug for it at
> >>> some point. This went away because the mechanism you were using seemed
> >>> to be working. The timing may be very different now with tomcat 7 and
> >>> systemd. I'll open a dogtag trac ticket for this.
> >>
> >> Ok.
> >>
> >>>
> >>>>>
> >>>>>>> Uninstalling failed because it tried to run pkidestroy and not
> >>>>>>> pkiremove.
> >>>>>
> >>>>> I was under the impression that pkidestroy was the correct command to
> >>>>> remove an upgraded instance. I'll check with Ade.
> >>>>>
> >>>>>> I'll test this too.
> >>>>>>
> >>>>>>> The contents of the file passed to pkispawn should be logged so we
> >>>>>>> can
> >>>>>>> see exactly what was passed in.
> >>>>>>>
> >>>>>> Its a pretty big file. You might want to only log the modifications.
> >>>>>> Or save the file somewhere.
> >>>>>
> >>>>> Our logs are pretty verbose, so that shouldn't be a problem. I'll
> >>>>> put it
> >>>>> in the next version of the patch.
> >>>>
> >>>> The question to ask is: would you need the contents of this file if all
> >>>> you got were logs and needed to evaluate why installation failed? In
> >>>> most cases this is yes.
> >>>>
> >>> Up to you guys. There is a patch I am working on in which I will be
> >>> logging the object that is being passed to the server from pkispawn.
> >>> That - and the diffs to the standard config file as I mentioned above -
> >>> will likely be sufficient to debug almost all cases.
> >>>
> >>> Make sure not to log any passwords.
> >>>
> >
> > Thanks for the catch. Attaching updated patch that sanitizes the passwords.
> >
> >>>>>>> DOGTAG:
> >>>>>>>
> >>>>>>> When upgrading using the dogtag-devel repo I had to specify
> >>>>>>> pki-tools.x86_64 otherwise it tried to install both 32 and 64-bit
> >>>>>>> versions (and failed).
> >>>>>>>
> >>>>>>> I ended up running: yum update pki-ca tomcatjss pki-tools.x86_64
> >>>>>>> --enablerepo=dogtag-devel --enablerepo=updates-testing
> >>>>>>>
> >>>>>> We'll look into this. I think I know why this happens.
> >>>>>>
> >>>>>>> What happens if someone manually upgrades pki-ca without first
> >>>>>>> updating
> >>>>>>> ipa? I think that pki-ca is going to need a Conflicts ipa < 3.0 in
> >>>>>>> it.
> >>>>>>
> >>>>>> We can add that.
> >>>>>>
> >>>>>>> certificate renewal failed. I spent far too long trying to figure
> >>>>>>> out
> >>>>>>> why tomcat wasn't listening on port 9180 but failed. I think 9180 is
> >>>>>>> actually the old server, right? So another missing dependency on a
> >>>>>>> fixed
> >>>>>>> certmonger?
> >>>>>>>
> >>>>>>> The best I could find was the certmonger error:
> >>>>>>>
> >>>>>>> ca-error: Error 7 connecting to
> >>>>>>> http://edsel.example.com:9180/ca/ee/ca/profileSubmit: Couldn't
> >>>>>>> connect
> >>>>>>> to server.
> >>>>>>>
> >>>>>
> >>>>> I'll set the certmonger min version to 0.60 as per Nalin's mail.
> >>>>
> >>>> Ok.
> >>>>
> >>>>>> Is this cert renewal on a dogtag 10 instance? Or the upgraded
> >>>>>> dogtag 9?
> >>>>>> If its dogtag 10, perhaps you do not have the certmonger version that
> >>>>>> has the relevant fix? If its dogtag 9, then we need to figure out
> >>>>>> whats
> >>>>>> going on. That reminds me - I need to file a bug to allow
> >>>>>> certmonger to
> >>>>>> talk to the newly defined dogtag ports. Do you have selinux
> >>>>>> permissive?
> >>>>>>
> >>>>>>> There is no man page for pkispawn/pkidestroy :-( According to the
> >>>>>>> FHS
> >>>>>>> these should not be in /bin but in /usr/sbin (not end-user
> >>>>>>> commands).
> >>>>>>>
> >>>>>> There is a trac ticket open for man pages for pkispawn and
> >>>>>> pkidestroy.
> >>>>>> We plan to complete this ticket by the time f18 is released.
> >>>>>>
> >>>>>> We'll look into the location of pkispawn/pkicreate.
> >>>>>>
> >>>>>>> The output of pkicreate/pkisilent was really terrible and not
> >>>>>>> usable at
> >>>>>>> all so we didn't display it when failures occurred. It looks like
> >>>>>>> that
> >>>>>>> has been addressed, at least for the case where a CA is already
> >>>>>>> configured and you try to install IPA. Perhaps we should capture
> >>>>>>> stderr
> >>>>>>> and display that instead of the command-line of pkispawn? Again, a
> >>>>>>> man
> >>>>>>> page would help with the integration.
> >>>>>>>
> >>>>>>> 2012-09-10T20:51:45Z DEBUG [2/18]: configuring certificate server
> >>>>>>> instance
> >>>>>>> 2012-09-10T20:51:45Z DEBUG args=/bin/pkispawn -s CA -f
> >>>>>>> /tmp/tmp_Urraq
> >>>>>>> 2012-09-10T20:51:45Z DEBUG stdout=
> >>>>>>> 2012-09-10T20:51:45Z DEBUG stderr=pkispawn : ERROR ....... PKI
> >>>>>>> subsystem 'CA' for instance 'pki-tomcat' already exists!
> >>>>>>>
> >>>>>>> 2012-09-10T20:51:45Z CRITICAL failed to configure ca instance
> >>>>>>> Command
> >>>>>>> '/bin/pkispawn -s CA -f /tmp/tmp_Urraq' returned non-zero exit
> >>>>>>> status 1
> >>>>>>>
> >>>>>> That may be a good idea. Of course. thats an IPA thing, right?
> >>>>
> >>>> Right, not a show-stopper for this but a new ticket should be opened to
> >>>> track it.
> >
> > https://fedorahosted.org/freeipa/ticket/3072
>
> Thanks.
>
> I tested a 2.2 -> 3.0 upgrade with dogtag 9 and it seems to have failed
> to restart something:
>
> [ post rpm -Uvh dist/rpms/freeipa*.rpm ]
>
> [root at edsel freeipa]# ipa cert-show 1
> ipa: ERROR: Certificate operation cannot be completed: Unable to
> communicate with CMS (Service Temporarily Unavailable)
> [root at edsel freeipa]# ipactl restart
> Restarting Directory Service
> Restarting KDC Service
> Restarting KPASSWD Service
> Restarting MEMCACHE Service
> Restarting HTTP Service
> Restarting CA Service
> [root at edsel freeipa]# ipa cert-show 1
> Certificate: MIIDjTCCAnWgAwIBAgIBATANBgk
> ...
>
> The apache log had this:
>
> [Tue Sep 11 14:35:42 2012] [error] (111)Connection refused: proxy: AJP:
> attempt to connect to 127.0.0.1:9447 (localhost) failed
> [Tue Sep 11 14:35:42 2012] [error] ap_proxy_connect_backend disabling
> worker for (localhost)
> [Tue Sep 11 14:35:42 2012] [error] proxy: AJP: failed to make connection
> to backend: localhost
>
> So I'm gathering that dogtag didn't restart properly, but I'm just
> guessing. It could have been the PKI-IPA ds instance too, I'm not sure
> where to check.
>
> I also noticed this in /var/log/ipaupgrade.log:
>
> 2012-09-11T18:28:22Z DEBUG args=/bin/systemctl start certmonger.service
> 2012-09-11T18:28:22Z DEBUG stdout=
> 2012-09-11T18:28:22Z DEBUG stderr=
> 2012-09-11T18:28:22Z DEBUG args=/bin/systemctl is-active certmonger.service
> 2012-09-11T18:28:22Z DEBUG stdout=active
> ...
> ... [ snip certutil output ]
> ...
> 2012-09-11T18:28:52Z DEBUG args=/usr/bin/getcert start-tracking -d
> /var/lib/pki-ca/alias -n auditSigningCert cert-pki-ca -c
> dogtag-ipa-renew-agent -C /usr/lib64/ipa/certmonger/renew_ca_cert
> "auditSigningCert cert-pki-ca" -P XXXXXXXX
> 2012-09-11T18:28:52Z DEBUG stdout=Please verify that the certmonger
> service is still running.
>
> 2012-09-11T18:28:52Z DEBUG stderr=
> 2012-09-11T18:28:52Z INFO File
> "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py",
> line 614, in run_script
> return_value = main_function()
>
> File "/usr/sbin/ipa-upgradeconfig", line 540, in main
> enable_certificate_renewal(api.env.realm)
>
> File "/usr/sbin/ipa-upgradeconfig", line 455, in
> enable_certificate_renewal
> ca.configure_renewal()
>
> File
> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line
> 1298, in configure_renewal
> self.dogtag_constants.ALIAS_DIR, 'renew_ca_cert "%s"' % nickname)
>
> File "/usr/lib/python2.7/site-packages/ipapython/certmonger.py", line
> 394, in dogtag_start_tracking
> (stdout, stderr, returncode) = ipautil.run(args, nolog=[pin])
>
> File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line
> 309, in run
> raise CalledProcessError(p.returncode, args)
>
> 2012-09-11T18:28:52Z INFO The ipa-upgradeconfig command failed,
> exception: CalledProcessError: Command '/usr/bin/getcert start-tracking
> -d /var/lib/pki-ca/alias -n auditSigningCert cert-pki-ca -c
> dogtag-ipa-renew-agent -C /usr/lib64/ipa/certmonger/renew_ca_cert
> "auditSigningCert cert-pki-ca" -P XXXXXXXX' returned non-zero exit status 1
>
> I'm not sure if this is related to this patch or not. If it isn't can
> you file a new ticket on it, or add it the 2.2 -> 3.0 upgrade ticket?
>
> rob
>
More information about the Freeipa-devel
mailing list