[Freeipa-users] RHEL 7 Upgrade experience so far

[Rob Crittenden]

Erinn Looney-Triggs wrote:
> On 07/27/2014 12:02 AM, Erinn Looney-Triggs wrote:
>> On 07/26/2014 07:12 PM, Erinn Looney-Triggs wrote:
>>> On 07/26/2014 05:25 PM, Erinn Looney-Triggs wrote:
>>>> Well it hasn't been all the pretty trying to move from RHEL
>>>> 6.5 to RHEL 7.
> 
>>>> I have two servers providing my ipa instances ipa and ipa2. 
>>>> Given that I don't have a great deal of spare capacity the
>>>> plan was to remove ipa2 from the replication agreement, modify
>>>> DNS so that only IPA was available in SRV logs (IPA does not
>>>> manage DNS at this point, was waiting for DNSSEC). As well, I
>>>> would change my sudo-ldap config files to point to ipa and
>>>> remove ipa2.
> 
>>>> Well that all worked well, installed RHEL 7 on the system and 
>>>> began working through the steps in the upgrade guide.
> 
>>>> First major problem was running into this bug: 
>>>> https://fedorahosted.org/freeipa/ticket/4375 ValueError: 
>>>> nsDS5ReplicaId has 2 values, one expected.
> 
>>>> Went and patched the replication.py file to get around that 
>>>> issue, and we moved on.
> 
>>>> Next up is my current issue: Exception from Java Configuration
>>>>  Servlet: Clone does not have all the required certificates.
> 
>>>> I suspect this is because I am running the CA as a subordinate 
>>>> to an AD CS instance, but I am unsure at this point.
> 
>>>> It has been a haul to get here, despite the short explanation.
>>>> It seems that my primary ipa instance is working on only a hit
>>>> or miss basis for kerberos tickets which has made all this a
>>>> bit of a pain. You can kinit as admin once it will fail unable
>>>> to find KDC, try again another three times, it will work. I
>>>> have even modified the krb5.conf file to point directly at the
>>>> server, thus bypassing DNS SRV lookups, however, that hasn't
>>>> worked.
> 
>>>> Point is, any help would be appreciated on the aforementioned 
>>>> error.
> 
>>>> -Erinn
> 
> 
>>> To reply to myself here, I believe the problem may be that I had 
>>> to renew the CA certificates and as such the certificates in 
>>> /root/cacert.p12 are no longer valid. It is this file that gets 
>>> bundled up with whatever else using ipa-replica-prepare, so I
>>> will have to create a new one that has the valid certificates in
>>> it.
> 
>>> One way or another though, if it isn't already documented, during
>>> a CA renewal this file should probably be updated with the
>>> correct certificates.
> 
>>> -Erinn
> 
>>> -Erinn
> 
> 
> 
>> Well thanks to this: 
>> http://www.freeipa.org/page/Howto/Change_Directory_Manager_Password
> 
>>  I have gotten a little further down the road an created a new 
>> cacert.p12 which looks to be complete.
> 
>> However, installation still fails in the same place:
> 
>> 2014-07-27T06:33:04Z DEBUG Starting external process 
>> 2014-07-27T06:33:04Z DEBUG args=/usr/sbin/pkispawn -s CA -f
>> /tmp/tmp5QGhUx 2014-07-27T06:33:25Z DEBUG Process finished, return
>> code=1 2014-07-27T06:33:25Z DEBUG stdout=Loading deployment
>> configuration from /tmp/tmp5QGhUx. Installing CA into
>> /var/lib/pki/pki-tomcat. Storing deployment configuration into 
>> /etc/sysconfig/pki/tomcat/pki-tomcat/ca/deployment.cfg. 
>> Installation failed.
> 
> 
>> 2014-07-27T06:33:25Z DEBUG stderr=pkispawn    : WARNING  ....... 
>> unable to validate security domain user/password through REST 
>> interface. Interface not available pkispawn    : ERROR    .......
>> Exception from Java Configuration Servlet: Clone does not have all
>> the required certificates
> 
>> 2014-07-27T06:33:25Z CRITICAL failed to configure ca instance
>> Command '/usr/sbin/pkispawn -s CA -f /tmp/tmp5QGhUx' returned
>> non-zero exit status 1 2014-07-27T06:33:25Z DEBUG   File 
>> "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py",
> 
> 
> line 638, in run_script
>> return_value = main_function()
> 
>> File "/usr/sbin/ipa-replica-install", line 667, in main CA =
>> cainstance.install_replica_ca(config)
> 
>> File 
>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py",
> 
> 
> line 1678, in install_replica_ca
>> subject_base=config.subject_base)
> 
>> File 
>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py",
> 
> 
> line 478, in configure_instance
>> self.start_creation(runtime=210)
> 
>> File 
>> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py",
>> line 364, in start_creation method()
> 
>> File 
>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py",
> 
> 
> line 604, in __spawn_instance
>> raise RuntimeError('Configuration of CA failed')
> 
>> 2014-07-27T06:33:25Z DEBUG The ipa-replica-install command failed, 
>> exception: RuntimeError: Configuration of CA failed
> 
> 
>> So some of the required certificates must be missing still.
> 
>> Unhelpfully, the ipa-server-install --uninstall process is not 
>> cleaning up everything after this failure, it leaves the CA intact
>> and the next run through the installer believes the CA is working
>> so it does not configure it. As such, I guess a re-install is
>> necessary or some other steps to truly clean everything that I
>> haven't found yet.
> 
>> -Erinn
> 
> Continuing on, in order to remove the CA I am manually running:
> pkidestroy -s CA -i pki-tomcat
> 
> And indeed there is a bug: https://fedorahosted.org/freeipa/ticket/2796
> 
> Interesting that the installer detects that the CA is installed, but
> the uninstaller does not detect it. I guess they are doing their
> detection in different ways.

The uninstaller doesn't rely on detection. There is a stored log of what
needs to be done. Unfortunately in this case the fact that the CA was
configured was added AFTER it was successfully installed and not when we
started, so if installation fails it can leave things half-installed but
not recorded.

> At this point I wanted to explore how feasible it would be to have a
> RHEL 7 replica without the CA replica portion, this ought to alleviate
> the KDC issues I seem to be having on the primary, which I have still
> to figure out.
> 
> So any reason not to do that? Would I simply be able to do a
> ipa-ca-install on the rhel 7 system at a future juncture and then
> perform the rest of the migration?

This would be a reasonable short-term stop-gap measure though if you can
live without a second CA. You would likely have the same problem with
ipa-ca-install, at least until we figure out what this missing cert
error means.

I've seen that error about missing certs before but I can't recall what
it means. I have the vague notion it is a little misleading though, and
that something else has already failed. I think we'll need one of the
dogtag devs to chime in. I'll poke them out-of-band.

rob