[Freeipa-users] Replica issue / Certificate Authority

[Christopher Young]

Ok.  I think I have a 'hint' here, but I could use some help getting this fixed.

Comparing the two IPA servers, I found the following (modified SOME of
the output myself):

on 'ipa02' (the 'good' one):
-----
ipa cert-show 1
  Issuing CA: ipa
  Certificate: <<<proper cert data here>>>
  Subject: CN=Certificate Authority,O=xxxx.LOCAL
  Issuer: CN=Certificate Authority,O=xxxx.LOCAL
  Not Before: Thu Jan 01 06:23:38 2015 UTC
  Not After: Mon Jan 01 06:23:38 2035 UTC
  Fingerprint (MD5): a6:aa:88:d4:66:e2:70:c1:e3:8c:37:0b:f3:eb:19:7d
  Fingerprint (SHA1):
11:c2:5a:58:bc:77:55:37:39:9b:13:b1:1a:a2:02:50:be:2e:a0:7f
  Serial number: 1
  Serial number (hex): 0x1
  Revoked: False
------


on 'ipa01'
-----
ipa cert-show 1
ipa: ERROR: Certificate operation cannot be completed: EXCEPTION
(Invalid Credential.)
-----

Thinking about this, I decided to just start checking for
inconsistencies and I noticed the following:

ipa02:
-----------
[root at ipa02 ~]# certutil -L -d /etc/httpd/alias/ -n ipaCert -a |
openssl x509 -text  | head
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 268304413 (0xffe001d)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: O=xxxxx.LOCAL, CN=Certificate Authority
        Validity
            Not Before: Nov 23 18:19:31 2016 GMT
            Not After : Nov 13 18:19:31 2018 GMT
        Subject: O=xxxxx.LOCAL, CN=IPA RA

----------
ipa01:
----------
[root at ipa01 tmp]# certutil -L -d /etc/httpd/alias/ -n ipaCert -a |
openssl x509 -text  | head
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 7 (0x7)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: O=xxxx.LOCAL, CN=Certificate Authority
        Validity
            Not Before: Jan  1 06:24:23 2015 GMT
            Not After : Dec 21 06:24:23 2016 GMT
        Subject: O=xxxx.LOCAL, CN=IPA RA

----------

So, it looks like somewhere in the process, the certificate got
renewed but not updated on 'ipa01'?  I apologize as I'm trying to
understand this.  I believe that my end goal is probably still the
same (verify replication and get things working properly on the
'ipa01' system.

Any help is very much appreciated!

-- Chris


On Fri, Dec 16, 2016 at 3:35 PM, Christopher Young
<mexigabacho at gmail.com> wrote:
> I'm hoping to provide enough information to get some help to a very
> important issue that I'm currently having.
>
> I have two IPA servers at a single location that recently had a
> replication issue that I eventually resolved by reinitializing one of
> the masters/replicas with one that seemed to be the most 'good'.
>
> In any case, somewhere in this process, the new IPA 4.4 was release
> with/for CentOS 7.3.
>
> At this moment, regular replication seems to be working properly (in
> that I don't have any obvious issues and web interfaces on both
> systems seem to be consistent for updates EXCEPT when it comes to the
> certificates).
>
> Before I get to the errors, here is the output of some of the commands
> that I would expect anyone would need:
>
> ----------
> [root at ipa01 ~]# ipa-replica-manage list
> ipa01.passur.local: master
> ipa02.passur.local: master
> -----
> [root at ipa01 ~]# ipa-replica-manage list -v ipa01.passur.local
> ipa02.passur.local: replica
>   last init status: None
>   last init ended: 1970-01-01 00:00:00+00:00
>   last update status: Error (0) Replica acquired successfully:
> Incremental update succeeded
>   last update ended: 2016-12-16 20:25:40+00:00
> -----
> [root at ipa01 ~]# ipa-replica-manage list -v ipa02.passur.local
> ipa01.passur.local: replica
>   last init status: None
>   last init ended: 1970-01-01 00:00:00+00:00
>   last update status: Error (0) Replica acquired successfully:
> Incremental update succeeded
>   last update ended: 2016-12-16 20:25:40+00:00
> -----
> [root at ipa01 ~]# ipa-replica-manage list-ruv
> Replica Update Vectors:
>         ipa01.passur.local:389: 4
>         ipa02.passur.local:389: 6
> Certificate Server Replica Update Vectors:
>         ipa02.passur.local:389: 97
>         ipa01.passur.local:389: 96
> ----------
>
>
> After the yum updates were applied to each system, I noticed that the
> results of 'ipa-server-upgrade' were quite different.  The 'ipa02'
> system went through without errors (this was also the system I used to
> reinitialize the other when I had a replication issue recently).
>
>
>
> On 'ipa01', I have following at the end of the 'ipaupgrade.log' file:
> ----------
> 2016-12-14T18:09:26Z ERROR IPA server upgrade failed: Inspect
> /var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
> 2016-12-14T18:09:26Z DEBUG   File
> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171,
> in execute
>     return_value = self.run()
>   File "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py",
> line 46, in run
>     server.upgrade()
>   File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py",
> line 1863, in upgrade
>     upgrade_configuration()
>   File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py",
> line 1785, in upgrade_configuration
>     ca_enable_ldap_profile_subsystem(ca)
>   File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py",
> line 336, in ca_enable_ldap_profile_subsystem
>     cainstance.migrate_profiles_to_ldap()
>   File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py",
> line 1984, in migrate_profiles_to_ldap
>     _create_dogtag_profile(profile_id, profile_data, overwrite=False)
>   File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py",
> line 1990, in _create_dogtag_profile
>     with api.Backend.ra_certprofile as profile_api:
>   File "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py",
> line 2060, in __enter__
>     raise errors.RemoteRetrieveError(reason=_('Failed to authenticate
> to CA REST API'))
>
> 2016-12-14T18:09:26Z DEBUG The ipa-server-upgrade command failed,
> exception: RemoteRetrieveError: Failed to authenticate to CA REST API
> 2016-12-14T18:09:26Z ERROR Unexpected error - see
> /var/log/ipaupgrade.log for details:
> RemoteRetrieveError: Failed to authenticate to CA REST API
> 2016-12-14T18:09:26Z ERROR The ipa-server-upgrade command failed. See
> /var/log/ipaupgrade.log for more information
> ----------
>
>
> In addition, when I go to the IPA web interface on the 'ipa01' system,
> I get the following when I try to view any of the certificates:
> ----------
> IPA Error 4301: CertificateOperationError
>
> Certificate operation cannot be completed: EXCEPTION (Invalid Credential.)
> ----------
>
>
> I was wondering if there was a method for taking all the CA
> details/tree/what have you from my 'ipa02' system and using it to
> repopulate the 'ipa01'.   Since everything else seems to be working
> correctly after a reinitialize on 'ipa01', I thought this would be the
> safest way, but I'm opening any solutions as I need to get this fixed
> ASAP.
>
> Please let me know any additional details that may help OR if there is
> a procedure that I could use to quickly and easily recreate 'ipa01'
> WITH the certificate authority properly working on both.  I may need
> some educate there.
>
>
> Thanks!
>
> -- Chris