[Freeipa-users] ipa-server-upgrade fails and CA cannot start

[Andrew C. Dingman]

For those of you who recognize me from non-public lists and chats, this
is a whole different setup from the one we've been discussing there.

This is on a RHEL 7 system, and unfortunately for me the CA master in
my personal IPA realm. When I attempted to update using yum on April
15th, the ipa-server-update script failed with what seems to be a dbus
error, and I have been unable to start the CA (and therefore ipa in
general) since. As a result, my personal systems are running on one IPA
server, which makes me more than a little nervous.

The relevant bit of the upgrade log seems to be:

2016-05-08T19:03:08Z DEBUG stderr=
2016-05-08T19:03:08Z INFO [Upgrading CA schema]
2016-05-08T19:03:08Z DEBUG flushing ldapi://%2fvar%2frun%2fslapd-
ACDINGMAN-COM.socket from SchemaCache
2016-05-08T19:03:08Z DEBUG retrieving schema for SchemaCache
url=ldapi://%2fvar%2frun%2fslapd-ACDINGMAN-COM.socket
conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x576e368>
2016-05-08T19:03:08Z DEBUG Processing schema LDIF file
/usr/share/pki/server/conf/schema-certProfile.ldif
2016-05-08T19:03:08Z DEBUG Not updating schema
2016-05-08T19:03:08Z INFO CA schema update complete (no changes)
2016-05-08T19:03:08Z INFO [Verifying that CA audit signing cert has 2
year validity]
2016-05-08T19:03:08Z DEBUG caSignedLogCert.cfg profile validity range
is 720
2016-05-08T19:03:08Z INFO [Update certmonger certificate renewal
configuration to version 4]
2016-05-08T19:03:08Z DEBUG Loading StateFile from
'/var/lib/ipa/sysupgrade/sysupgrade.state'
2016-05-08T19:03:08Z ERROR Failed to get request: bus, object_path and
dbus_interface must not be None.
2016-05-08T19:03:08Z ERROR IPA server upgrade failed: Inspect
/var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
2016-05-08T19:03:08Z DEBUG   File "/usr/lib/python2.7/site-
packages/ipapython/admintool.py", line 171, in execute
    return_value = self.run()
  File "/usr/lib/python2.7/site-
packages/ipaserver/install/ipa_server_upgrade.py", line 50, in run
    raise admintool.ScriptError(str(e))

2016-05-08T19:03:08Z DEBUG The ipa-server-upgrade command failed,
exception: ScriptError: bus, object_path and dbus_interface must not be
None.
2016-05-08T19:03:08Z ERROR bus, object_path and dbus_interface must not
be None.

There's a whole lot more, nearly 4MiB of log even when I reduce it to
my most recent attempt to run the upgrade script.

"getcert list" successfully shows 8 certificate requests being tracked.
Four are in "MONITORING" status, four in "NEED_CA". The NEED_CA
requests all indicate expiration back in February, and look like
crucial certificates: CN=CA Subsystem, CN=IPA RA, CN=CA Audit
and CN=OCSP Subsystem.

On the working replica, all eight are in "MONITORING" status and have
expiration dates in 2017 or later. I have not attempted the package
update on that system. Should I consider promoting this one to CA
master, force-deleting the old one, and reinstalling it as a new
system?

Please let me know what other information would be helpful for
diagnostics. The current state of all packages on the broken master is
up to earlier today from the official Red Hat content distribution
network.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ipaupgrade.log
Type: text/x-log
Size: 3902031 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20160508/c5e00d88/attachment.bin>