[Freeipa-users] free ipa hangs

[Aaron Collins]

Every day or so one of my masters hangs.  The processes is still running no errors in the logs, the tcp port is still listening but it’s not fulling requests.  This also hangs any machines querying from the host as the sssd timeout doesn’t seem to take affect during the tcp connection.  From looking through the archives and troubleshooting docs I’ve taken a stack trace, but I can’t make heads or tails of what it means.  I was hoping someone here could review it and tell me perhaps where I should be looking.
Stack Trace: https://dl.dropboxusercontent.com/u/13113270/stacktrace.1485223167.txt

Using FreeIPA 4.4
[root at ipa-dc1.core2<mailto:root at ipa-dc1.core2>]# rpm -qa |grep ipa-server
ipa-server-4.4.0-14.el7.centos.1.1.x86_64
ipa-server-common-4.4.0-14.el7.centos.1.1.noarch
[root at ipa-dc1.core2<mailto:root at ipa-dc1.core2>]# rpm -qa |grep 389-ds
389-ds-base-snmp-1.3.5.10-12.el7_3.x86_64
389-ds-base-debuginfo-1.3.5.10-12.el7_3.x86_64
389-ds-base-libs-1.3.5.10-12.el7_3.x86_64
389-ds-base-1.3.5.10-12.el7_3.x86_64
[root at ipa-dc1.core2<mailto:root at ipa-dc1.core2>]# uname -a
Linux ipa-dc1.core2.cloud.cheggnet.com 3.10.0-123.8.1.el7.x86_64 #1 SMP Mon Sep 22 19:06:58 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

In addition I’ve also noticed I have ghost replicas {15,24,13} which I can’t seem to remove and a high number of changes not sure if this is normal.

[root at ipa-dc1.core2<mailto:root at ipa-dc1.core2>]#  ldapsearch -xLLL -H ldap://localhost  -D "cn=directory manager"  -b dc=randomnet,dc=com  '(&(nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff)(objectclass=nstombstone))' nscpentrywsi
dn: cn=replica,cn=dc\3Drandomnet\2Cdc\3Dcom,cn=mapping tree,cn=config
nscpentrywsi: dn: cn=replica,cn=dc\3Drandomnet\2Cdc\3Dcom,cn=mapping tree,cn=config
nscpentrywsi: cn: replica
nscpentrywsi: createTimestamp: 20170113010933Z
nscpentrywsi: creatorsName: cn=Directory Manager
nscpentrywsi: modifiersName: cn=Multimaster Replication Plugin,cn=plugins,cn=config
nscpentrywsi: modifyTimestamp: 20170124195814Z
nscpentrywsi: nsDS5Flags: 1
nscpentrywsi: nsDS5ReplicaBindDN: cn=replication manager,cn=config
nscpentrywsi: nsDS5ReplicaBindDN: …
nscpentrywsi: nsDS5ReplicaId: 28
nscpentrywsi: nsDS5ReplicaName: f135c48c-d92c11e6-ba4cb212-9904938c
nscpentrywsi: nsDS5ReplicaRoot: dc=randomnet,dc=com
nscpentrywsi: nsDS5ReplicaType: 3
nscpentrywsi: nsState:: HAAAAAAAAADLsYdYAAAAAAEAAAAAAAAAuwAAAAAAAAAKAAAAAAAAAA
==
nscpentrywsi: nsds5ReplicaLegacyConsumer: off
nscpentrywsi: nsds5replicabinddngroup: cn=replication managers,cn=sysaccounts,cn=etc,dc=randomnet,dc=com
nscpentrywsi: nsds5replicabinddngroupcheckinterval: 60
nscpentrywsi: objectClass: nsds5replica
nscpentrywsi: objectClass: top
nscpentrywsi: objectClass: extensibleobject
nscpentrywsi: numSubordinates: 3
nscpentrywsi: nsds50ruv: {replicageneration} 564e51c0000000040000
nscpentrywsi: nsds50ruv: {replica 28 ldap://ipa-dc1.core2.cloud.RANDOMNET.COM:389} 587829930000001c0000 5887b2780018001c0000
nscpentrywsi: nsds50ruv: {replica 17 ldap://ipa.RANDOMNET.COM:389} 57ec1b37000000110000 5887b2c6000c00110000
nscpentrywsi: nsds50ruv: {replica 22 ldap://ipa-dc03.core2.cloud.RANDOMNET.COM:389} 5818c9ac000100160000 58877a26000100160000
nscpentrywsi: nsds50ruv: {replica 9 ldap://ipa-dc2.core2.cloud.RANDOMNET.COM:389} 56c57637000000090000 58878ffe000600090000
nscpentrywsi: nsds50ruv: {replica 27 ldap://ipa-dc02.corp1.cloud.RANDOMNET.COM:389} 587747710000001b0000 5877d75c0009001b0000
nscpentrywsi: nsds50ruv: {replica 29 ldap://ipa-dc01.test3.cloud.RANDOMNET.COM:389} 5878372a0000001d0000 588795e20001001d0000
nscpentrywsi: nsds50ruv: {replica 30 ldap://ipa-dc02.test3.cloud.RANDOMNET.COM:389} 58783e2d0000001e0000 588799c50003001e0000
nscpentrywsi: nsds50ruv: {replica 15} 5875c4330002000f0000 587836f20009000f0000
nscpentrywsi: nsds50ruv: {replica 24} 5877d808000500180000 58781d6a000500180000
nscpentrywsi: nsds50ruv: {replica 13} 581802c70000000d0000 581802c70000000d0000
nscpentrywsi: nsds5agmtmaxcsn: dc=randomnet,dc=com;ipa-dc1.core2.cloud.randomnet.com-to-ipa-dc03.core2.cloud.RANDOMNET.COM;ipa-dc03.core2.cloud.RANDOMNET.COM;389;22;588778510002001c0000
nscpentrywsi: nsds5agmtmaxcsn: dc=randomnet,dc=com;ipa-dc1.core2.cloud.randomnet.com-to-ipa-dc2.core2.cloud.RANDOMNET.COM;ipa-dc2.core2.cloud.RANDOMNET.COM;389;9;588778510002001c0000
nscpentrywsi: nsds5agmtmaxcsn: dc=randomnet,dc=com;ipa-dc1.core2.cloud.randomnet.com-to-ipa.RANDOMNET.COM;ipa.RANDOMNET.COM;389;17;588778510002001c0000
nscpentrywsi: nsruvReplicaLastModified: {replica 28 ldap://ipa-dc1.core2.cloud.RANDOMNET.COM:389} 5887b1bc
nscpentrywsi: nsruvReplicaLastModified: {replica 17 ldap://ipa.RANDOMNET.COM:389} 5887b20d
nscpentrywsi: nsruvReplicaLastModified: {replica 22 ldap://ipa-dc03.core2.cloud.RANDOMNET.COM:389} 5887796c
nscpentrywsi: nsruvReplicaLastModified: {replica 9 ldap://ipa-dc2.core2.cloud.RANDOMNET.COM:389} 58878f46
nscpentrywsi: nsruvReplicaLastModified: {replica 27 ldap://ipa-dc02.corp1.cloud.RANDOMNET.COM:389} 00000000
nscpentrywsi: nsruvReplicaLastModified: {replica 29 ldap://ipa-dc01.test3.cloud.RANDOMNET.COM:389} 58879526
nscpentrywsi: nsruvReplicaLastModified: {replica 30 ldap://ipa-dc02.test3.cloud.RANDOMNET.COM:389} 5887990a
nscpentrywsi: nsruvReplicaLastModified: {replica 15} 00000000
nscpentrywsi: nsruvReplicaLastModified: {replica 24} 00000000
nscpentrywsi: nsruvReplicaLastModified: {replica 13} 00000000
nscpentrywsi: nsds5ReplicaChangeCount: 58090
nscpentrywsi: nsds5replicareapactive: 0

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20170125/e7cbc817/attachment.htm>