<div dir="ltr"><div><div><div><div><div><div><div>I was able to fix that may be temporarily... when i checked the network.. there was another process that was running and consuming a lot of network ( i have no idea who did that. I need to seriously start restricting people access to this machine ) </div>after killing that perfomance improved drastically </div>But now, suddenly I started experiencing the same hang. </div>This time , I gert the following error when checked dmesg [ 301.236976] ns-slapd[3124]: segfault at 0 ip 00007f1de416951c sp 00007f1dee1dba70 error 4 in libcos-plugin.so[7f1de4166000+b000] [ 1116.248431] TCP: request_sock_TCP: Possible SYN flooding on port 88. Sending cookies. Check SNMP counters. [11831.397037] ns-slapd[22550]: segfault at 0 ip 00007f533d82251c sp 00007f5347894a70 error 4 in libcos-plugin.so[7f533d81f000+b000] [11832.727989] ns-slapd[22606]: segfault at 0 ip 00007f6231eb951c sp 00007f623bf2ba70 error 4 in libcos-plugin.so[7f6231eb6000+b00 </div>and in /var/log/dirsrv/example-com/errors [23/Aug/2016:12:49:36 +0000] DSRetroclPlugin - delete_changerecord: could not delete change record 3291138 (rc: 32) [23/Aug/2016:12:49:36 +0000] DSRetroclPlugin - delete_changerecord: could not delete change record 3291139 (rc: 32) [23/Aug/2016:12:49:36 +0000] DSRetroclPlugin - delete_changerecord: could not delete change record 3291140 (rc: 32) [23/Aug/2016:12:49:36 +0000] DSRetroclPlugin - delete_changerecord: could not delete change record 3291141 (rc: 32) [23/Aug/2016:12:49:36 +0000] DSRetroclPlugin - delete_changerecord: could not delete change record 3291142 (rc: 32) [23/Aug/2016:12:49:36 +0000] DSRetroclPlugin - delete_changerecord: could not delete change record 3291143 (rc: 32) [23/Aug/2016:12:49:36 +0000] DSRetroclPlugin - delete_changerecord: could not delete change record 3291144 (rc: 32) [23/Aug/2016:12:49:36 +0000] DSRetroclPlugin - delete_changerecord: could not delete change record 3291145 (rc: 32) [23/Aug/2016:12:49:50 +0000] - Retry count exceeded in delete [23/Aug/2016:12:49:50 +0000] DSRetroclPlugin - delete_changerecord: could not delete change record 3292734 (rc: 51) </div>Can i do something about this error.. I treid to restart ipa a couple of time but that did not help </div>Thanks </div>Rakesh </div><div class="gmail_extra"> <div class="gmail_quote">On Mon, Aug 22, 2016 at 2:27 PM, Petr Spacek <<a href="mailto:pspacek@redhat.com" target="_blank">pspacek@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 19.8.2016 19:32, Rakesh Rajasekharan wrote: > I am running my set up on AWS cloud, and entropy is low at around 180 . > > I plan to increase it bu installing haveged . But, would low entropy by any > chance cause this issue of intermittent hang . > Also, the hang is mostly observed when registering around 20 clients > together Possibly, I'm not sure. If you want to dig into this, I would do this: 1. look what process hangs on client (using pstree command or so) $ pstree 2. look to what server and port is the hanging client connected to $ lsof -p <PID of the hanging process> 3. jump to server and see what process is bound to the target port $ netstat -pn 4. see where the process if hanging $ strace -p <PID of the hanging process> I hope it helps. Petr^2 Spacek <div class="HOEnZb"><div class="h5"> > On Fri, Aug 19, 2016 at 7:24 PM, Rakesh Rajasekharan < > <a href="mailto:rakesh.rajasekharan@gmail.com">rakesh.rajasekharan@gmail.com</a>> wrote: > >> yes there seems to be something thats worrying.. I have faced this today >> as well. >> There are few hosts around 280 odd left and when i try adding them to IPA >> , the slowness begins.. >> >> all the ipa commands like ipa user-find.. etc becomes very slow in >> responding. >> >> the SYNC_RECV are not many though just around 80-90 and today that was >> around 20 only >> >> >> I have for now increased tcp_max_syn_backlog to 5000. >> For now the slowness seems to have gone.. but I will do a try adding the >> clients again tomorrow and see how it goes >> >> Thanks >> Rakesh >> >> The issues >> >> On Fri, Aug 19, 2016 at 12:58 PM, Petr Spacek <<a href="mailto:pspacek@redhat.com">pspacek@redhat.com</a>> wrote: >> >>> On 18.8.2016 17:23, Rakesh Rajasekharan wrote: >>>> Hi >>>> >>>> I am migrating to freeipa from openldap and have around 4000 clients >>>> >>>> I had openned a another thread on that, but chose to start a new one >>> here >>>> as its a separate issue >>>> >>>> I was able to change the nssslapd-maxdescriptors adding an ldif file >>>> >>>> cat nsslapd-modify.ldif >>>> dn: cn=config >>>> changetype: modify >>>> replace: nsslapd-maxdescriptors >>>> nsslapd-maxdescriptors: 17000 >>>> >>>> and running the ldapmodify command >>>> >>>> I have now started moving clients running an openldap to Freeipa and >>> have >>>> today moved close to 2000 clients >>>> >>>> However, I have noticed that IPA hangs intermittently. >>>> >>>> running a kinit admin returns the below error >>>> kinit: Generic error (see e-text) while getting initial credentials >>>> >>>> from the /var/log/messages, I see this entry >>>> >>>> prod-ipa-master-int kernel: [104090.315801] TCP: request_sock_TCP: >>>> Possible SYN flooding on port 88. Sending cookies. Check SNMP counters. >>> >>> I would be worried about this message. Maybe kernel/firewall is doing >>> something fishy behind your back and blocking some connections or so. >>> >>> Petr^2 Spacek >>> >>> >>>> Aug 18 13:00:01 prod-ipa-master-int systemd[1]: Started Session 4885 of >>>> user root. >>>> Aug 18 13:00:01 prod-ipa-master-int systemd[1]: Starting Session 4885 of >>>> user root. >>>> Aug 18 13:01:01 prod-ipa-master-int systemd[1]: Started Session 4886 of >>>> user root. >>>> Aug 18 13:01:01 prod-ipa-master-int systemd[1]: Starting Session 4886 of >>>> user root. >>>> Aug 18 13:02:40 prod-ipa-master-int python[28984]: ansible-command >>> Invoked >>>> with creates=None executable=None shell=True args= removes=None >>> warn=True >>>> chdir=None >>>> Aug 18 13:04:37 prod-ipa-master-int sssd_be: GSSAPI Error: Unspecified >>> GSS >>>> failure. Minor code may provide more information (KDC returned error >>>> string: PROCESS_TGS) >>>> >>>> Could it be possible that its due to the initial load of adding the >>> clients >>>> or is there something else that I need to take care of. </div></div></blockquote></div> </div>