[rhelv6-list] SOLVED: Lots of (thousands) tcp-connections in state TIME_WAIT on heavy load NFS-server

[Collins, Kevin [BEELINE]]

No problem - always happy to help on things like this. You might also want to investigate the use of nscd - it may help alleviate some of these problems, too.

Kevin

-----Original Message-----
From: Masopust, Christian [mailto:christian.masopust at siemens.com] 
Sent: Tuesday, January 11, 2011 12:26 PM
To: 'Red Hat Enterprise Linux 6 (Santiago) discussion mailing-list'
Cc: Collins, Kevin [BEELINE]
Subject: SOLVED: Lots of (thousands) tcp-connections in state TIME_WAIT on heavy load NFS-server


Hi Kevin,

thanks a lot for your answer. Indeed it was a "NIS-problem" that lead to that much
short lived connections....  NFS's mountd is using netgroup from NIS, also hosts
come from NIS... I've checked now the netgroups (have been created by the an admin
that already left company :-)) and found that there are lots of nested netgroups,
some netgroups used in /etc/exports are no longer existing and so on...

I've now changed hosts and netgroup on my server to use only files and it works 
wonderful now !!!

Again thanks a lot for pointing me in the right direction :-))
Christian


-----Ursprüngliche Nachricht-----
Von: rhelv6-list-bounces at redhat.com [mailto:rhelv6-list-bounces at redhat.com] Im Auftrag von Collins, Kevin [BEELINE]
Gesendet: Dienstag, 11. Jänner 2011 17:36
An: Red Hat Enterprise Linux 6 (Santiago) discussion mailing-list
Betreff: Re: [rhelv6-list] Lots of (thousands) tcp-connections in state TIME_WAIT on heavy load NFS-server

Port :962 is being connected to... you want to find out what is opening
those connections. You could try using lsof on the source ports instead,
although if you have very short lived processes it might be difficult.

I don't have any RHEL6 with NIS (we use LDAP), so I can't comment on if
it is "normal". I don't think it is behavior I have seen with NIS on
previous versions or RHEL.

Kevin

-----Original Message-----
From: rhelv6-list-bounces at redhat.com
[mailto:rhelv6-list-bounces at redhat.com] On Behalf Of Masopust, Christian
Sent: Tuesday, January 11, 2011 6:45 AM
To: 'rhelv6-list at redhat.com'
Subject: [rhelv6-list] Lots of (thousands) tcp-connections in state
TIME_WAIT on heavy load NFS-server

Hi,
 
I've recently installed a new (big) HP ProLiant DL585-G7 with RHEL 6
working as NFS-server.
 
Now I've seen thousands of tcp-connections in TIME_WAIT state. I've
already set 
net.ipv4.tcp_fin_timeout = 10, but the number of the connections in
TIME_WAIT remains almost
the same (also setting tcp_fin_timeout to 1 doesn't change anything
here). 

Checking "netstat -tn" shows: 
 
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address               Foreign Address
State
......
tcp        0      0 127.0.0.1:47120             127.0.0.1:962
TIME_WAIT
tcp        0      0 127.0.0.1:689               127.0.0.1:962
TIME_WAIT
tcp        0      0 127.0.0.1:46500             127.0.0.1:962
TIME_WAIT
tcp        0      0 127.0.0.1:46927             127.0.0.1:962
TIME_WAIT
tcp        0      0 127.0.0.1:46694             127.0.0.1:962
TIME_WAIT
.......
 
and lsof for port 962 shows:
 
lsof -i :962
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
ypbind  46372 root    6u  IPv4 444939      0t0  TCP *:962 (LISTEN)

so, is it ypbind that is having that much (short lived) connections???
why??

or what else could be the reason for having that much local connections?

any idea?

thanks a lot,
Christian

_______________________________________________
rhelv6-list mailing list
rhelv6-list at redhat.com
https://www.redhat.com/mailman/listinfo/rhelv6-list

_______________________________________________
rhelv6-list mailing list
rhelv6-list at redhat.com
https://www.redhat.com/mailman/listinfo/rhelv6-list