[Linux-cluster] LVS Director + TG3 = problem ?

[FM]

Hello,
With RHEL 4.4 CS :
My director : HP DL360 (TG3 RH module), 2 Gb NIC (bonding mode=0) behind
a PIX firewall (100 Mb NIC). All servers connected to  Gb CISCO switches

I'm having a hard time with this server. the LVS seems to be ok (web
sites are available for users ;) ). But, in the logs, I have a lot of
nanny disabling real servers and then enabling them 1 second later. And
when connecting using SSH, I have a connection time out or when the
connection succeed, the ssh session freezes after several minutes. I
talked to CISCO admin and traffic is not crazy in the director ports.

I read in the past that the TG3 is not recommended by HP. Do you have 
that kind of issue with the mix hp broadcom NIC + tg3 RH module ?


Trying to tune the TCP stack, I add this in the sysctl.conf :
# This setting changes the maximum network receive buffer size
net.core.rmem_max= 16777216

# This setting changes the maximum network send buffer size
net.core.wmem_max= 16777216

# This sets the kernel's minimum, default, and maximum TCP receive 
buffer sizes.
net.ipv4.tcp_rmem=4096 87380 16777216

# This sets the kernel's minimum, default, and maximum TCP send buffer 
sizes.
net.ipv4.tcp_wmem=4096 65536 16777216






Here is a ipvsadm output :

  Virtual Server version 1.2.0 (size=4096)
Prot LocalAddress:Port Scheduler Flags
   -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.3.157:443 wlc persistent 300 mask 255.255.255.0
   -> 192.168.3.29:443             Route   3      0          0
   -> 192.168.3.25:443             Route   1      0          0
   -> 192.168.3.27:443             Route   1      0          0
   -> 192.168.3.26:443             Route   1      0          0
   -> 192.168.3.24:443             Route   1      0          0
TCP  192.168.3.158:443 wlc persistent 300 mask 255.255.255.0
   -> 192.168.3.25:443             Route   1      0          0
   -> 192.168.3.27:443             Route   1      0          0
   -> 192.168.3.26:443             Route   1      0          0
   -> 192.168.3.24:443             Route   1      0          0
TCP  192.168.3.159:443 wlc persistent 300 mask 255.255.255.0
   -> 192.168.3.29:443             Route   3      0          0
   -> 192.168.3.25:443             Route   1      0          0
   -> 192.168.3.27:443             Route   1      0          0
   -> 192.168.3.26:443             Route   1      0          0
   -> 192.168.3.24:443             Route   1      0          0
TCP  192.168.3.136:443 wlc persistent 300
   -> 192.168.3.25:443             Route   1      0          1
   -> 192.168.3.27:443             Route   1      0          0
   -> 192.168.3.24:443             Route   1      0          0
   -> 192.168.3.26:443             Route   1      0          0
TCP  192.168.3.147:80 wlc
   -> 192.168.3.24:80              Route   1      2          218
   -> 192.168.3.25:80              Route   1      2          252
   -> 192.168.3.26:80              Route   1      3          252
   -> 192.168.3.27:80              Route   1      2          217
TCP  192.168.3.158:80 wlc
   -> 192.168.3.24:80              Route   1      0          47
   -> 192.168.3.25:80              Route   1      0          40
   -> 192.168.3.26:80              Route   1      0          48
   -> 192.168.3.27:80              Route   1      0          47
TCP  192.168.3.159:80 wlc
   -> 192.168.3.24:80              Route   1      0          2
   -> 192.168.3.27:80              Route   1      0          1
   -> 192.168.3.26:80              Route   1      0          1
   -> 192.168.3.25:80              Route   1      0          1
TCP  192.168.3.156:80 wlc
   -> 192.168.3.24:80              Route   1      0          0
   -> 192.168.3.25:80              Route   1      0          0
   -> 192.168.3.27:80              Route   1      0          0
   -> 192.168.3.26:80              Route   1      0          0
TCP  192.168.3.157:80 wlc
   -> 192.168.3.25:80              Route   1      1          31
   -> 192.168.3.24:80              Route   1      0          50
   -> 192.168.3.26:80              Route   1      0          50
   -> 192.168.3.27:80              Route   1      0          48
TCP  192.168.3.135:80 wlc
   -> 192.168.3.25:80              Route   1      0          0
   -> 192.168.3.24:80              Route   1      0          1
   -> 192.168.3.27:80              Route   1      0          1
   -> 192.168.3.26:80              Route   1      0          0
TCP  192.168.3.132:80 wlc
   -> 192.168.3.24:80              Route   1      0          1
   -> 192.168.3.25:80              Route   1      0          1
   -> 192.168.3.27:80              Route   1      0          2
   -> 192.168.3.26:80              Route   1      0          1
TCP  192.168.3.133:80 wlc
   -> 192.168.3.25:80              Route   1      0          0
   -> 192.168.3.24:80              Route   1      0          0
   -> 192.168.3.27:80              Route   1      0          0