[Linux-cluster] >1500 MTU on EL5 causes things to go sideways

[Gordan Bobic]

Digimer wrote:

>   I've got a two-node EL5.6 cluster with a handful of interfaces on the
> nodes. When I do not specify MTU=x, everything works great. However, I
> want to use jumbo frames, and so the fun begins.
> 
>   As soon as I define MTU=2000 (for example), then cman on one note will
> start but not stop (the other node stops fine). Also, 'ccs_tool update
> /etc/cluster/cluster.conf' fails with:
> 
> ====
> [root at xenmaster003 ~]# ccs_tool update /etc/cluster/cluster.confFailed
> to receive COMM_UPDATE_NOTICE_ACK from xenmaster004.iplink.net.
> Hint: Check the log on xenmaster004.iplink.net for reason.
> 
> Failed to update config file.
> ====
> 
>   Nothing at all gets written to the target node's log files though.
> 
>   Is there some subtle magic needed to get jumbo frames working in the
> cluster? Am I missing the forest for the trees? :)

This may be a silly question, but as a first pass:

1) Do you have the MTU set on both nodes to the same value at the same time?

2) Have you confirmed that your NICs actually work properly with jumbo 
frames? Some Gb NICs can't handle them.

3) Does your switch support jumbo frames and have you explicitly enabled 
them on the switch?

I have used jumbo frames up to 16KB (on Intel NICs with a cross-over 
cable on a 2-node cluster) with no problems.

Gordan