[Linux-cluster] >1500 MTU on EL5 causes things to go sideways

[Digimer]

On 04/12/2011 11:16 AM, Gordan Bobic wrote:
> Digimer wrote:
> 
>>   I've got a two-node EL5.6 cluster with a handful of interfaces on the
>> nodes. When I do not specify MTU=x, everything works great. However, I
>> want to use jumbo frames, and so the fun begins.
>>
>>   As soon as I define MTU=2000 (for example), then cman on one note will
>> start but not stop (the other node stops fine). Also, 'ccs_tool update
>> /etc/cluster/cluster.conf' fails with:
>>
>> ====
>> [root at xenmaster003 ~]# ccs_tool update /etc/cluster/cluster.confFailed
>> to receive COMM_UPDATE_NOTICE_ACK from xenmaster004.iplink.net.
>> Hint: Check the log on xenmaster004.iplink.net for reason.
>>
>> Failed to update config file.
>> ====
>>
>>   Nothing at all gets written to the target node's log files though.
>>
>>   Is there some subtle magic needed to get jumbo frames working in the
>> cluster? Am I missing the forest for the trees? :)
> 
> This may be a silly question, but as a first pass:

No worries, I routinely make silly mistakes, so silly questions are
appropriate. :)

> 1) Do you have the MTU set on both nodes to the same value at the same
> time?

Yes.

> 2) Have you confirmed that your NICs actually work properly with jumbo
> frames? Some Gb NICs can't handle them.

Yes.

> 3) Does your switch support jumbo frames and have you explicitly enabled
> them on the switch?

Yes and yes.

> I have used jumbo frames up to 16KB (on Intel NICs with a cross-over
> cable on a 2-node cluster) with no problems.
> 
> Gordan

Thanks for the questions. Still answer-hunting though. :P

-- 
Digimer
E-Mail: digimer at alteeve.com
AN!Whitepapers: http://alteeve.com
Node Assassin:  http://nodeassassin.org