[Linux-cluster] Bonding Interfaces: Active Load Balancing & LACP

[Digimer]

As an aside; I was using the DGS-3100 switches stacked. The new 
generation of DGS-3120 switches I also used stacked, and are a *marked* 
improvement over the 3100 series. I've not gone back to re-test the 
other bond modes on these switches, as I must live within Red Hat's 
supported configuration. However, this thread might just motivate me to 
pull aside a test cluster and do some of my own testing again, on the 
new switches.

Digimer

On 06/07/2012 04:51 AM, Radu Rendec wrote:
> I also experimented with D-Link DGS-3xxx switches and the bonding
> driver, but in a quite strange configuration: 2 distinct switches
> without any "knowledge" of each other, and with each server having NIC
> #1 connected in one switch and NIC #2 in the other.
>
> In my case, the bonding driver actually splitted the traffic between the
> 2 links and I could achieve higher speeds than with a single link. This
> is my config (nothing fancy):
>
> [root at host ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond2
> DEVICE=bond2
> TYPE=Bonding
> BONDING_OPTS="miimon=50"
> ONBOOT=yes
> BOOTPROTO=none
> BRIDGE=br2
>
> The bond interface was assigned to a bridge in my case, because I needed
> to give network access to some VMs (besides the physical host).
>
> If you don't need 802.1q tagging, I think you could try to create 2
> VLANs in your switch and for all servers have one NIC in one vlan and
> and another NIC in the other vlan and then a bond across those 2 NICs.
>
> The drawback is that everything that is connected in this manner needs
> to have exactly 2 NICs connected in the 2 VLANs. The other problem is
> that if one NIC fails in one of the servers, it won't receive the
> packets that are sent on the corresponding VLAN, so the server will not
> receive half of the traffic that is meant for it.
>
> I'm also curious about the results of your experiments. Please post back
> if you have time.
>
> Thanks,
>
> Radu Rendec
>
> On Thu, 2012-06-07 at 00:35 -0400, Digimer wrote:
>> I know that the only *supported* bond is Active/Passive (mode=1), which
>> of course provides no performance benefit.
>>
>> I tested all types, using more modest D-Link DGS-3100 switches and all
>> other modes failed at some point in failure and recovery testing. If you
>> want to experiment, I'd suggest tweaking corosync's timeouts to be
>> (much?) more generous.
>>
>> I'm curious to hear back on what your experimenting finds.
>>
>> Digimer
>>
>> On 06/07/2012 12:12 AM, Eric wrote:
>>> I'm currently using the HP Procurve 2824 24-port Gigabit Ethernet switch
>>> to for a backside network for synchronizing file systems between the
>>> nodes in the group. Each host has 4 Gigabit NIC's and the goal is to
>>> bond two of the Gigabit NIC's together to create a 2 Gbps link from any
>>> host to any other host but what I'm finding is that the bonded links are
>>> only capable of 1 Gbps from any host to any other host. Is it possible
>>> to create a multi-Gigabit link between two hosts (without having to
>>> upgrade to 10G) using a switch that "uses the SA/DA (Source
>>> Address/Destination Address) method of distributing traffic across the
>>> trunked links"?
>>>
>>> The problem, at least as far as I can tell, comes down to the limitation
>>> of ARP resolution (in the host) and mac-address tables (in the switch):
>>>
>>> When configured to use Active Load Balancing, the kernel driver leaves
>>> each of the interface's MAC addresses unchanged. In this scenario, when
>>> Host A sends sends traffic to host Host B, the kernel uses the MAC
>>> address of only one of Host B's NIC's as the DA. When the packet arrives
>>> at the switch, the switch consults the mac-address table for the DA and
>>> then sends the packet to the interface connected to the NIC with MAC
>>> address equal to DA. Thus packets from Host A to Host B will only leave
>>> the switch through one interface - the interface connected to the NIC
>>> with MAC address equal to DA. This has the effect of limiting the
>>> throughput from Host A to Host B to the speed of the one interface
>>> connected to the NIC with MAC address equal to DA.
>>>
>>> When configured to use IEEE 802.3ad (LACP), the kernel driver assigns
>>> the same MAC address to all of the hosts' interfaces. In this scenario,
>>> when Host A sends traffic to Host B, the kernel uses Host B's shared MAC
>>> address as the DA. When the packet arrives at the switch, the switch
>>> creates a hash based on the SA/DA pair, consults the mac-address table
>>> for the DA, and and assigns the flow (i.e., traffic from Host A to Host
>>> B) to one of the interfaces connected to Host B. Thus packets from Host
>>> A to Host B will only leave the switch through one interface - the
>>> interface determined by the SA/DA hash. This has the effect of limiting
>>> the throughput from Host A to Host B to the speed of the one interface
>>> determined by the hashing method. However, if the flow (from Host A to
>>> Host B's shared MAC address) were to be distributed across the different
>>> interfaces in a round-robin fashion (as the packets were leaving the
>>> switch) the throughput between the hosts would equal the aggregate of
>>> the links (IIUC).
>>>
>>> Is this a limitation of the the Procurve's implementation of LACP? Do
>>> other switches use different methods of distributing traffic across the
>>> trunked links? Is there another method of aggregating the links between
>>> the two hosts (e.g., multipathing)?
>>>
>>> TIA,
>>> Eric Pretorious
>>> Truckee, CA
>>>
>>>
>>> --
>>> Linux-cluster mailing list
>>> Linux-cluster at redhat.com
>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>
>>
>
>
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster


-- 
Digimer
Papers and Projects: https://alteeve.com