[Linux-cluster] Communication between LVS nodes

[Stuart Auchterlonie]

Firstly apologies for answering a 2 month old question.
answers below.

David Goldsmith wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Assume I have an LVS cluster setup with two LVS routers and 4 LVS member
> nodes.  Two of the nodes provide load balanced web servers.  Two of the
> nodes provide load-balanced proxy servers.
> 
> External customers connecting to the site can be passed to either of the
> two web server nodes.
> 
> Internal folks doing Internet browsing can be passed to either of the
> two proxy server nodes (assuming their web browser is configured to use
> the proxy).
> 
> Can the web servers in the LVS cluster use the LVS interface to the
> proxy servers rather than communicating directly to one of the two proxy
> server nodes?

Yes they can, you just have to abide by a few specific rules.
(I'm also assuming you are using DR)

1. The virtual ip address of the proxy service is different to
    the virtual ip address of the web service
2. The loopback adapter on each node only contains the virtual
    ip address of the service it's providing, either proxy or
    web, but not both. Otherwise the node will attempt to talk
    to itself.
3. The internal cluster network between the LVS directors and the
    LVS nodes is only used to forward packets from the directors
    to the real nodes, and is not used as the default route to the
    network.
4. The virtual ip addresses don't live in the same ip network as
    the realservers providing the service. ie.
    - 10.1.1.0/24 = virtual ip network
    - 10.1.2.0/24 = private cluster network
    - 10.1.3.0/25 = real server's externally routed network.

Essentially the whole arp problem is being avoided by putting
the virtual services on a different network, forcing connections
from the web servers (in this case) to be routed out to the
virtual addresses, rather than being allowed to "see" the real
servers and suffer the arp problem.

Maybe a picture will help

------------------------------------------ Virt Network (A)
              |               |
          Director        Director
              |               |
------------------------------------------ Priv Network (B)
    |           |           |          |
  RealWeb    RealWeb    RealProxy   RealProxy
    |           |           |          |
------------------------------------------ Real Server Net (C)

So any outbound connections from the RealWeb servers must
originate on (C) and due to being on different networks
get routed to (A) and then onto the RealProxy servers.


That's essentially what i have at the moment except that
the RealWeb servers are IIS nodes (ugh!) and network (C) is
split further into (C) & (D), with the different types of
realservers on different networks.


I am going to sit down and put this on a website one day
since it's in the "think it might work, but dunno" of every
document i read while setting it up.

Oh and as a footnote for the archives the windows boxes
can't bind to a /32 network on their loopback adapter,
so you will have to plan on /30 spacing of your virtual
ip addresses.


Regards,
Stuart Auchterlonie


> 
> If not, and the web server nodes are configured to connect to one
> specific proxy node, that would seem to create a possible failure point.
> 
> Thanks
> - --
> David Goldsmith