[Linux-cluster] Re: Csnap server cluster membership interface

[Daniel Phillips]

On Thursday 21 October 2004 12:12, Benjamin Marzinski wrote:
> > Since we can get cluster ports automagically, we can afford to have
> > a separate service group for each metadevice, i.e., per snapshot
> > store to be used by the cluster.  The name of the service group
> > will be formed from a unique id stored in the snapshot metadata at
> > mksnapstore time, prepended with csnap/ or something like that.
>
> The seperate service groups would definitely work, but it only works
> easily with the assigned ports per service group idea you mentioned.

Not so, it's perfectly workable without separate ports.

> I am leaning for the "one agent per node" idea, where the agent
> handles all the servers.
>
> There are only 256 ports available per node with cluster sockets.

This limit is entirely arbitrary.

> Some applications may be port gready.  Having the requirement that
> cman needs to find a port that is available on all the cluster nodes
> for each origin+snapshot pair is something that I wouldn't fault Dave
> for not wanting to include.

Allowing different services to bind to the same port on different nodes 
is an extremely bad idea.  If we're going to the trouble of having our 
own address family, let's not let the port assignments get fragmented.

> If you have one agent, you need only one service group,

Not so.  As I mentioned earlier, you can export virtual devices so that 
the membership set is different for every metadevice.  With one service 
group, all you do is force the agent to re-implement service groups, 
what kind of sense does that make?

> and one port.

True.  Whether that matters is another question.

Multiple service groups and multiple ports are two separate problems.  I 
have pointed out how nice it would be to let cman pair up ports and 
service groups, but please don't get distracted by that.

> Here is a one agent per node idea. The agent has a defined port that
> it always grabs, and all agents belong to the "csnap" service group.
> The agent starts up before the servers and clients, and opens a unix
> socket for communication with them.
> they open a connection to the agent, and pass in some unique
> identifier, and wait to be instantiated as the master. When a client
> connects, it requests a server, using the same identifier.
> Finding/instantiating a server works the same as it currently does
>
> When the agent gets the lock to instantiate one of it's servers as
> the new master, it contacts all the other agents in the "csnap" group
> at their defined port, and asks for all their clients of that server,
> again using the unique identifier. Once it builds up the list, it
> notifies the server that it is now master, and passes it the list of
> clients. The server waits for connections and then goes on it's merry
> way.

Yes, this is pretty much as originally described, except for omitting 
some details of what has to be done to complete the failover, e.g., an 
agent may need to connect to a server briefly for the sole purpose of 
satisfying the server that it has accounted for all the former snapshot 
clients.  (Note that the agents always make the client connections 
anyway, so this is no big challenge.)

I'd say we've satisfied the goal of knowing we can accommodate all 
possible cluster device topology and you ought to go ahead and 
implement a prototype, assuming a single well-known cluster port if you 
wish.

Regards,

Daniel