[Linux-cluster] Where to go with cman ?

[Lars Marowsky-Bree]

On 2005-07-18T09:10:30, Patrick Caulfield <pcaulfie at redhat.com> wrote:

A late reply is better than no reply I guess.

FWIW, the Linux-HA project has been going through some of the same
thoughts. 

(Not wanting to make any statements about the future of NCS here, which
is a different group, but I know that Robert is keenly aware of
OpenAIS/libtotem etc too.)

Linux-HA is looking to integrate with OpenAIS. The exact design is yet
to be understood, but what is clear so far is that I'm convinced we'll
support running on top of OpenAIS as an additional messaging layer, and
figure out a way how to keep our Concensus Cluster Membership and
OpenAIS's one synchronized.

We're not quite ready to abandon our existing communication layer
completely in exchange with OpenAIS, for a variety of good reasons:

- We support some scenarios which OpenAIS doesn't; we can support raw
  serial communication links for example, which is very useful for 2
  node clusters (which make up 100% of our existing userbase and
  probably even with the ability to support N nodes, will continue to be
  in excess of 90% of all installations.)

- OpenAIS relies on the cluster network to be protected by IP/Sec (or
  other forms of network layer security, and be it just "don't you dare
  run this over anything but private links"). This is well for some
  scenarios, but makes setup more complex.
  
- A gradual shiftover is required; both in terms of managing risk for
  our own release cycle (we understand what we have, while we have yet
  to work out the quirks of OpenAIS), as well as allowing customers to
  do a rolling upgrade.

- It will allow other applications to interoperate with our comm layer
  just fine - you can simply add OpenAIS to an existing cluster.

Of course, there's some downside to this, namely that we cannot rely on
advanced OpenAIS features like extended virtual synchrony being
available everywhere, so we cannot ourselves take full advantage of the
simplifications this might offer for our join protocol - we can, but we
have to support both code paths for the transition period.

I believe this is a step in the right direction, and fully support the
reasons you mentioned.

> The disadvantages are
> - Need to learn the internals of someone else's code.

Again, I think this is offset by the increased number of people pounding
at the code.

> - We don't have full control over the code. Although we can obviously
> fork it if we feel the need it would, obviously be preferable not to.

I'd suggest to not consider this an option, it'll kill the
interoperability again. ;-)

> - non-compatibility with "old" cman, making rolling upgrades har or
> even impossible. I'm not sure what to do about this yet, but it's
> worth pointing out that the DLM has a new line-protocol too.

Maybe something like the strategy I described above and which we'll use
might work for you here too?

> - openAIS is BSD licensed, I don't think this is a problem but it
> probably needs checking.

Consuming a BSD licensed library in GPL'ed code should not be a problem.

> In short, I'm advocating adopting the openAIS core (libtotem
> basically) as CMAN's communications/membership protocol. If we're
> going to do a "CMAN V2" that has anything significant over V1 then
> re-inventing it is going to be a huge amount of work that someone else
> has already done.

Again, this would go quite a step towards resolving cluster
interoperability issues.


Sincerely,
    Lars Marowsky-Brée <lmb at suse.de>

-- 
High Availability & Clustering
SUSE Labs, Research and Development
SUSE LINUX Products GmbH - A Novell Business	 -- Charles Darwin
"Ignorance more frequently begets confidence than does knowledge"

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/linux-cluster/attachments/20050724/b2986121/attachment.sig>