[Linux-cluster] Replication for iSCSI target?

[Michael McGlothlin]

All the storage appliances I've looked at were either severely limited
(2TB limit, no replication) or expensive ($4000 per node) and I'm not
a fan of black boxes which in my experience make the easy stuff easier
and the hard stuff a nightmare. I've got 15+ years of Linux (RedHat
preferred) experience and more with various other Unixes and have
experimented quite a bit with clustering but have never used it in
production so I just want to get any feedback I can from people who've
already done this. I'm not afraid of configuring things from the
command line, compiling code, etc.

It's my understanding that ESXi doesn't allow the use of random file
systems such as ZFS or Gluster directly so they'll have to be accessed
by NFS or iSCSI. NFS is probably easier but iSCSI seems more cluster
friendly as you can easily have multiple IP addresses for the same
data store. The general idea is so each physical machine will host a
VM that is a cluster filesystem node and will by default access that
filesystem from it's local VM (off the local RAID) and if the local
node should stop responding it will automatically switch to another
node. And by identifying all the nodes as being the same data store
then if the physical machine goes down there should be no problem in
automatically bringing those VMs back up on another physical host.

So I'm pretty sure we want a clustering filesystem that supports
replication across all nodes. Speed isn't nearly the issue for me that
reliability is. DRBD sounds good in that it works at the block level
and I'm pretty sure that as VMware locks individual VMs so that only
one ESXi server can access it at a time that there would be no
stepping on toes leading to filesystem corruption. Gluster sounds good
in that it doesn't make me jump through hoops to set up more than two
nodes and I do expect to be expanding to more than the current three
nodes in the near future. Either one is simple enough to get working
for the basics but I'm just trying to figure out the best
configuration for using it in the iSCSI configuration.


Thanks,
Michael McGlothlin