[Linux-cluster] CLVM and AoE

[Aaron Stewart]

Hey Bowie,

Wow.. That's perfect.  Thanks for the response.

I have a question about whether GFS is a requirement.. Since each lv is 
a separate partition mounted on xen, does GFS make sense, or can we use 
ext3/xfs/etc.?

-=Aaron

Bowie Bailey wrote:
> Aaron Stewart wrote:
>   
>> I'm currently in process of setting up a Coraid ATA over Ethernet
>> device as a backend storage for multiple systems that export
>> individual partitions to Xen virtual servers.  In our discussions
>> with Coraid, they suggested looking into CLVM in order to handle this.
>>
>> Obviously, I have some questions.. :)
>>
>> - Has anyone used this kind of setup?  I have very little experience
>> with Redhat's cluster management, but have a fairly high level of
>> expertise overall in this arena.
>>     
>
> I don't know anything about Xen, but I am using this same basic setup
> on my systems.
>
>   
>> - How does management of LVM logical volumes occur?  Do we need to
>> maintain one server that administers the volume group?
>>     
>
> The management is distributed.  You can manage the cluster and volume
> groups from any node.
>
>   
>> - What kind of pitfalls should we be aware of?
>>     
>
> Some people have complained about throughput issues with GFS.  Our
> application doesn't require high throughput, so I can't comment on
> this.  I haven't found any issues in my testing so far.
>
>   
>> Can anyone point to any experience or any HOWTO's that discuss setting
>> something like this up?
>>     
>
> There are a few documents, but most of the ones that I've seen are out
> of date.  If you have specific questions, you can ask here.
>
> If you don't have it already, here is the yum config with the current
> cluster RPMs for CentOS.  Just drop it in a file in /etc/yum.repos.d/.
> Note that the current cluster RPMs are for the new 2.6.9-34.EL kernel.
>
> ----------------------------
> [csgfs]
> name=CentOS-4 - CSGFS
> baseurl=http://mirror.centos.org/centos/$releasever/csgfs/$basearch/
> gpgcheck=1
> enabled=1
> ----------------------------
>
> The only thing you need to build from source is the AoE driver from
> CoRaid.
>
>   
>> Here's the setup:
>>
>> 1. Coraid SR1520 configured in one lblade, exported via AoE on a
>> dedicated storage network as one LUN
>> 2. Centos4.2 on all cluster nodes
>> 3. logical volumes get masked when getting passed into Xen, so on the
>> Dom0 controller it should look like /dev/VolGroup00/{xenvmID} (which
>> shows up in the virtual as /dev/sda1)
>> 4. only one host need access to a given logical volume at any given
>> time.  If migration needs to occur, the volume should be unmounted and
>> remounted on another physical system.
>>     
>
> This can be done, but the cluster will not do it for you.  Each
> logical volume can be accessed by as many nodes as you need.  Note
> that you need one GFS journal per node that needs simultaneous access.
>
>   
>> 5. Despite the fact that AoE is a layer 4 protocol, apparently it can
>> coexist with IP on the same network interface, so we can transport
>> cluster metadata over the same interface.  Barring that, there is a
>> second (public) interface on each box.
>> 6. We want to avoid a single point of failure (such as a second AoE
>> server that exports luns from lvm lv's)
>>     
>
> Now that DLM is the recommended locking manager, everything is
> distributed.  Your only single point of failure is the CoRaid box.
>
>   

-------------- next part --------------
A non-text attachment was scrubbed...
Name: aaron.vcf
Type: text/x-vcard
Size: 289 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/linux-cluster/attachments/20060412/7961a6c6/attachment.vcf>