<div dir="ltr">Sorry, but I have not clear the clvm role.<div><br></div><div>CLVM shares VG metadata over a cluster and makes possible a wide cluster administration (RedHat documentation says).</div><div><br></div><div>In this way a CLVM Cluster must have a CMAN Cluster up and running.</div>
<div><br></div><div>So, if I have already a shared storage, the only thing I can do is to make a GFS filesystem and export this one to clients machine. In this way the shared storage could be accessed by multiples machines.</div>
<div><br></div><div>In this scenario clvm is not useful because the shared lock on filesystem is guaranted by GFS.</div><div><br></div><div>Let's pose I have different machines that I want to join in a cluster. Each machine has a storage that I want share with the other machines to create a large storage.</div>
<div><br></div><div>With CLVM, stands to RedHat guide, I can create a cluster that "presenting each cluster computer with the same view of the logical volumes".[1]</div><div><br></div><div>So I have:</div><div><br>
</div><div>node 1:</div><div> VG1 (local)</div><div> VG2 (node2)</div><div> VG3 (node3)</div><div><div><div><br></div><div><div>node 2:</div><div> VG1 (node1)</div><div> VG2 (local)</div><div> VG3 (node3)</div>
<div><br></div><div><div>node 3:</div><div> VG1 (node1)</div><div> VG2 (node2)</div><div> VG3 (local)</div><div><br></div><div>This should be what the RedHat CLVM guide stands for "the same view of the logical volumes"</div>
<div><br></div><div>From this point, Node1 is the shared storage. In this example it is visible from alle the cluster's nodes.</div><div><br></div><div>So if I stroke an "lvcreate", I have to see the newly create LV on the other nodes of the cluster.</div>
<div>It is true?</div><div><br></div><div>If this is true, gndb is not necessary and the layout becomes really simple.</div><div><br></div><div>Thanks for your time!</div><div><br></div><div>[1] <a href="http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5.2/html/Cluster_Logical_Volume_Manager/LVM_Cluster_Overview.html">http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5.2/html/Cluster_Logical_Volume_Manager/LVM_Cluster_Overview.html</a></div>
<div><br></div></div><div class="gmail_quote">2008/10/2 Xavier Montagutelli <span dir="ltr"><<a href="mailto:xavier.montagutelli@unilim.fr">xavier.montagutelli@unilim.fr</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="Ih2E3d">On Thursday 02 October 2008 12:28, Angelo Compagnucci wrote:<br>
> Ok, this could be clear, but in the Cluster_Logical_Volume_Manager.pdf I've<br>
> read (bottom of page 3):<br>
> "The clmvd daemon is the key clustering extension to LVM. The clvmd daemon<br>
> runs in each cluster computer and distributes LVM metadata updates in a<br>
> cluster, presenting each cluster computer with the same view of the logical<br>
> volumes"<br>
><br>
> This is a picture of wath I have in mind:<br>
<br>
</div>This picture doesn't show the difference between a GNBD server (which doesn't<br>
know anything about the use of the exported block device : it doesn't know<br>
the VG for example) and the GNBD clients (which actually use the block device<br>
as PV). May I add some layers ? Not exactly what I have in mind but I am not<br>
a ascii art expert :<br>
<div class="Ih2E3d"><br>
---------------------------<br>
| GFS filesystem |<br>
---------------------------<br>
| LV |<br>
---------------------------<br>
| VG |<br>
---------------------------<br>
| PV1 | PV2 | PV3 |<br>
</div>.---------.--------.--------.<br>
| CLVM |<br>
.---------.--------.--------.<br>
| cluster basis (dlm,...) |<br>
.---------.--------.--------.<br>
| Node4 | Node5 | Node6 |<br>
.---------.--------.--------.<br>
(Node4,5,6 have access to the three GNBD devices)<br>
\ | | /<br>
\___|_|___/<br>
/ | | \<br>
/ | | \<br>
/ | | \<br>
.---------.--------.---------.<br>
<div class="Ih2E3d">| GNBD1 | GNBD2 | GNBD3 |<br>
.---------.--------.---------.<br>
| hda1 | hda1 | hda1 |<br>
| Node1 | Node2 | Node3 |<br>
.---------.--------.---------.<br>
<br>
><br>
> In this case the clvm features are not useful because there is only one<br>
> machine (that could not be a node of a cluster) that have the lvm over GNBD<br>
> exported devices. So the nodes doesn't know nothing about the other nodes.<br>
<br>
</div>If your GNBD* devices are accessed by only one other node. But if the GNBD are<br>
served to multiple nodes (nodes4,5,6), then CLVM is useful.<br>
<div class="Ih2E3d"><br>
><br>
> Let's pose this situation:<br>
><br>
> -----------------------------------------------<br>
> | GFS |<br>
> -----------------------------------------------<br>
> | LV |<br>
> -----------------------------------------------<br>
> | VG1 | VG2 |<br>
> -----------------------------------------------<br>
> | PV1 | PV2 |<br>
> | Node1 | Node2 |<br>
> -----------------------------------------------<br>
> | CLVM coordinates |<br>
> -----------------------------------------------<br>
><br>
> In this situatuation makes sense to have a clustered lvm because if I have<br>
> to make some maintenance over VGs, CLVM can lock and unlock the interested<br>
> device.<br>
><br>
> Is this the correct behaviour??<br>
<br>
</div>Perhaps I miss your point, but it doesn't make sense if the block devices are<br>
local to each node. How could Node2 have access to the block device on Node1<br>
(showed as PV1) ?<br>
<br>
CLVM is useful only when you have a shared storage.<br>
<div class="Ih2E3d"><br>
> In the contrary, which is the CLVM role in a cluster?<br>
<br>
</div>>From what I know, CLVM protects the metadata parts of LVM on the shared<br>
storage. And when you make one operation the shared storage on one node (for<br>
example, create a new LV), all the nodes are aware of the change.<br>
<div><div></div><div class="Wj3C7c"><br>
<br>
><br>
><br>
> 2008/10/2 Xavier Montagutelli <<a href="mailto:xavier.montagutelli@unilim.fr">xavier.montagutelli@unilim.fr</a>><br>
><br>
> > On Wednesday 01 October 2008 17:39, Angelo Compagnucci wrote:<br>
> > > Hi to all,This is my first post on this list. Thanks in advance for<br>
> > > every answer.<br>
> > ><br>
> > > I've already read every guide in this matter, this is the list:<br>
> > ><br>
> > > Cluster_Administration.pdf<br>
> > > Cluster_Logical_Volume_Manager.pdf<br>
> > > Global_Network_Block_Device.pdf<br>
> > > Cluster_Suite_Overview.pdf<br>
> > > Global_File_System.pdf<br>
> > > CLVM.pdf<br>
> > > RedHatClusterAdminOverview.pdf<br>
> > ><br>
> > > The truth is that I've not clear a point about CLVM.<br>
> > ><br>
> > > Let's me make an example:<br>
> > ><br>
> > > In this example CLVM and the Cluster suite are fully running without<br>
> > > problems. Let's pose the same configuration of cluster.conf and<br>
> > > lvm.conf and the nodes of the cluster are joined and operatives.<br>
> ><br>
> > Does your example include a shared storage (GNBD, iSCSI, SAN, ...) ?<br>
> ><br>
> > > NODE1:<br>
> > ><br>
> > > pvcreate /dev/hda3<br>
> > ><br>
> > > NODE2:<br>
> > ><br>
> > > pvcreate /dev/hda2<br>
> > ><br>
> > > Let's pose that CLVM spans LVM metadata across the cluster, if I stroke<br>
> ><br>
> > the<br>
> ><br>
> > > command:<br>
> > ><br>
> > > pvscan<br>
> > ><br>
> > > I should see /dev/sda2 and /dev/sda3<br>
> > ><br>
> > > and then I can create a vg with<br>
> > ><br>
> > > vgcreate /dev/sda2 /dev/sda3 ...<br>
> > ><br>
> > > The question is: How LVM metadata sharing works? I have to use GNBD on<br>
> ><br>
> > the<br>
> ><br>
> > > row partion to share a device between nodes? I can create a GFS over a<br>
> > > spanned volume group? Are shareable only logical volumes?<br>
> ><br>
> > I have the feeling that something is not clear here. I am not an expert,<br>
> > but :<br>
> ><br>
> > GNBD is just a mean to export a block device on the IP network. A GNBD<br>
> > device<br>
> > is accessible to multiple nodes at the same time, and thus you can<br>
> > include that block device in a CLVM Volume Group. Instead of GNBD, you<br>
> > can also use any other shared storage (iSCSI, FC, ...). Be careful, from<br>
> > what I have understood, some SAN storage are not sharable between many<br>
> > hosts (NBD, AoE for example) !<br>
> ><br>
> > After that, you have the choice :<br>
> ><br>
> > - to make one LV with a shared filesystem (GFS). You can then mount the<br>
> > same<br>
> > filesystem on many nodes at the same time.<br>
> ><br>
> > - to make many LV with an ext3 / xfs / ... filesystem. But you then have<br>
> > to<br>
> > make sure that one LV is mounted on only one node at a given time.<br>
> ><br>
> > But the type of filesystem is independant, this is a higher component.<br>
> ><br>
> > In this picture, CLVM is only a low-level component, avoiding the<br>
> > concurrent<br>
> > access of many nodes on the LVM metadata written on the shared storage.<br>
> ><br>
> > The data are not "spanned" across the local storage of many nodes (well,<br>
> > I suppose you *could* do that, but you would need other tools / layers ?)<br>
> ><br>
> > Other point : if I remember correctly, the Red Hat doc says it's not<br>
> > recommended to use GFS on a node that exports a GNBD device. So if you<br>
> > use GNBD as a shared storage, I suppose it's better to specialize one or<br>
> > more nodes as GNBD "servers".<br>
> ><br>
> ><br>
> > HTH<br>
> ><br>
> > > Thanks for your answers!!<br>
> ><br>
> > --<br>
> > Xavier Montagutelli Tel : +33 (0)5 55 45 77 20<br>
> > Service Commun Informatique Fax : +33 (0)5 55 45 75 95<br>
> > Universite de Limoges<br>
> > 123, avenue Albert Thomas<br>
> > 87060 Limoges cedex<br>
> ><br>
> > --<br>
> > Linux-cluster mailing list<br>
> > <a href="mailto:Linux-cluster@redhat.com">Linux-cluster@redhat.com</a><br>
> > <a href="https://www.redhat.com/mailman/listinfo/linux-cluster" target="_blank">https://www.redhat.com/mailman/listinfo/linux-cluster</a><br>
<br>
</div></div>--<br>
<div><div></div><div class="Wj3C7c">Xavier Montagutelli Tel : +33 (0)5 55 45 77 20<br>
Service Commun Informatique Fax : +33 (0)5 55 45 75 95<br>
Universite de Limoges<br>
123, avenue Albert Thomas<br>
87060 Limoges cedex<br>
<br>
--<br>
Linux-cluster mailing list<br>
<a href="mailto:Linux-cluster@redhat.com">Linux-cluster@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/linux-cluster" target="_blank">https://www.redhat.com/mailman/listinfo/linux-cluster</a><br>
</div></div></blockquote></div><br></div></div></div></div>