<div>Hi Chrissie,</div>
<div> </div>
<div>Indeed, by default when creating the VG, it is clustered, thus when creating the LV it is active on all nodes.</div>
<div> </div>
<div>To avoid data corruption, I have re created the VG as non clustered (vgcreate -c n vgXX) then created the LV which got activated only on the node where it got created.</div>
<div> </div>
<div>Then changed the VG to clustered (vgchange -c y VGXX) and activated it exclusively on this node.</div>
<div> </div>
<div>But, I could reproduce the behaviour of bypassing the exclusive flag:</div>
<div> </div>
<div>On node B, re change the VG to non clustered though it is activated exclusively on node A.</div>
<div> </div>
<div>and then activate it on node B and it works.</div>
<div> </div>
<div>The thing I'm trying to point is that simply by erasing the clustered flag you can bypass the exclusive activation.</div>
<div> </div>
<div>I think a barrier is necessary to prevent this to happen, removing the clustered flag from a VG should be possible only if the node holding the VG exclusively is down (does the lock manager DLM report which node holds exclusively a VG ?)</div>
<div> </div>
<div>Thanks</div>
<div> </div>
<div>Brem</div>
<div> </div>
<div> </div>
<div> </div>
<div> </div>
<div> </div>
<div><br><br> </div>
<div><span class="gmail_quote">2009/7/21, Christine Caulfield <<a href="mailto:ccaulfie@redhat.com">ccaulfie@redhat.com</a>>:</span>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">Hiya,<br><br>I've just tried this on my cluster and it works fine.<br><br>What you need to remember is that lvcreate on one node will also activate the LV on all nodes in the cluster - it does an implicit lvchange -ay when you create it. What I can't explain is why vgchange -ae seemed to work fine on node A, it should give the same error as on node B because LVs are open shared on both nodes.<br>
<br>Its not clear to me when you tagged the VG as clustered, so that might be contributing to the problem. When I create a new VG on shared storage it automatically gets labelled clustered so I have never needed to do this explicitly. If you create a non-clustered VG you probably ought to deactivate it on all nodes first as it could mess up the locking otherwise. This *might* be the cause of your troubles.<br>
<br>The error on clvmd startup can be ignored. It's caused by clvmd ussing a background command with --no_locking so that it can check which LVs (if any) are already active and re-acquire locks for them<br><br>Sorry this isn't conclusive, The exact order in which things are happening is not clear to me.<br>
<br>Chrissie.
<div><span class="e" id="q_1229cbd467cb3a00_1"><br><br>On 07/21/2009 10:21 AM, brem belguebli wrote:<br></span></div>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<div><span class="e" id="q_1229cbd467cb3a00_3">Hi all,<br>I think there is something to clarify about using CLVM across a cluster<br>in a active/passive mode without GFS.<br> From my understanding, CLVM keeps LVM metadata coherent among the<br>
cluster nodes and provides a cluster wide locking mechanism that can<br>prevent any node from trying to activate a volume group if it has been<br>activated exclusively (vgchange -a e VGXXX) by another node (which<br>needs to be up).<br>
I have been playing with it to check this behaviour but it doesn't seem<br>to make what is expected.<br>I have 2 nodes (RHEL 5.3 X86_64, cluster installed and configured) , A<br>and B using a SAN shared storage.<br>I have a LUN from this SAN seen by both nodes, pvcreate'd<br>
/dev/mpath/mpath0 , vgcreate'd vg10 and lvcreate'd lvol1 (on one node),<br>created an ext3 FS on /dev/vg10/lvol1<br>CLVM is running in debug mode (clvmd -d2 ) (but it complains about<br>locking disabled though locking set to 3 on both nodes)<br>
On node A:<br> vgchange -c y vg10 returns OK (vgs --> vg10 1 1 0<br>wz--nc)<br> vgchange -a e --> OK<br> lvs returns lvol1 vg10 -wi-a-<br>On node B (while things are active on A, A is UP and member of the<br>
cluster ):<br> vgchange -a e --> Error locking on node B: Volume is busy on<br>another node<br> 1 logical volume(s) in volume group<br>"vg10" now active<br>It activates vg10 even if it sees it busy on another node .<br>
on B, lvs returns lvol1 vg10 -wi-a-<br>as well as on A.<br>I think the main problem comes from the fact that, as it is said when<br>starting CLVM in debug mode, WARNING: Locking disabled. Be careful!<br>This could corrupt your metadata.<br>
IMHO, the algorithm should be as follows:<br>VG is tagged as clustered (vgchange -c y VGXXX)<br>if a node (node P) tries to activate the VG exclusively (vgchange -a VGXXX)<br>ask the lock manager to check if VG is not already locked by another<br>
node (node X)<br>if so, check if node X is up<br>if node X is down, return OK to node P<br>else<br>return NOK to node P (explicitely that VG is held exclusively by node X)<br>Brem<br>PS: this shouldn't be a problem with GFS or other clustered FS (OCFS,<br>
etc...) as no node should try to activate exclusively any VG.<br><br><br></span></div>------------------------------------------------------------------------<br><br>--<br>Linux-cluster mailing list<br><a onclick="return top.js.OpenExtLink(window,event,this)" href="mailto:Linux-cluster@redhat.com" target="_blank">Linux-cluster@redhat.com</a><br>
<a onclick="return top.js.OpenExtLink(window,event,this)" href="https://www.redhat.com/mailman/listinfo/linux-cluster" target="_blank">https://www.redhat.com/mailman/listinfo/linux-cluster</a><br></blockquote><br>--<br>Linux-cluster mailing list<br>
<a onclick="return top.js.OpenExtLink(window,event,this)" href="mailto:Linux-cluster@redhat.com" target="_blank">Linux-cluster@redhat.com</a><br><a onclick="return top.js.OpenExtLink(window,event,this)" href="https://www.redhat.com/mailman/listinfo/linux-cluster" target="_blank">https://www.redhat.com/mailman/listinfo/linux-cluster</a><br>
</blockquote></div><br>