<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Nov 27, 2014 at 3:33 PM, Zdenek Kabelac <span dir="ltr"><<a href="mailto:zkabelac@redhat.com" target="_blank">zkabelac@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Dne 27.11.2014 v 15:26 Gianluca Cecchi napsal(a):<span class=""><br> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> Hello,<br> I'm unable to deactivate an lvm.<br> <br> My system is RHEL 6.5 with lvm2-2.02.100-8.el6.x86_64 and kernel<br> 2.6.32-431.29.2.el6.x86_64<br> <br> I get error code 5 with message<br> Logical volume VG_AAA_TEMP/LV_AAA_TEMP in use.<br> <br> You can find output of<br> lvchange -d -d -d -d -d -d -an VG_AAA_TEMP/LV_AAA_TEMP<br> here:<br> <a href="https://drive.google.com/file/d/0BwoPbcrMv8mvTjlBMkRUbG9nczA/view?usp=sharing" target="_blank">https://drive.google.com/file/<u></u>d/<u></u>0BwoPbcrMv8mvTjlBMkRUbG9nczA/<u></u>view?usp=sharing</a><br> <br> </blockquote> <br></span> Not really accessible.<br></blockquote><div><br></div><div>strange, do you mean the google docs link?</div><div>I tried with a browser without access to any gmail account and I'm able to download it....</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> <br> But anyway - if you have problem with 'semaphore' resouces - you could 'recycle' old ones -<br> <br> 'dmsetup udevcomplete_all'<br></blockquote><div><br></div><div>This is actually a production server with many other LVs... Is there any drawback in the command above?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> <br> Of course it's hard to guess what experiments are you doing and would could lead to uncompleted cockies (stuck udev scans)<br></blockquote><div><br></div><div>Actually no experiment at all.</div><div>The node is part of a rhel 2-nodes production cluster with HA_LVM based services.</div><div>We need to relocate many services to the other node for a planned maintenance, but it seems that this one is able to stop the lvm resources, but not cleanly deactivate the LVs. We get messages like</div><div><br></div><div><div>Nov 26 17:35:29 orapr2 rgmanager[5765]: [lvm] Deactivating VG_AAA_TEMP/LV_AAA_TEMP</div><div>Nov 26 17:35:29 orapr2 rgmanager[5786]: [lvm] Making resilient : lvchange -an VG_AAA_TEMP/LV_AAA_TEMP</div><div>Nov 26 17:35:29 orapr2 rgmanager[5809]: [lvm] Resilient command: lvchange -an VG_AAA_TEMP/LV_AAA_TEMP --config devices{filter=["a|/dev/ma</div><div>pper/360a9800037543544465d424</div><div>Nov 26 17:35:34 orapr2 rgmanager[5883]: [lvm] lv_exec_resilient failed</div><div>Nov 26 17:35:34 orapr2 rgmanager[5908]: [lvm] lv_activate_resilient stop failed on VG_AAA_TEMP/LV_AAA_TEMP</div><div>Nov 26 17:35:34 orapr2 rgmanager[5928]: [lvm] Unable to deactivate VG_AAA_TEMP/LV_AAA_TEMP</div><div>Nov 26 17:35:34 orapr2 rgmanager[5948]: [lvm] Failed to stop VG_AAA_TEMP/LV_AAA_TEMP</div><div>Nov 26 17:35:34 orapr2 rgmanager[5968]: [lvm] Attempting cleanup of VG_AAA_TEMP</div><div>Nov 26 17:35:34 orapr2 rgmanager[5989]: [lvm] VG_AAA_TEMP now consistent</div><div>Nov 26 17:35:34 orapr2 rgmanager[6013]: [lvm] Deactivating VG_AAA_TEMP/LV_AAA_TEMP</div><div>Nov 26 17:35:34 orapr2 rgmanager[6033]: [lvm] Making resilient : lvchange -an VG_AAA_TEMP/LV_AAA_TEMP</div><div>Nov 26 17:35:35 orapr2 rgmanager[6056]: [lvm] Resilient command: lvchange -an VG_AAA_TEMP/LV_AAA_TEMP --config devices{filter=["a|/dev/ma</div><div>pper/360a9800037543544465d424</div><div>Nov 26 17:35:39 orapr2 rgmanager[6648]: [lvm] lv_exec_resilient failed</div><div>Nov 26 17:35:40 orapr2 rgmanager[6670]: [lvm] lv_activate_resilient stop failed on VG_AAA_TEMP/LV_AAA_TEMP</div><div>Nov 26 17:35:40 orapr2 rgmanager[6690]: [lvm] Unable to deactivate VG_AAA_TEMP/LV_AAA_TEMP</div><div>Nov 26 17:35:40 orapr2 rgmanager[6710]: [lvm] Failed second attempt to stop VG_AAA_TEMP/LV_AAA_TEMP</div><div>Nov 26 17:35:40 orapr2 rgmanager[20260]: stop on lvm "LV_AAA_TEMP" returned 1 (generic error)</div><div>Nov 26 17:35:40 orapr2 rgmanager[20260]: Marking service:AAA as 'disabled', but some resources may still be allocated!</div><div>Nov 26 17:35:40 orapr2 rgmanager[20260]: Service service:AAA is disabled</div></div><div><br></div><div>And of course the other node then is unable to activate the service due to LV maintained open from the first one:<br></div><div><br></div><div><div>Nov 26 17:35:40 orapr1 rgmanager[18596]: Starting disabled service service:AAA</div><div>Nov 26 17:35:41 orapr1 rgmanager[31420]: [lvm] Someone else owns this logical volume</div><div>Nov 26 17:35:41 orapr1 rgmanager[18596]: start on lvm "LV_AAA_TEMP" returned 1 (generic error)</div><div>Nov 26 17:35:41 orapr1 rgmanager[18596]: #68: Failed to start service:AAA; return value: 1</div></div><div><br></div><div>So I'm trying to reproduce the cluster command to see how to clean the situation, using this particular service (named AAA) that is not critical as the other ones running on the node </div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> <br> Do you happen to have some suspend devices in your table ?<br> (dmsetup info -c should show them)</blockquote><div><br></div><div>It seems not so. Only (L)ive states...</div><div><br></div><div><div>[root@orapr2 ~]# dmsetup info -c | awk '{print $4}' | sort | uniq -c</div><div> 77 L--w</div><div> 1 Stat</div></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span class=""><br> <br> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> [root@orapr2 ~]# lvs VG_AAA_TEMP/LV_AAA_TEMP<br> LV VG Attr LSize Pool Origin Data% Move Log<br> Cpy%Sync Convert<br> LV_AAA_TEMP VG_AAA_TEMP -wi-ao---- 1020.00m<br> <br> How can I see the responsible for the reference that apparently keeps it open?<br> <br> Open count: 1<br> so I can check and eventually fix??<br> <br> </blockquote> <br></span> dmsetup ls --tree<br> <br> is usually good in shows deps between devs (i.e. target A holds target B)<br> <br> Regards<br> <br> Zdenek<br> <br> <br></blockquote><div><br></div><div>it returns no particular output related</div><div>...</div><div> VG_AAA_TEMP-LV_AAA_TEMP (253:49)</div><div> └─360a9800037543544465d424130533177 (253:4)</div><div> ├─ (130:128)</div><div> ├─ (129:32)</div><div> ├─ (68:48)</div><div> ├─ (8:96)</div><div> ├─ (8:288)</div><div> ├─ (133:224)</div><div> ├─ (69:192)</div><div> └─ (66:160)</div><div>...</div><div><br></div><div>BTW: I'm testing this one but it seems that the problem is general, in the sense that each LV gets this kind of behaviour trying to deactivating it...</div><div><br></div><div>Thanks in advance for any other insight and let me know if I can send it the debug log of lvchange command in case you are not yet able to access it...</div><div><br></div><div>Gianluca</div></div></div></div>