[lvm-devel] Question: the failure handling for sanlock

[Leo Yan]

Hi David,

On Mon, Feb 15, 2021 at 10:31:27AM -0600, David Teigland wrote:
> On Sun, Feb 14, 2021 at 06:13:03PM +0800, Leo Yan wrote:
> >  538         /*
> >  539          * FIXME: here is where we should implement a strong form of
> >  540          * blkdeactivate, and if it completes successfully, automatically call
> >  541          * do_drop() afterward.  (The drop step may not always be necessary
> >  542          * if the lvm commands run while shutting things down release all the
> >  543          * leases.)
> >  544          *
> >  545          * run_strong_blkdeactivate();
> >  546          * do_drop();
> >  547          */
> > 
> > My first question is why lvmlockctl comments out the failure handling code so
> > cannot automatically deactivate drives and cleanup VG/LV locks by default?
> 
> Only because that forceful deactivate has not been implemented.
> 
> > The second question is what's a suggested flow for failure handling for sanlock?
> > If I understand correctly, we can rely on the command "blkdeactivate" to
> > deactivates block devices; but if we deactivate the whole block device, it
> > might also hurt other VGs/LVs resident on the same drive.  So should we firstly
> > deactivate VGs or LVs with "vgchange" or "lvchange" commands rather than
> > deactivate the whole block device?  We might consider there have no chance to
> > access drive for the drive or fabric malfunction, so cannot make succuss for
> > commands "vgchange" and "lvchange".
> 
> We only need to forcefully deactivate LVs in the failed VG.  The lvmlockd
> man page section "sanlock lease storage failure" describes how this
> process can work manually, and mentions doing it automatically in the
> future.  In the manual process you'd try to stop applications using the
> fs, unmount file systems on the LVs, then try to deactivate the LVs.
> Since there are storage issues, the unmount and deactivate might hang, in
> which case you could try dmsetup wipe_table on the LVs.  You might just
> skip immediately to that last step since there's not a lot of time.
> 
> There was a script written in 2017 to automate that final dmsetup step.
> It was finished or very close, but I don't remember what kept us from
> including it.  An idea to make dmsetup do more of the work perhaps left us
> unsure what to do.  Here is the script and some discussion:
> https://listman.redhat.com/archives/lvm-devel/2017-September/msg00011.html
> 
> There was also a small lvmlockctl patch to replace the
> run_strong_blkdeactivate() comment with running the script.

Yes, I went through the two patches, one is for the script
blkdeactivate.sh and another is for lvmlockctl.

And I saw you suggested to use the dmsetup commands:

  dmsetup info -c -S "uuid=~LVM && vgname=$VG && lv_layer=\"\"" \
        -o name --noheadings | xargs dmsetup wipe_table

> If you can get this to work for your case I'd like to get it included in
> lvm.

I will verify the changes and share back the result.

Thank you for suggestions!

Leo