[dm-devel] [PATCH] multipath-tools: add more info about max_sectors_kb in multipath.conf.5

[Martin Wilck]

On Sat, 2021-03-06 at 16:30 +0100, Xose Vazquez Perez wrote:
> Change of the default value in the kernel:
>  
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d2be537c3ba35
> 
> To query the device:
> sg_inq -p 0xb0 /dev/sdX | grep "[ml] transfer length:"
> 
> 
> Note: some arrays does not report any value.
>       3PAR: 65534 blocks
>       RDAC: not reported
>       Hitachi VSP: query error
> 
> Cc: Martin Wilck <mwilck at suse.com>
> Cc: Benjamin Marzinski <bmarzins at redhat.com>
> Cc: Christophe Varoqui <christophe.varoqui at opensvc.com>
> Cc: DM-DEVEL ML <dm-devel at redhat.com>
> Signed-off-by: Xose Vazquez Perez <xose.vazquez at gmail.com>
> ---
>  multipath/multipath.conf.5 | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/multipath/multipath.conf.5 b/multipath/multipath.conf.5
> index 8ef3a747..21963813 100644
> --- a/multipath/multipath.conf.5
> +++ b/multipath/multipath.conf.5
> @@ -1240,9 +1240,11 @@ The default is: \fB0\fR
>  .B max_sectors_kb
>  Sets the max_sectors_kb device parameter on all path devices and the
> multipath
>  device to the specified value.
> +If the storage device(array, hard disk, ...) does not export the
> value to the
> +system, the kernel sets it to 512(kernel < 4.3) or 1024(kernel >=
> 4.3).

I don't think this information belongs into the multipath.conf(5) man
page. We shouldn't start documenting changes of kernel defaults in this
man page, unless we intend to continue doing so and keep the
information up-to-date, and I don't think we can promise that. Also,
the kernel major/minor number information may easily be wrong for
distribution kernels that have backported the respective change. Kernel
behavior should be documented in the kernel documentation as provided
by the distribution.

The way the kernel handles max_sectors and max_hw_sectors is actually
pretty complex, and I wouldn't even attempt to try and explain it in
this man page.

That said, what's actually important for users to realize is the fact
that changes of max_sectors_kb don't propagate up the storage stack.
Once a DM device is first created, it will inherit the max_sectors_kb
value of the dm target(s). But if the value is changed for low-level
devices later, upper-level devices won't see the change. A typical
mistake is to run an udev rule decreasing max_sectors_kb for a SCSI
device after a multipath map has already been set up on that SCSI
device (e.g. in the initrd). The  multipath map will have a larger
max_sectors_kb value than the path device, and IO may actually fail for
that reason.

Thus it's generally recommended to use multipath.conf for applying
max_sectors_kb changes. If other devices (e.g. LVM) are stacked on top
of multipath, the same reasoning applies; therefore max_sectors_kb
shouldn't be changed at runtime without double-checking the entire
storage stack.

Regards,
Martin

>  .RS
>  .TP
> -The default is: \fB<device dependent>\fR
> +The default is: \fB<device dependent>\fR, or kernel provided if
> there is no value.