[dm-devel] [PATCH] multipathd: handle fpin events

[Martin Wilck]

On Mon, 2021-12-13 at 15:40 +1000, Erwin van Londen wrote:
> Hello Martin,
> 
> That is exactly the reason why we would need to modularise this.
> Access characteristics between different transport protocols can
> differ significantly and thus are susceptible to different
> indicators, tolerances and other variations that would need a
> different config setup. You don't really want to mix and match config
> options between the various options as that would significantly
> increase complexity and thus make the setup more errorprone. Things
> may often seem very clear cut from a develop(ers/ment) perspective
> however from a system administration side it gets often very
> confusing when even the smallest things start to contradict each
> other.
> 
> When tcp packets timeout from a iscsi based device we're 100% relying
> on tcp to sort itself out based on RTT values and slow start
> behaviour whereas FC is far more reliant on all upper levels from
> scsi/nvme-o-f side. These tolerances should be configurable
> independantly.

Hm, I don't know. Configuration is yet another issue. The nice thing
about Muneendra's FPIN patches it that they don't need _any_ device-
specific configuration on multipathd's part. The notifications are
received from the fabric, and multipathd just passively reacts on them.
If there's any configuration, it's done on the fabric level, which is
where it belongs IMHO. 

Consider the complex configuration parameters that we currently use for
marginal path detection ("marginal_path_err_rate_threshold" etc.). We
allow configuring them at all levels that multipathd supports
(defaults, device, multipath, overrides). I wonder if anybody is using
multiple sets of these parameters for different arrays (or even
multipaths). It sounds daunting to figure out the right settings for
the given environment, and doing that separately for different targets
seems almost impossible. Since this feature was first created, I've
thought we need a simple setup that "just works" for most users, most
of the time, with just one or even zero parameters, rather than this
intimidating complexity. That would probably boost adoption of this
feature quite a bit. I can't create a simplified parameter set because
I don't have access to networks suitable for testing and deriving
proper parameter sets.

Moreover, I believe that configuring this for individual multipath maps
or storage array vendor/product makes very little sense in general. As
you say correctly, what matters here is properties of the transport and
fabric (type, site-specific configuration, and topology, like
distance). We don't support fabric-specific configuration currently,
and we can't support anything site-specific except via the "defaults"
configuration section.

What I can currently conceive of for the future is this:
 - use FPIN for FC (perhaps just for certain ports)
 - (maybe) use some other to-be-developed notification mechanism for
other transports (ideally also configured in the fabric, with zero
config parameters in multipath-tools)
 - use either marginal_path_err_rate_threshold or nothing for the rest,
but just with one set of parameters.

Regards
Martin