[dm-devel] dm-multipath - IO queue dispatch based on FPIN Congestion/Latency notifications.

[Erwin van Londen]

Hello Muneendra,

On Mon, 2021-04-05 at 11:00 +0530, Muneendra Kumar M wrote:
> Hi Erwin,
> Below are my replies.
>  
> On Thu, 2021-04-01 at 10:16 +0000, Martin Wilck wrote:
> > On Thu, 2021-04-01 at 12:48 +1000, Erwin van Londen wrote:
> > > >  
> > > > Benjamin has added a marginal_path group(multipath marginal
> > > > pathgroups) in
> > > > the dm-multipath.
> > > >
> https://patchwork.kernel.org/project/dm-devel/cover/1564763622-31752-1-git
> > > > -send-email-bmarzins at redhat.com/
> > > >  
> > > > One of the intention of the Benjamin's patch (support for
> > > > maginal
> > > > path) is
> > > > to support for the FPIN events we receive from fabric.
> > > > On receiving the fpin-li our intention was to  place all the
> > > > paths
> > > > that
> > > > are affected into the marginal path group.
> > > I think this should all be done in kernel space as we're talking
> > > sub-
> > > millisecond timings here when it comes to fpins and the reaction
> > > time
> > > expected. I may be wrong but I'll leave that up to you.
> >  
> > Sub-ms latency is impossible with this setup  (kernel -> broadcom
> > FC
> > daemon -> multipathd -> kernel). It's only suitable for "fatal"
> > FPINs
> > that would suggest taking a path offline on the time scale of
> > minutes.
> > I suppose that would work well for LN FPINs, but not for the other
> > types.
> 
> >>I agree. I was hoping the FC drivers would be able to play a role
> in this and provide a direct hook into the FPIN notifications in such
> a way that userspace daemons would not be required and multipath
> would >>be able to play a direct role here.
> >>When it comes to latency in a san we're indeed talking about sub-ms
> when it comes to impacting other parts of the fabrics having an
> immediate effect on multiple initiators and targets due to the shared
> nature >>of the beast.
> >> 
> >  
> > > >  
> > > > On receiving the congestion notifications our intention is to
> > > > slowdown the
> > > > work load gradually from the host until it stops receiving the
> > > > congestion
> > > > notifications.
> > > > We need to validate the same how we can achieve the same of
> > > > decreasing the
> > > > workloads with the help of dm-multipath.
> > > Would it be possible to piggyback on the service time path
> > > selector
> > > in this when it pertains latency?  
> >  
> > Not on service-time itself, but someone could write a new path
> > selector
> > algorithm. IMO we'd still have the problem that this would be seen
> > as a
> > layering violation. In the long run dm-mpath may need to add
> > transport-
> > specific callbacks. But for a proof-of-concept, a selector
> > algorithm
> > with layering violations would be ok, I believe.
> 
> >>Is that an offer of volunteering?? :-)
> [Muneendra]To address all the issues we are planning to come up with
> new dm-path selector algorithm which should address 
> the above concerns where FC drivers will do a direct hook into the
> FPIN notifications in such a way that userspace daemons would not be
> required and multipath would be able to play a 
> direct role here.
> Will come up with more details regarding the new dm-path selector
> algorithm for FPIN notifications.

That is awesome. Thank you very much. If you need any input or feedback
then please let me know.
>  
> Regards,
> Muneendra.
>  
> 
> This electronic communication and the information and any files
> transmitted with it, or attached to it, are confidential and are
> intended solely for the use of the individual or entity to whom it is
> addressed and may contain information that is confidential, legally
> privileged, protected by privacy laws, or otherwise restricted from
> disclosure to anyone else. If you are not the intended recipient or
> the person responsible for delivering the e-mail to the intended
> recipient, you are hereby notified that any use, copying,
> distributing, dissemination, forwarding, printing, or copying of this
> e-mail is strictly prohibited. If you received this e-mail in error,
> please return the e-mail to the sender, delete it from your computer,
> and destroy any printed copy of it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/dm-devel/attachments/20210405/b1f61fed/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 871 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/dm-devel/attachments/20210405/b1f61fed/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://listman.redhat.com/archives/dm-devel/attachments/20210405/b1f61fed/attachment.sig>