<html><head><style></style></head><body lang="EN-US" link="blue" vlink="purple" style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><div>Hello Muneendra,</div><div><br></div><div>On Mon, 2021-04-05 at 11:00 +0530, Muneendra Kumar M wrote:</div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div class="WordSection1"><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Hi Erwin,</span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Below are my replies.</span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p><div><p class="MsoNormal">On Thu, 2021-04-01 at 10:16 +0000, Martin Wilck wrote:</p></div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div><p class="MsoNormal">On Thu, 2021-04-01 at 12:48 +1000, Erwin van Londen wrote:</p></div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div><p class="MsoNormal"> </p></div><div><p class="MsoNormal">Benjamin has added a marginal_path group(multipath marginal</p></div><div><p class="MsoNormal">pathgroups) in</p></div><div><p class="MsoNormal">the dm-multipath.</p></div><div><p class="MsoNormal"><a href="https://patchwork.kernel.org/project/dm-devel/cover/1564763622-31752-1-git">https://patchwork.kernel.org/project/dm-devel/cover/1564763622-31752-1-git</a></p></div><div><p class="MsoNormal"><a href="mailto:-send-email-bmarzins@redhat.com">-send-email-bmarzins@redhat.com</a>/</p></div><div><p class="MsoNormal"> </p></div><div><p class="MsoNormal">One of the intention of the Benjamin's patch (support for maginal</p></div><div><p class="MsoNormal">path) is</p></div><div><p class="MsoNormal">to support for the FPIN events we receive from fabric.</p></div><div><p class="MsoNormal">On receiving the fpin-li our intention was to place all the paths</p></div><div><p class="MsoNormal">that</p></div><div><p class="MsoNormal">are affected into the marginal path group.</p></div></blockquote><div><p class="MsoNormal">I think this should all be done in kernel space as we're talking sub-</p></div><div><p class="MsoNormal">millisecond timings here when it comes to fpins and the reaction time</p></div><div><p class="MsoNormal">expected. I may be wrong but I'll leave that up to you.</p></div></blockquote><div><p class="MsoNormal"> </p></div><div><p class="MsoNormal">Sub-ms latency is impossible with this setup (kernel -> broadcom FC</p></div><div><p class="MsoNormal">daemon -> multipathd -> kernel). It's only suitable for "fatal" FPINs</p></div><div><p class="MsoNormal">that would suggest taking a path offline on the time scale of minutes.</p></div><div><p class="MsoNormal">I suppose that would work well for LN FPINs, but not for the other</p></div><div><p class="MsoNormal">types.</p></div></blockquote><div><p class="MsoNormal"><span style="color:#1f497d">>></span>I agree. I was hoping the FC drivers would be able to play a role in this and provide a direct hook into the FPIN notifications in such a way that userspace daemons would not be required and multipath would <span style="color:#1f497d">>></span>be able to play a direct role here.</p></div><div><p class="MsoNormal"><span style="color:#1f497d">>></span>When it comes to latency in a san we're indeed talking about sub-ms when it comes to impacting other parts of the fabrics having an immediate effect on multiple initiators and targets due to the shared nature <span style="color:#1f497d">>></span>of the beast.<span style="color:#1f497d"></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">>> </span></p></div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div><p class="MsoNormal"> </p></div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div><p class="MsoNormal"> </p></div><div><p class="MsoNormal">On receiving the congestion notifications our intention is to</p></div><div><p class="MsoNormal">slowdown the</p></div><div><p class="MsoNormal">work load gradually from the host until it stops receiving the</p></div><div><p class="MsoNormal">congestion</p></div><div><p class="MsoNormal">notifications.</p></div><div><p class="MsoNormal">We need to validate the same how we can achieve the same of</p></div><div><p class="MsoNormal">decreasing the</p></div><div><p class="MsoNormal">workloads with the help of dm-multipath.</p></div></blockquote><div><p class="MsoNormal">Would it be possible to piggyback on the service time path selector</p></div><div><p class="MsoNormal">in this when it pertains latency? </p></div></blockquote><div><p class="MsoNormal"> </p></div><div><p class="MsoNormal">Not on service-time itself, but someone could write a new path selector</p></div><div><p class="MsoNormal">algorithm. IMO we'd still have the problem that this would be seen as a</p></div><div><p class="MsoNormal">layering violation. In the long run dm-mpath may need to add transport-</p></div><div><p class="MsoNormal">specific callbacks. But for a proof-of-concept, a selector algorithm</p></div><div><p class="MsoNormal">with layering violations would be ok, I believe.</p></div></blockquote><div><p class="MsoNormal"><span style="color:#1f497d">>></span>Is that an offer of volunteering?? <img border="0" width="16" height="16" id="_x0000_i1025" src="cid:image001.png@01D72A0A.4F10A750" alt=":-)"></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">[Muneendra]To address all the issues we are planning to come up with new dm-path selector algorithm which should address </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">the above concerns where FC drivers will do a direct hook into the FPIN notifications in such a way that userspace daemons would not be required and multipath would be able to play a </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">direct role here.</span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Will come up with more details regarding the new dm-path selector algorithm for FPIN notifications.</span></p><p class="MsoNormal"></p></div></div></blockquote><div><br></div><div><div>That is awesome. Thank you very much. If you need any input or feedback then please let me know.</div></div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div class="WordSection1"><div><p class="MsoNormal"> </p><p class="MsoNormal">Regards,</p><p class="MsoNormal">Muneendra.</p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p></div></div><div><br></div><div><span style="background-color:rgb(255,255,255)"><font size="2">This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it.</font></span></div></blockquote></body></html>