<div dir="ltr"><div><div><div><div>That patch did not resolve the issue.<br><br></div>But I did find one useful information while testing things.<br></div>If I disable CSM, board will boot using GPU in the third slot connected to chipset.<br></div>This is probably due to it having the lowest PCI id.<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 27, 2017 at 11:33 PM, Nick Sarnie <span dir="ltr"><<a href="mailto:commendsarnex@gmail.com" target="_blank">commendsarnex@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><div><div class="h5"><div class="gmail_quote">On Mon, Mar 27, 2017 at 5:12 PM, Stano Lano <span dir="ltr"><<a href="mailto:chl.xxx5x@gmail.com" target="_blank">chl.xxx5x@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Following errors were seen in all over the output:<br> AMD-Vi: Completion-Wait loop timed out<br></div>In the end I had kernel panic due to problems with IRQ<br><div><br>When I connected serial console I noticed that the problem starts at GPU that was connected to chipset slot:<br><br>[ 2.337432] AMD-Vi: IOMMU performance counters supported<br>[ 2.354206] iommu: Adding device 0000:00:01.0 to group 0<br>[ 2.359543] iommu: Adding device 0000:00:01.1 to group 0<br>[ 2.364882] iommu: Adding device 0000:00:01.3 to group 0<br>[ 2.370264] iommu: Adding device 0000:00:02.0 to group 1<br>[ 2.375659] iommu: Adding device 0000:00:03.0 to group 2<br>[ 2.381001] iommu: Adding device 0000:00:03.1 to group 2<br>[ 2.386334] iommu: Adding device 0000:00:03.2 to group 2<br>[ 2.391724] iommu: Adding device 0000:00:04.0 to group 3<br>[ 2.397114] iommu: Adding device 0000:00:07.0 to group 4<br>[ 2.402450] iommu: Adding device 0000:00:07.1 to group 4<br>[ 2.407845] iommu: Adding device 0000:00:08.0 to group 5<br>[ 2.413183] iommu: Adding device 0000:00:08.1 to group 5<br>[ 2.418580] iommu: Adding device 0000:00:14.0 to group 6<br>[ 2.423914] iommu: Adding device 0000:00:14.3 to group 6<br>[ 2.429324] iommu: Adding device 0000:00:18.0 to group 7<br>[ 2.434664] iommu: Adding device 0000:00:18.1 to group 7<br>[ 2.439999] iommu: Adding device 0000:00:18.2 to group 7<br>[ 2.445335] iommu: Adding device 0000:00:18.3 to group 7<br>[ 2.450667] iommu: Adding device 0000:00:18.4 to group 7<br>[ 2.455999] iommu: Adding device 0000:00:18.5 to group 7<br>[ 2.461334] iommu: Adding device 0000:00:18.6 to group 7<br>[ 2.466673] iommu: Adding device 0000:00:18.7 to group 7<br>[ 2.472007] iommu: Adding device 0000:01:00.0 to group 0<br>[ 2.477346] iommu: Adding device 0000:03:00.0 to group 0<br>[ 2.482681] iommu: Adding device 0000:03:00.1 to group 0<br>[ 2.488013] iommu: Adding device 0000:03:00.2 to group 0<br>[ 2.493346] iommu: Adding device 0000:1d:00.0 to group 0<br>[ 2.498679] iommu: Adding device 0000:1d:04.0 to group 0<br>[ 2.504010] iommu: Adding device 0000:1d:06.0 to group 0<br>[ 2.509342] iommu: Adding device 0000:1d:07.0 to group 0<br>[ 2.642002] AMD-Vi: Completion-Wait loop timed out<br>[ 2.646807] iommu: Adding device 0000:21:00.0 to group 0<br>[ 2.779532] AMD-Vi: Completion-Wait loop timed out<br>[ 2.784341] iommu: Adding device 0000:21:00.1 to group 0<br>[ 2.916985] AMD-Vi: Completion-Wait loop timed out<br>[ 2.921788] iommu: Adding device 0000:25:00.0 to group 0<br>[ 3.054520] AMD-Vi: Completion-Wait loop timed out<br>[ 3.059326] iommu: Adding device 0000:26:00.0 to group 0<br>[ 3.191976] AMD-Vi: Completion-Wait loop timed out<br>[ 3.196781] iommu: Adding device 0000:27:00.0 to group 0<br>[ 3.329510] AMD-Vi: Completion-Wait loop timed out<br>[ 3.334311] iommu: Adding device 0000:28:04.0 to group 0<br>[ 3.466965] AMD-Vi: Completion-Wait loop timed out<br>[ 3.471766] iommu: Adding device 0000:29:00.0 to group 2<br>[ 3.516568] iommu: Adding device 0000:29:00.1 to group 2<br>[ 3.649180] AMD-Vi: Completion-Wait loop timed out<br>[ 3.653987] iommu: Adding device 0000:2a:00.0 to group 2<br>[ 3.786686] AMD-Vi: Completion-Wait loop timed out<br>[ 3.791490] iommu: Adding device 0000:2a:00.1 to group 2<br>[ 3.924102] AMD-Vi: Completion-Wait loop timed out<br>[ 3.928906] iommu: Adding device 0000:2b:00.0 to group 4<br>[ 4.061614] AMD-Vi: Completion-Wait loop timed out<br>[ 4.066420] iommu: Adding device 0000:2b:00.2 to group 4<br>[ 4.199051] AMD-Vi: Completion-Wait loop timed out<br>[ 4.203854] iommu: Adding device 0000:2b:00.3 to group 4<br>[ 4.336576] AMD-Vi: Completion-Wait loop timed out<br>[ 4.341379] iommu: Adding device 0000:2c:00.0 to group 5<br>[ 4.474015] AMD-Vi: Completion-Wait loop timed out<br>[ 4.478816] iommu: Adding device 0000:2c:00.2 to group 5<br>[ 4.518478] iommu: Adding device 0000:2c:00.3 to group 5<br>[ 4.651291] AMD-Vi: Completion-Wait loop timed out<br><br><br>For now my workaround is to use Radeon HD 6450 for my kodi seat.<br></div><div>Not ideal but it will do.<br></div><div>From the 3 GPUs I tested RX 460 & R7 260 would not allow me to boot due to the described problem.<br></div></div><div class="m_5228788957807098421gmail-HOEnZb"><div class="m_5228788957807098421gmail-h5"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 27, 2017 at 5:12 PM, Nick Sarnie <span dir="ltr"><<a href="mailto:commendsarnex@gmail.com" target="_blank">commendsarnex@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">The ticket is still open, but it wouldn't fix a hang on startup. Do you see any errors in dmesg?<br></div><div class="m_5228788957807098421gmail-m_266893697011342239HOEnZb"><div class="m_5228788957807098421gmail-m_266893697011342239h5"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 27, 2017 at 7:28 AM, Stano Lano <span dir="ltr"><<a href="mailto:chl.xxx5x@gmail.com" target="_blank">chl.xxx5x@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div><div><div><h1></h1> Hello,<br><br></div><div>did you have any luck with the ASUS support.<br></div><div>I should have the same ASUS Prime X370-Pro.<br>But when I enable IOMMU I am not able to boot, when the IOMMU is disable I have no problem to boot.<br><br></div><div>I am on BIOS 0511 and kernel 4.10.5<br></div><div>Tried with 0502, 0504 & 0511 BIOSes but no luck with any of them.<br></div><div>Also tried with Fedora 25, 25 beta & Ubuntu 17.04 beta. Same behavior.<br></div><div><br></div><div>Thanks<br></div><div>Stano<br></div><div><br></div><div><hr> <ul><li><i>From</i>: Nick Sarnie <commendsarnex gmail com></li><li><i>To</i>: Alex Williamson <alex l williamson gmail com></li><li><i>Cc</i>: vfio-users <vfio-users redhat com></li><li><i>Subject</i>: Re: [vfio-users] Ryzen/X370 Chipset and IOMMU Groups</li><li><i>Date</i>: Fri, 10 Mar 2017 13:53:26 -0500</li></ul><br>Yeah, that is unfortunate.<br><br></div>Thanks for helping with this issue. I've sent a ticket to Asus, but I'm not expecting much. Then again, I felt the same way and Gigabyte actually sent me a fixed bios, so who knows.<br><br></div>I'll keep you updated.<br></div><div><br></div><div>Thanks again,<br></div><div>Sarnex<br></div></div><br><div class="gmail_quote">On Fri, Mar 10, 2017 at 1:37 PM, Alex Williamson <span dir="ltr"><<a rel="nofollow" href="mailto:alex%20l%20williamson%20gmail%20com" target="_blank">alex l williamson gmail com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="m_5228788957807098421gmail-m_266893697011342239m_5690907581944691599m_-9144441347565743421gmail-">On Fri, Mar 10, 2017 at 11:05 AM, Nick Sarnie <span dir="ltr"><<a rel="nofollow" href="mailto:commendsarnex%20gmail%20com" target="_blank">commendsarnex gmail com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div><div>Hi Alex,<br><br></div>I don't see either of the options. I couldn't find Common Options at all, and here's a screenshot of the CBS settings:<br><br><a rel="nofollow" href="https://i.imgur.com/9hQUHX0.jpg" target="_blank">https://i.imgur.com/9hQUHX0.jp<wbr>g</a><br><br></div>Is this something I should ask Asus to add?<br></div></div></div></blockquote><div><br></div></span><div>I suppose it wouldn't hurt to try to start the discussion with Asus. The video I found was this one:</div><div><br></div><div><a rel="nofollow" href="https://youtu.be/pipR5xhrLo0?t=20" target="_blank">https://youtu.be/pipR5xhrLo0?t<wbr>=20</a><br></div><div><br></div><div>At that start time you can see an NBIO Common Options menu on an ASRock system, but I never saw him open it and I couldn't find any documentation on what might be in there in an asrock mb manual (not an endorsement for asrock, perhaps they just have a BIOS more similar to the AMD sample implementation). If AMD put it into a menu of debug options, it's really no surprise that consumer firmware dropped it. Too bad. This feels like a repeat of the difficulty we had trying to find motherboards that allowed the IOMMU to be enabled when AMD-Vi came out. I wonder if AVIC requires yet another BIOS option that consumers will need to gamble with.</div></div></div></div> </blockquote></div></div> </blockquote></div><br></div> </div></div></blockquote></div><br></div> </div></div></blockquote></div></div></div>Apparently this will fix it<br><br><a href="https://lists.freedesktop.org/archives/amd-gfx/2017-March/006862.html" target="_blank">https://lists.freedesktop.org/<wbr>archives/amd-gfx/2017-March/<wbr>006862.html</a><br></div></div> </blockquote></div><br></div>