[vfio-users] Ryzen/X370 Chipset and IOMMU Groups

Tue Mar 28 09:18:48 UTC 2017

That patch did not resolve the issue.

But I did find one useful information while testing things.
If I disable CSM, board will boot using GPU in the third slot connected to
chipset.
This is probably due to it having the lowest PCI id.

On Mon, Mar 27, 2017 at 11:33 PM, Nick Sarnie <commendsarnex at gmail.com>
wrote:

>
> On Mon, Mar 27, 2017 at 5:12 PM, Stano Lano <chl.xxx5x at gmail.com> wrote:
>
>> Following errors were seen in all over the output:
>>     AMD-Vi: Completion-Wait loop timed out
>> In the end I had kernel panic due to problems with IRQ
>>
>> When I connected serial console I noticed that the problem starts at GPU
>> that was connected to chipset slot:
>>
>> [    2.337432] AMD-Vi: IOMMU performance counters supported
>> [    2.354206] iommu: Adding device 0000:00:01.0 to group 0
>> [    2.359543] iommu: Adding device 0000:00:01.1 to group 0
>> [    2.364882] iommu: Adding device 0000:00:01.3 to group 0
>> [    2.370264] iommu: Adding device 0000:00:02.0 to group 1
>> [    2.375659] iommu: Adding device 0000:00:03.0 to group 2
>> [    2.381001] iommu: Adding device 0000:00:03.1 to group 2
>> [    2.386334] iommu: Adding device 0000:00:03.2 to group 2
>> [    2.391724] iommu: Adding device 0000:00:04.0 to group 3
>> [    2.397114] iommu: Adding device 0000:00:07.0 to group 4
>> [    2.402450] iommu: Adding device 0000:00:07.1 to group 4
>> [    2.407845] iommu: Adding device 0000:00:08.0 to group 5
>> [    2.413183] iommu: Adding device 0000:00:08.1 to group 5
>> [    2.418580] iommu: Adding device 0000:00:14.0 to group 6
>> [    2.423914] iommu: Adding device 0000:00:14.3 to group 6
>> [    2.429324] iommu: Adding device 0000:00:18.0 to group 7
>> [    2.434664] iommu: Adding device 0000:00:18.1 to group 7
>> [    2.439999] iommu: Adding device 0000:00:18.2 to group 7
>> [    2.445335] iommu: Adding device 0000:00:18.3 to group 7
>> [    2.450667] iommu: Adding device 0000:00:18.4 to group 7
>> [    2.455999] iommu: Adding device 0000:00:18.5 to group 7
>> [    2.461334] iommu: Adding device 0000:00:18.6 to group 7
>> [    2.466673] iommu: Adding device 0000:00:18.7 to group 7
>> [    2.472007] iommu: Adding device 0000:01:00.0 to group 0
>> [    2.477346] iommu: Adding device 0000:03:00.0 to group 0
>> [    2.482681] iommu: Adding device 0000:03:00.1 to group 0
>> [    2.488013] iommu: Adding device 0000:03:00.2 to group 0
>> [    2.493346] iommu: Adding device 0000:1d:00.0 to group 0
>> [    2.498679] iommu: Adding device 0000:1d:04.0 to group 0
>> [    2.504010] iommu: Adding device 0000:1d:06.0 to group 0
>> [    2.509342] iommu: Adding device 0000:1d:07.0 to group 0
>> [    2.642002] AMD-Vi: Completion-Wait loop timed out
>> [    2.646807] iommu: Adding device 0000:21:00.0 to group 0
>> [    2.779532] AMD-Vi: Completion-Wait loop timed out
>> [    2.784341] iommu: Adding device 0000:21:00.1 to group 0
>> [    2.916985] AMD-Vi: Completion-Wait loop timed out
>> [    2.921788] iommu: Adding device 0000:25:00.0 to group 0
>> [    3.054520] AMD-Vi: Completion-Wait loop timed out
>> [    3.059326] iommu: Adding device 0000:26:00.0 to group 0
>> [    3.191976] AMD-Vi: Completion-Wait loop timed out
>> [    3.196781] iommu: Adding device 0000:27:00.0 to group 0
>> [    3.329510] AMD-Vi: Completion-Wait loop timed out
>> [    3.334311] iommu: Adding device 0000:28:04.0 to group 0
>> [    3.466965] AMD-Vi: Completion-Wait loop timed out
>> [    3.471766] iommu: Adding device 0000:29:00.0 to group 2
>> [    3.516568] iommu: Adding device 0000:29:00.1 to group 2
>> [    3.649180] AMD-Vi: Completion-Wait loop timed out
>> [    3.653987] iommu: Adding device 0000:2a:00.0 to group 2
>> [    3.786686] AMD-Vi: Completion-Wait loop timed out
>> [    3.791490] iommu: Adding device 0000:2a:00.1 to group 2
>> [    3.924102] AMD-Vi: Completion-Wait loop timed out
>> [    3.928906] iommu: Adding device 0000:2b:00.0 to group 4
>> [    4.061614] AMD-Vi: Completion-Wait loop timed out
>> [    4.066420] iommu: Adding device 0000:2b:00.2 to group 4
>> [    4.199051] AMD-Vi: Completion-Wait loop timed out
>> [    4.203854] iommu: Adding device 0000:2b:00.3 to group 4
>> [    4.336576] AMD-Vi: Completion-Wait loop timed out
>> [    4.341379] iommu: Adding device 0000:2c:00.0 to group 5
>> [    4.474015] AMD-Vi: Completion-Wait loop timed out
>> [    4.478816] iommu: Adding device 0000:2c:00.2 to group 5
>> [    4.518478] iommu: Adding device 0000:2c:00.3 to group 5
>> [    4.651291] AMD-Vi: Completion-Wait loop timed out
>>
>>
>> For now my workaround is to use Radeon HD 6450 for my kodi seat.
>> Not ideal but it will do.
>> From the 3 GPUs I tested RX 460 & R7 260 would not allow me to boot due
>> to the described problem.
>>
>> On Mon, Mar 27, 2017 at 5:12 PM, Nick Sarnie <commendsarnex at gmail.com>
>> wrote:
>>
>>> The ticket is still open, but it wouldn't fix a hang on startup. Do you
>>> see any errors in dmesg?
>>>
>>> On Mon, Mar 27, 2017 at 7:28 AM, Stano Lano <chl.xxx5x at gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> did you have any luck with the ASUS support.
>>>> I should have the same ASUS Prime X370-Pro.
>>>> But when I enable IOMMU I am not able to boot, when the IOMMU is
>>>> disable I have no problem to boot.
>>>>
>>>> I am on BIOS 0511 and kernel 4.10.5
>>>> Tried with 0502, 0504 & 0511 BIOSes but no luck with any of them.
>>>> Also tried with Fedora 25, 25 beta & Ubuntu 17.04 beta. Same behavior.
>>>>
>>>> Thanks
>>>> Stano
>>>>
>>>> ------------------------------
>>>>
>>>>    - *From*: Nick Sarnie <commendsarnex gmail com>
>>>>    - *To*: Alex Williamson <alex l williamson gmail com>
>>>>    - *Cc*: vfio-users <vfio-users redhat com>
>>>>    - *Subject*: Re: [vfio-users] Ryzen/X370 Chipset and IOMMU Groups
>>>>    - *Date*: Fri, 10 Mar 2017 13:53:26 -0500
>>>>
>>>>
>>>> Yeah, that is unfortunate.
>>>>
>>>> Thanks for helping with this issue. I've sent a ticket to Asus, but I'm
>>>> not expecting much. Then again, I felt the same way and Gigabyte actually
>>>> sent me a fixed bios, so who knows.
>>>>
>>>> I'll keep you updated.
>>>>
>>>> Thanks again,
>>>> Sarnex
>>>>
>>>> On Fri, Mar 10, 2017 at 1:37 PM, Alex Williamson <alex l williamson
>>>> gmail com> wrote:
>>>>
>>>>> On Fri, Mar 10, 2017 at 11:05 AM, Nick Sarnie <commendsarnex gmail com
>>>>> > wrote:
>>>>>
>>>>>> Hi Alex,
>>>>>>
>>>>>> I don't see either of the options. I couldn't find Common Options at
>>>>>> all, and here's a screenshot of the CBS settings:
>>>>>>
>>>>>> https://i.imgur.com/9hQUHX0.jpg
>>>>>>
>>>>>> Is this something I should ask Asus to add?
>>>>>>
>>>>>
>>>>> I suppose it wouldn't hurt to try to start the discussion with Asus.
>>>>> The video I found was this one:
>>>>>
>>>>> https://youtu.be/pipR5xhrLo0?t=20
>>>>>
>>>>> At that start time you can see an NBIO Common Options menu on an
>>>>> ASRock system, but I never saw him open it and I couldn't find any
>>>>> documentation on what might be in there in an asrock mb manual (not an
>>>>> endorsement for asrock, perhaps they just have a BIOS more similar to the
>>>>> AMD sample implementation).  If AMD put it into a menu of debug options,
>>>>> it's really no surprise that consumer firmware dropped it.  Too bad.  This
>>>>> feels like a repeat of the difficulty we had trying to find motherboards
>>>>> that allowed the IOMMU to be enabled when AMD-Vi came out.  I wonder if
>>>>> AVIC requires yet another BIOS option that consumers will need to gamble
>>>>> with.
>>>>>
>>>>
>>>
>> Apparently this will fix it
>
> https://lists.freedesktop.org/archives/amd-gfx/2017-March/006862.html
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20170328/a2c447ac/attachment.htm>