[vfio-users] vga pass-thru and different gfx cards.

[Torbjorn Jansson]

On 2015-09-10 21:02, Alex Williamson wrote:
> On Thu, Sep 10, 2015 at 12:57 PM, Torbjorn Jansson <
> torbjorn.jansson at mbox200.swipnet.se> wrote:
>
>> On 2015-09-10 20:48, Alex Williamson wrote:
>>
>>> On Thu, Sep 10, 2015 at 12:44 PM, Torbjorn Jansson <
>>>
>> torbjorn.jansson at mbox200.swipnet.se> wrote:
>>>
>>> On 2015-09-10 19:47, Alex Williamson wrote:
>>>>
>>>>> It's not clear to me why there are numerous reports of reset problems on
>>>>> nvidia.  AFAIK, there's no broken hardware for nvidia in this space like
>>>>> there is for AMD.  So if you're having reset problems, maybe it's
>>>>> because
>>>>> you're not binding both GPU and audio functions to vfio-pci and
>>>>> therefore
>>>>> vfio-pci can't do a bus reset on VM reset.  Maybe your motherboard
>>>>> freaks
>>>>> out on the bus reset.  Maybe it's a ROM issue like Blank suggests.
>>>>>
>>>>>
>>>>> both functions are bound to pci-stub via grub command line using:
>>>> pci-stub.ids=
>>>>
>>>> i forgot to say, i'm unfortunately stuck on linux kernel 4.0.4 due to
>>>> issues with drivers for a dvb card.
>>>> and my distro is fedora 22
>>>>
>>>
>>>
>>> pci-stub is only to prevent host driver from binding to the device,
>>> eventually the device needs to get bound to vfio-pci or else you're not
>>> using vfio to do the assignment.
>>>
>>
>> ok, and that gets taken care of by libvirt when i power up the vm?
>> i guess i have to check what driver is bound to the 2 functions once the
>> vm is up and also check what happens once it is powered off.
>>
>
> If you're assigning both the audio and GPU functions then libvirt should
> automatically re-bind them to vfio-pci, so long as you haven't specified
> driver='kvm' for the hostdev entry.  You can always double check while the
> VM is running or look in the libvirt log to make sure vfio-pci is being
> used rather than pci-assign.
>

a bit more testing and experimenting.
i checked with: "find /sys/kernel/iommu_groups/ -type l"
and apparently i had  the gfx card in the same iommu group as other 
stuff, this might have been due to recent changes in the gfx cards i did 
since last time i tried but don't remember exactly.

anyway, i swapped the gfx card with another card and was able to get it 
in its own group.
after adjusting the qemu-kvm.vga script with the new id numbers i tried 
to start my vm.
vm appears to not start at all and i get:

kernel:NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! 
[qemu-system-x86:2168]

at least it didn't crash the host right away.
after a short wile the host was completely unresponsive, last i saw from 
top had load avg around 28