[vfio-users] [help] 2 identical GPUs in Arch

[Garland Key]

Once both cards are bound to pci-stub, do I need to unbind them before
binding them to nvidia and vfio-pci?  I just realized my script never
unbinds the gpus from the kernel, so hopefully it will work this time.

On Tue, Feb 23, 2016 at 1:47 PM Garland Key <david.garland.key at gmail.com>
wrote:

> Okay - I'll try it with pci-stub grabbing both at boot, then I'll unbind
> both cards from the kernel in initramfs, then I'll assign one to vfio-pci
> and the other to nvidia.  From there, systemd will take over and try to
> boot x (I think) and if this script works, I will have overcome this
> hurdle.  Hit me back if you think I'm not understanding something.
>
> On Tue, Feb 23, 2016 at 1:38 PM Ben J <btpprograms at gmail.com> wrote:
>
>> Did vfio-pci bind? I would bind the vfio before nvidia in your script.
>> You can try to unbind from nvidia but I know it generally doesn't work
>> well. So for #2 I wouldn't expect that to work. I'm pretty bad with the
>> boot process so I'll leave that for someone else.
>> On Feb 23, 2016 1:33 PM, "Garland Key" <david.garland.key at gmail.com>
>> wrote:
>>
>>> @Alex & @Ben J
>>>
>>> I tried binding both cards to pci-stub in boot options and then running
>>> the following script in initramfs:
>>>
>>>     echo 0000:01:00.0 > /sys/bus/pci/drivers/pci-stub/unbind
>>>     echo 0000:01:00.1 > /sys/bus/pci/drivers/pci-stub/unbind
>>>     echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/unbind
>>>     echo 0000:02:00.1 > /sys/bus/pci/drivers/pci-stub/unbind
>>>     echo 0000:01:00.0 > /sys/bus/pci/drivers/nvidia/bind
>>>     echo 0000:01:00.1 > /sys/bus/pci/drivers/nvidia/bind
>>>     echo 0000:02:00.0 > /sys/bus/pci/drivers/vfio-pci/bind
>>>     echo 0000:02:00.1 > /sys/bus/pci/drivers/vfio-pci/bind
>>>
>>> This didn't work (nvidia driver doesn't attach to either card) but I
>>> think I'm understanding more and will find a solution with the more
>>> understanding I gain (call me Captain Obvious).
>>>
>>> I have a two questions that should help me understand what to do.
>>>
>>> 1. After initramfs runs, systemd takes over.  At what point in this
>>> process does the nvidia driver actually try to load and when does X try to
>>> start?
>>>
>>> 2. Should I just not use pci-stub and instead just run a script in
>>> initramfs that unbinds both cards and then binds each one to the driver
>>> that I want?
>>>
>>> On Sun, Feb 21, 2016 at 12:58 PM Garland Key <
>>> david.garland.key at gmail.com> wrote:
>>>
>>>> I checked dmesg and there wasn't anything that stood out.  There was a
>>>> DRM error message for nvidia but it wasn't critical.  I'll look into this
>>>> more tomorrow.  I have to go to sleep (I work at night).  Thanks, Alex.
>>>> I'll let you know if and when the advice you and Ben have shared works.
>>>>
>>>> On Sun, Feb 21, 2016 at 12:40 PM Alex Williamson <
>>>> alex.williamson at redhat.com> wrote:
>>>>
>>>>> On Sun, Feb 21, 2016 at 10:01 AM, Garland Key <
>>>>> david.garland.key at gmail.com> wrote:
>>>>>
>>>>>> It seems xorg crashed because the nvidia kernel module failed.
>>>>>> Here's the xorg log file:
>>>>>>
>>>>>
>>>>> Did the intended device get bound to the nvidia module?  Are there
>>>>> dmesg errors?  You could try creating a file like:
>>>>>
>>>>> /etc/X11/xorg.conf.d/10-nvidia.conf:
>>>>> Section "Device"
>>>>>     Identifier "Device0"
>>>>>     Driver "nvidia"
>>>>>     VendorName "NVIDIA Corporation"
>>>>>     BusID "PCI:1:0:0"
>>>>>     # Or PCI:2:0:0, depending on which is intended for the host
>>>>> EndSection
>>>>>
>>>>> Maybe Xorg is complaining because it's trying to use both cards when
>>>>> you only intend for it to use one.
>>>>>
>>>>
>>> _______________________________________________
>>> vfio-users mailing list
>>> vfio-users at redhat.com
>>> https://www.redhat.com/mailman/listinfo/vfio-users
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160223/ea0e9684/attachment.htm>