[vfio-users] Direct Assignment of Non-PCI devices to VM

[Micah Morton]

On Wed, Nov 4, 2020 at 5:30 AM Roja Eswaran <reswara1 at binghamton.edu> wrote:
>
> I am using Jetson Xavier NX which integrates a Realtek RTL8211FDI Gigabit Ethernet controller.  The on-module Ethernet controller supports 10/100/1000 Gigabit Ethernet IEEE 802.3u Media Access Controller (MAC).
>  I am trying to assign Non-PCI NIC mentioned above directly to the Vanilla VM. As it's not a PCI device, I have no idea how I can achieve this.
>
> 1) Is it possible to do a direct assignment on Non-PCI devise to VM using VFIO drivers?

I don't see any mention of an IOMMU in any of these 3 DT files:

https://elixir.bootlin.com/linux/latest/source/arch/arm64/boot/dts/nvidia/tegra194-p2972-0000.dts
https://elixir.bootlin.com/linux/latest/source/arch/arm64/boot/dts/nvidia/tegra194-p2888.dtsi
https://elixir.bootlin.com/linux/latest/source/arch/arm64/boot/dts/nvidia/tegra194.dtsi

Does the Jetson Xavier NX board have an iommu? If not, this is the
most fundamental reason you wouldn't be able to use VFIO for
assignment on that board -- especially if the ethernet controller is
DMA-capable, since you won't be able to ensure the ethernet device's
DMAs operate on the memory assigned to the guest.

You could check for this by doing "dmesg | grep -i -e iommu -e smmu"
on the device and seeing if it prints anything, or looking at
/sys/kernel/iommu_groups to see if that dir exists.

Beyond the IOMMU, vfio-platform/QEMU in their current state wouldn't
be able to handle the embedded PHY device here:
https://elixir.bootlin.com/linux/latest/source/arch/arm64/boot/dts/nvidia/tegra194-p2888.dtsi#L43
, which would be another hurdle.

> 2) If not, are there any other drivers that would help me achieve the direct assignment?
> The direct assignment is important because we are trying to reduce the CPU utilization even though virtio drivers give us a very small network latency overhead. Any comments or insights are really appreciated. Thank you so much for your time!
>
>
> --
> Thanks,
> Roja Eswaran
> Ph.D. Student, Operating Systems and Networks Laboratory
> State University of New York, Binghamton
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users