[vfio-users] Single function PCIe NIC card having multiple queue. one queue for one VM. Can i use VFIO here?

[Alex Williamson]

On Fri, 23 Sep 2016 14:42:58 +0530
Sasikumar Natarajan <sasikumar.natarajan at vvdntech.in> wrote:

> Hi all,
> 
[diagram:https://www.redhat.com/archives/vfio-users/2016-September/msg00097.html]> 
> 
> We are trying to share the PCIe NIC card with multiple VMs. PCI NIC card
> has one Physical function (NO SRIOV support).
> 
> The NIC has multiple queues (each with TX and RX descriptor ring). Packets
> are placed in queue based on packet classifier(Ex: MAC addresses). So each
> queue can be used for a network interface, therefore host can have multiple
> virtual network interfaces. We are planning to assign each queue to each VM
> (emulate like PCI Virual functions, but our HW doesnt support Virtual
> function). Packet buffer for corresponding queue will reside in VM's memory.
> 
> DMA in the PCI card need to push/pull packet data to/from directly to the
> VM's memory (for this we need IOMMU support).
> 
> How VFIO can be used to achieve this ?
> A if VFIO is not possible, any other methods can be used here ?
> are there any other methods without any intermediate CPU memory copy?

You show a data path through the IOMMU, but without SR-IOV the physical
device only uses a single requester ID and therefore the same context
entry and IOMMU page tables regardless of the VM for which a DMA is
routed.  Currently vfio requires an IOMMU to provide isolation and
translation with per device and VM granularity.  There's work underway
upstream to create vfio "mediated" devices where a software component
in the host fills in the gaps versus a device that supports full
SR-IOV, including device isolation, but this latter component is still
provided via the device through things like per-process GTT (graphics
Translation Tables) in the case of vGPUs.  You could create a mediated
device that doesn't do DMA, or polices all devices programming, but
what's the point versus something like virtio if the performance
impact is more than trivial?  For DMA, there needs to be isolation and
translation provided at a different level.  Thanks,

Alex