[vfio-users] Single function PCIe NIC card having multiple queue. one queue for one VM. Can i use VFIO here?

[Sasikumar Natarajan]

Hi all,

                     _______________________
_______________________
                    |                       |
|                       |
                    |   Virtual Machine 1   |            |   Virtual
Machine 2   |
                    |      ___________      |            |
___________       |
                    |     |           |     |            |    |
|      |
                    |     | VM1 Memory|     |            |    |VM2 Memory
|      |
                    |     |___________|     |            |
|___________|      |
                    |             ^         |            |
^             |
                    |____________ | ________|            |________ |
____________|
                                  |                                |
                                  |                                |

________V________________________________V___________
                          |       |
|           |
                          |       |           IOMMU
|           |

|_______|________________________________|___________|
                                  ^                                ^
                                  |                                |
                         [To/From VM1 Buffers]            [To/From VM2
Buffers]     HOST

----------------|--------------------------------|------------------------
                        _________ | ______________________________
|_____________   PCI Device
                       |          |          PCI NIC CARD
|             |
                       |          |      (Single PCI Function)
|             |
                       |
__|________________________________|__           |
                       |       |  |                                |
|          |
                       |       |  |        PCIe DMA                |
|          |
                       |       |_ |_______________________________
|__|          |
                       |          |
|             |
                       |          V
V             |
                       |     Queue for VM1                   Queues for
VM2      |
                       |       |RX|TX|
|RX|TX|          |
                       |       |RX|TX|
|RX|TX|          |
                       |       |RX|TX|
|RX|TX|          |
                       |          ^
^             |
                       |          |       ___________________
|             |
                       |          |      |                   |
|             |
                       |          ------>|  Queue Classifier
|<----              |
                       |
|___________________|                   |
                       |                 |
|                   |
                       |                 |        MAC
|                   |
                       |                 |___________________|
------         |

|__________________________________________|Port|_________|
                                                                  ------
                                                                    ||
                                                                    ||
                                                                    ||___

|___----> To Network Fabric




We are trying to share the PCIe NIC card with multiple VMs. PCI NIC card
has one Physical function (NO SRIOV support).

The NIC has multiple queues (each with TX and RX descriptor ring). Packets
are placed in queue based on packet classifier(Ex: MAC addresses). So each
queue can be used for a network interface, therefore host can have multiple
virtual network interfaces. We are planning to assign each queue to each VM
(emulate like PCI Virual functions, but our HW doesnt support Virtual
function). Packet buffer for corresponding queue will reside in VM's memory.

DMA in the PCI card need to push/pull packet data to/from directly to the
VM's memory (for this we need IOMMU support).

How VFIO can be used to achieve this ?
A if VFIO is not possible, any other methods can be used here ?
are there any other methods without any intermediate CPU memory copy?




*Regards,*

Sasikumar Natarajan
​.​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160923/b4e8434a/attachment.htm>