[vfio-users] VFIO and coherency with DMA

[Stark, Derek]

Hello,

I've been experimenting with VFIO with one of our FPGA cards using a Xilinx part and XDMA IP core. It's been smooth progress so far and I've had no issues with bar access and also DMA mapping/transfers to and from the card. All in all, I'm finding it a very nice userspace driver framework.

I'm hoping someone can help clarify my understanding of how VFIO works for DMA in terms of coherence. I'm on a standard x86_64 Intel Xeon platform.

In the  code I see:
VFIO
/*
* IOMMU enforces DMA cache coherence (ex. PCIe NoSnoop stripping).  This
* capability is subject to change as groups are added or removed.
*/
#define VFIO_DMA_CC_IOMMU                               4


Which implies that IOMMU sets the mappings up as coherent.... is this understanding correct?

I'm more used to having scatter gather based DMAs where you need to sync for the CPU or the device depending upon who owns/accesses the memory.

The use case I am specifically looking at is if a DMA mapping is setup through VFIO and then left open whilst data is transferred from the device to host memory and then the CPU is processing this data. The pinned/mapped data buffer is reused repeatedly as part of a ring of buffers. It's only at the point of closing down this application that the buffer would be unmapped in vfio.

Is there any sync type functions or equivalents I need to be aware of in this case? Can VFIO DMA mapped memory buffers be safely used in this way?

Thanks in advance for any information you can help with.

Derek

________________________________



CONFIDENTIALITY NOTICE: This message (including any attachments) may contain Molex confidential information, protected by law. If this message is confidential, forwarding it to individuals, other than those with a need to know, without the permission of the sender, is prohibited.

This message is also intended for a specific individual. If you are not the intended recipient, you should delete this message and are hereby notified that any disclosure, copying, or distribution of this message or taking of any action based upon it, is strictly prohibited.

English | Chinese | Japanese
www.molex.com/confidentiality.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20200213/82558afc/attachment.htm>