[vfio-users] Can VFIO pin only a specific region of guest mem when use pass through devices?

[arjenvanweelden at gmail.com]

On Sun, 2018-10-28 at 20:11 +0800, Simon Guo wrote:
> Hi,
> 
> I am using network device pass through mode with qemu x86(-device
> vfio-pci,host=0000:xx:yy.z) 
> and “intel_iommu=on” in host kernel command line, and it shows the
> whole guest memory 
> were pinned(vfio_pin_pages()), viewed by the “top” RES memory output.
> I understand it is due 
> to device can DMA to any guest memory address and it cannot be
> swapped.
> 
> However can we just pin a rang of address space allowed by iommu on
> that device, 
> instead of pin whole address space? I do notice some code like
> vtd_host_dma_iommu(). 
> Maybe there is already some way to enable that?
> 
> Sorry if I missed some basics. I googled some but no luck to find the
> answer yet. Please 
> let me know if any discussion already raised on that.
> 
> Any other suggestion will also be appreciated. For example, can we
> modify the guest network 
> card driver to allocate only from a specific memory region(zone), and
> qemu advises guest 
> kernel to only pin that memory region(zone) accordingly? 
> 
> Thanks,
> - Simon
> 

I agree that it would be very convenient to be able to memory-balloon a
VM with passthrough devices. All I could find was this presentation
from August 2011 which talks about PPR: 
https://www.linux-kvm.org/images/b/b1/2011-forum-amd-iommuv2-kvm.pdf

"Get rid of guest memory pinning when all assigned devices support PPR
– DMA may be a bit slower on memory overcommit
– But: removes a major disadvantage of direct device assignment"

It looks like it might be possible for some combinations of CPU and PCI
devices, sometime in the future?

hope this helps, Arjen