<div dir="ltr"><div class="gmail_default" style="font-size:small">Hi Alex,</div><div class="gmail_default" style="font-size:small"><br></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><span style="font-size:13px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">VFIO does not setup<br></span><span style="font-size:13px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">mappings per DMA buffer, it maps all of guest physical memory through<br></span><span style="font-size:13px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">the IOMMU so that the device can DMA to any GPA and it's already<br></span><span style="font-size:13px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">mapped. We call this transparent assignment, the guest can perform DMA<br></span><span style="font-size:13px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">without any knowledge of or interaction with the host IOMMU.</span>
</blockquote><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Could you briefly explain how all of guest physical memory is mapped through IOMMU.</div><div class="gmail_default" style="font-size:small">I do see a ``vfio_dma_map`` call with `size` almost equal to my guest memory size, is</div><div class="gmail_default" style="font-size:small">this the place where IOMMU is instructed to enable such transparent assignment you</div><div class="gmail_default" style="font-size:small">mentioned? Since the guest driver is issuing IOs with GPA, how does the device do </div><div class="gmail_default" style="font-size:small">DMA to the corresponding physical DMA address? Is the GPA->HPA a static mapping</div><div class="gmail_default" style="font-size:small">so the HPA can be calculated instead of walking the page tables (which those DMA APIs</div><div class="gmail_default" style="font-size:small">do)? </div><div class="gmail_default" style="font-size:small"><br></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><br><span style="font-size:13px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">It's software, anything is possible, but you'll need to subvert<br></span><span style="font-size:13px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">something to do that for you. One subversion we have in place is the<br></span><span style="font-size:13px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">no-iommu mode of vfio, which does not provide any mapping services and<br></span><span style="font-size:13px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">taints the host kernel. This mode is explicitly not intended for<br></span><span style="font-size:13px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">device assignment, you'll be lucky to boot a VM at all and it will<br></span><span style="font-size:13px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">probably take the host down in the process, thus the tainting. Please<br></span><span style="font-size:13px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">don't report bugs against unsafe options that explicitly taint the<br></span><span style="font-size:13px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">kernel as an indication not to use them. Thanks,</span></blockquote><div class="gmail_default" style="font-size:small">
<br></div><div class="gmail_default" style="font-size:small">Seems this is what I need. I will have a try.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Thank you so much!!</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Best,</div><div class="gmail_default" style="font-size:small">Huaicheng</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small"><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Jun 5, 2018 at 5:25 PM Alex Williamson <<a href="mailto:alex.williamson@redhat.com">alex.williamson@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Tue, 5 Jun 2018 15:16:55 -0700<br>
Huaicheng Li <<a href="mailto:huaicheng@cs.uchicago.edu" target="_blank">huaicheng@cs.uchicago.edu</a>> wrote:<br>
<br>
> Hi,<br>
> <br>
> I assigned a PCIe device to my QEMU VM and learnt that in this case, IOMMU<br>
> maintains GPA->HPA translation. What I<br>
> want to achieve is to pass commands with host physical addresses to the<br>
> device from inside the VM and see if the DMA<br>
> will successfully do that for me.<br>
> <br>
> (1). How does VFIO setup the GPA->HPA mapping for each DMA buffer? which<br>
> source file can I refer to for such information?<br>
> Currently I'm looking at VFIO_IOMMU_DMA_MAP, it seems it only maps the base<br>
> address of QEMU guest memory region to<br>
> IOVA. Does this mean that later all GPA->HPA translation are based on that<br>
> plus some offsets?<br>
<br>
Unless you enable an emulated IOMMU in the guest, VFIO does not setup<br>
mappings per DMA buffer, it maps all of guest physical memory through<br>
the IOMMU so that the device can DMA to any GPA and it's already<br>
mapped. We call this transparent assignment, the guest can perform DMA<br>
without any knowledge of or interaction with the host IOMMU. If you do<br>
configure an emulated IOMMU, then the guest device driver will<br>
configure a mapping through the gIOMMU, which will create a mapping<br>
through the pIOMMU. In either case, all of this happens through<br>
MemoryListeners in the vfio code. It's mostly a difference in which<br>
AddressSpace the device is using.<br>
<br>
> (2). Is it possible for me to disable IOMMU page tables as I just want to<br>
> give the device a host physical address?<br>
<br>
It's software, anything is possible, but you'll need to subvert<br>
something to do that for you. One subversion we have in place is the<br>
no-iommu mode of vfio, which does not provide any mapping services and<br>
taints the host kernel. This mode is explicitly not intended for<br>
device assignment, you'll be lucky to boot a VM at all and it will<br>
probably take the host down in the process, thus the tainting. Please<br>
don't report bugs against unsafe options that explicitly taint the<br>
kernel as an indication not to use them. Thanks,<br>
<br>
Alex<br>
</blockquote></div>