<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">On 7/18/19 10:29 AM, Daniel Henrique
Barboza wrote:<br>
</div>
<blockquote type="cite"
cite="mid:161d6e05-6a38-262f-56da-7824258b14ef@gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<font size="+1">Hi,<br>
<br>
I have a PoC that enables partial coldplug assignment of
multifunction<br>
PCI devices with managed mode. At this moment, Libvirt can't
handle<br>
this scenario - the code will detach only the hostdevs from the
XML,<br>
when in fact the whole IOMMU needs to be detached. This can be<br>
verified by the fact that Libvirt handles the unmanaged scenario<br>
well, as long as the user detaches the whole IOMMU beforehand.<br>
<br>
I have played with 2 approaches. The one I am planning to
contribute<br>
back is a change inside virHostdevGetPCIHostDeviceList(), that<br>
adds the extra PCI devices for detach/re-attach in case a PCI<br>
Multifunction device in managed mode is presented in the XML.<br>
</font></blockquote>
<p><br>
</p>
<p><font size="+1">If you're thinking of doing that automatically,
then I should warn you that we had discussed that a long time
ago, and decided that it was a bad idea to do it because it was
likely someone would, e.g. try to assign an audio device to
their guest that happened to be one function on a multifunction
device that also contained a disk controller (or some other
device) that the host needed for proper operation.</font></p>
<p><font size="+1"><br>
</font></p>
<p><font size="+1">It may be that in *your* particular case, you
understand that the functions you don't want to assign to the
guest are not otherwise used, and it's not dangerous to suddenly
detach them from their host driver. But you can't assume that
will always be the case.</font></p>
<p><font size="+1"><br>
</font></p>
<p><font size="+1">If you *really* can't accept just assigning all
the devices in that IOMMU group to the guest (thus making them
all explicitly listed in the config, and obvious to the
administrator that they won't be available on the host) and
simply not using them, then you either need to separately detach
those particular functions from the host, or come up with a way
of having the domain config explicitly list them as "detached
from the host but not actually attached to the guest".</font></p>
<p><font size="+1"><br>
</font></p>
<br>
<blockquote type="cite"
cite="mid:161d6e05-6a38-262f-56da-7824258b14ef@gmail.com"><font
size="+1"> Now, there's a catch. Inside both </font><font
size="+1"><font size="+1">virHostdevPreparePCIDevices()<br>
and virHostdevReAttachPCIDevices() there are code to
save/restore<br>
the network configuration for SR-IOV devices. T</font>hese
functions iterates<br>
in the hostdevs list, instead of the pcidevs list I'm changing.
The final<br>
result, given that the current conditions used for SR-IOV
matches the<br>
conditions for multifunction PCI devices as well, is that not
all virtual<br>
functions will get their network configuration saved/restored.<br>
</font></blockquote>
<p><br>
</p>
<p><font size="+1">If you're not going to use a device (which is
implied by the fact that it's not in the hostdevs list) then
nothing about its network config will change, so there is no
reason to save/restore it.</font></p>
<p><font size="+1"></font><br>
</p>
<blockquote type="cite"
cite="mid:161d6e05-6a38-262f-56da-7824258b14ef@gmail.com"><font
size="+1"> <br>
For example, a guest that uses 3 of 4 functions of a PCI
MultiFunction<br>
card, let's say functions 0,1 and 3. The code will handle the
detach<br>
of all the IOMMU, including the function 2 that isn't declared
in the<br>
XML.</font></blockquote>
<p><br>
</p>
<p>Again, the above sentence implies that you're wanting to make
this completely automatic, which we previously decided was
something we didn't want to do. <br>
</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:161d6e05-6a38-262f-56da-7824258b14ef@gmail.com"><font
size="+1"> However, since function 2 isn't a hostdev, its
network config<br>
will not be restored after the VM shutdown.<br>
</font></blockquote>
<p><br>
</p>
<p><font size="+1">You're talking about something that will never
occur - on every SRIOV network card I've ever seen each VF is in
its own IOMMU group, and can be assigned to a guest independent
of what's done with any other VF. I've never seen a case (except
maybe once with a newly released motherboard that had broken
IOMMU firmware(?)) where a VF was in the same IOMMU group as any
other device.<br>
</font></p>
<p><font size="+1"></font><br>
</p>
<blockquote type="cite"
cite="mid:161d6e05-6a38-262f-56da-7824258b14ef@gmail.com"><font
size="+1"> <br>
Now comes the question: how much effort should be spent into
making<br>
the network config of all the functions be restored? Is this a
blocker<br>
for the whole code to be accepted or, given it is proper
documented<br>
somewhere, it can be done later on?</font></blockquote>
<br>
</body>
</html>