<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">On 12/23/2015 11:01 AM, Ziviani .
wrote:<br>
</div>
<blockquote
cite="mid:CAH=L8t1BmS_zj-qzyc8m+gOzdFVKJ0p4Jn_fkP8J+U77ECR8JQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_default" style="font-family:courier
new,monospace;font-size:small;color:#0b5394">Hi Laine,</div>
<div class="gmail_default" style="font-family:courier
new,monospace;font-size:small;color:#0b5394"><br>
</div>
<div class="gmail_default" style="font-family:courier
new,monospace;font-size:small;color:#0b5394">This (hot
plugging all functions at once) is something I was thinking
about. What if we could create a xml file passing the IOMMU
group instead of only one function per time, would it be
feasible?</div>
<div class="gmail_default" style="font-family:courier
new,monospace;font-size:small;color:#0b5394">I could start
working on a proof of concept if the community thinks it's a
valid path.</div>
<div class="gmail_default" style="font-family:courier
new,monospace;font-size:small;color:#0b5394"><br>
</div>
<div class="gmail_default" style="font-family:courier
new,monospace;font-size:small;color:#0b5394">Do you know how
is currently working on it? I could offer some help if they
need.</div>
</div>
</blockquote>
<br>
(Please reply inline rather than top-posting. It makes it much
easier to follow the context of the conversation.)<br>
<br>
What do you mean by "passing the IOMMU group"? Do you mean *just*
the iommu group, excluding the information about the devices? This
doesn't seem like a good idea, since afaik the iommu group number is
something just conjured up by the kernel at boot time, and isn't
necessarily predictable or stable between host reboots. Also, it
wouldn't allow for assigning only some of the devices/functions in a
group while leaving others inactive.<br>
<br>
I think there are two reasonable possibilities:<br>
<br>
1) Follow the apparent path of qemu - accept separate attach calls,
one for each function, and use the attach of function 0 as the
"action" button that causes all the functions to be attached.<br>
<br>
2) Enhance the attach API to accept multiple <hostdev>
elements in the XML for a single call, and do "whatever is proper
for the current hypervisor" to attach them.<br>
<br>
<br>
As for detach, it's really only possible to detach *all* functions,
and it would take more bookkeeping to allowing marking each function
for removal and then removing the device when all functions had been
marked, so maybe we only allow detach of function 0, and that will
always detach everything? (not sure, that's just an idea). <br>
<br>
As far as I know, nobody is currently working on anything like this
for libvirt, so this is your chance to get your hands dirty!<br>
<br>
(It just occurred to me that method (1) of multifunction attach
method outlined above will also need similar extra bookkeeping, just
as the "mark each function for removal" detach method would, and
this extra bookkeeping would need to survive a restart of libvirtd
in the middle of a series of attach/detach calls, making it more
complicated, so maybe the 2nd methods would be better. I'd love to
hear opinions though.)<br>
<br>
<blockquote
cite="mid:CAH=L8t1BmS_zj-qzyc8m+gOzdFVKJ0p4Jn_fkP8J+U77ECR8JQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_default" style="font-family:courier
new,monospace;font-size:small;color:#0b5394"><br>
</div>
<div class="gmail_default" style="font-family:courier
new,monospace;font-size:small;color:#0b5394">Thank you :)</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Dec 21, 2015 at 3:53 PM, Laine
Stump <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:laine@laine.org" target="_blank">laine@laine.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>
<div class="h5">
<div>On 12/21/2015 08:29 AM, Ziviani . wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_default">Hello list!</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">I'm new here and
interested in hot-plug multi-function PCI
devices. Basically I'd like to know why Libvirt
does not support it. I've been through the
archives and basically found this thread:</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default"><font face="courier
new, monospace" color="#0b5394"><a
moz-do-not-send="true"
href="https://www.redhat.com/archives/libvir-list/2011-May/msg00457.html"
target="_blank"><a class="moz-txt-link-freetext" href="https://www.redhat.com/archives/libvir-list/2011-May/msg00457.html">https://www.redhat.com/archives/libvir-list/2011-May/msg00457.html</a></a></font><br>
</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default"><font face="courier
new, monospace" color="#0b5394">But Qemu seems
to handle it accordingly:</font></div>
<div class="gmail_default"><font face="courier
new, monospace" color="#0b5394">
<div class="gmail_default">virsh
qemu-monitor-command --hmp fedora-23
'device_add vfio-pci,host=00:16.0,addr=08.0'</div>
<div class="gmail_default">virsh
qemu-monitor-command --hmp fedora-23
'device_add vfio-pci,host=00:16.3,addr=08.3'</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">GUEST:</div>
<div class="gmail_default">
<div class="gmail_default"># lspci</div>
<div class="gmail_default">(snip)</div>
<div class="gmail_default">00:08.0
Communication controller: Intel
Corporation 8 Series HECI #0 (rev 04)<br>
</div>
<div class="gmail_default">00:08.3 Serial
controller: Intel Corporation 8 Series
HECI KT (rev 04)</div>
<div><br>
</div>
</div>
<div class="gmail_default">However, using
Libvirt:<br>
</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">
<div class="gmail_default">% virsh
attach-device fedora-23
pci_0000_00_16_0.xml --live</div>
<div class="gmail_default">Device attached
successfully</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">% virsh
attach-device fedora-23
pci_0000_00_16_3.xml --live<br>
</div>
<div class="gmail_default">error: Failed to
attach device from pci_0000_00_16_3.xml</div>
<div class="gmail_default">error: internal
error: Only PCI device addresses with
function=0 are supported</div>
<div><br>
</div>
<div>I made some changes on domain_addr.c[1]
for testing and it worked.</div>
<div><br>
</div>
<div>[1]<a moz-do-not-send="true"
href="https://gist.github.com/jrziviani/1da184c7fd0b413e0426"
target="_blank">https://gist.github.com/jrziviani/1da184c7fd0b413e0426</a></div>
<div><br>
</div>
<div>
<div>% virsh attach-device fedora-23
pci_0000_00_16_3.xml --live</div>
<div>Device attached successfully</div>
</div>
<div><br>
</div>
<div>
<div class="gmail_default">GUEST:</div>
<div class="gmail_default">
<div class="gmail_default"># lspci</div>
<div class="gmail_default">(snip)</div>
<div class="gmail_default">00:08.0
Communication controller: Intel
Corporation 8 Series HECI #0 (rev 04)<br>
</div>
<div class="gmail_default">00:08.3
Serial controller: Intel Corporation 8
Series HECI KT (rev 04)</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">So there is
more to it that I'm not aware?</div>
</div>
</div>
</div>
</font></div>
</div>
</blockquote>
<br>
</div>
</div>
<font color="#0b5394"><font face="courier new, monospace">You're
relying on behavior in the guest OS for which there is
no standard (and which, by definition, doesn't work on
real hardware, so no guest OS will be expecting it; a
friend more familiar with this has told me that
probably qemu is sending an (acpi?) "device check" to
the guest for each function that is added, and in your
case it's apparently "doing the right thing" in
response to that). But just because it is successful
in this one case doesn't mean that it will be
successful in all situations; likely it won't be. So
while the qemu monitor takes the laissez-faire
approach of allowing you to try it and letting you
pick up the pieces when it fails, libvirt prevents it
because it is bound to fail, and thus not supportable.<br>
<br>
There has recently been some work in qemu to "save up"
any requests to attach devices with function > 0,
then present them all to the guest at once when
function 0 is attached. This is the only standard way
to handle hotplug of multiple functions in a slot. Hot
unplug can only happen for all functions in the slot
at once. I'm not sure of the current status of that
work, but once it is in and stable, libvirt will
support it.<br>
<br>
</font></font>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_default"><font face="courier new,
monospace" color="#0b5394">
<div class="gmail_default">
<div><br>
</div>
<div>Thank you!<br>
</div>
</div>
</font></div>
</div>
<br>
<span class="HOEnZb"><font color="#888888">
<fieldset></fieldset>
<br>
<pre>--
libvir-list mailing list
<a moz-do-not-send="true" href="mailto:libvir-list@redhat.com" target="_blank">libvir-list@redhat.com</a>
<a moz-do-not-send="true" href="https://www.redhat.com/mailman/listinfo/libvir-list" target="_blank">https://www.redhat.com/mailman/listinfo/libvir-list</a></pre>
</font></span></blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>