[libvirt] [PATCH v5 0/4] qemu: Allow PCI virtio on ARM "virt" machine

[Laine Stump]

On 08/06/2015 09:38 AM, Michal Privoznik wrote:
> On 06.08.2015 15:06, Pavel Fedin wrote:
>>  Hello!
>>
>>> ACKed all the patches, squashed in the 2/4 amendment and pushed. Sorry
>>> for the delay.
>>  Thank you very much.
>>
>>  By the way, just curious, why is there a restriction to plug in most of devices only into PCI, not
>> PCIe? On machines with PCIe host this seems to do nothing good, only prevents them from using MSI-X.

The original patches to support pcie-root severely restricted what could
plug into what because in real hardware you can't plug a PCI device into
a PCIe slot (physically it doesn't work), and although it did happen to
work with qemu, I had no information to tell me that, e.g. plugging a
PCI device into a PCIe slot, or PCIe device into a PCI slot would
continue to work in the future. I've since received assurances from qemu
developers that this will continue to work, so the restriction has been
loosened (see below).

>> For example virtio-scsi still suffers from this problem. I ignore it for now because we currently
>> don't have vhost-scsi support in qemu.
>>  Comments say that "PCIe is not hotpluggable".

You cannot hotplug directly into pcie-root. This is true in the real
world, and either is or should be true in qemu. hotplug with pcie is
supported by other pcie controllers that are connected into pcie-root at
guest startup time.

>>  But why?

because that's the way it was originally implemented in hardware?

>>  And why do we mandate hotplug?

We don't (any longer) mandate it. But in the absence of any information
to the contrary, we assume that the user might want it, so we try not to
make a setup that will surprise and disappoint them. In the past, the
rules about insisting on a hotpluggable port (and only plugging PCI
devices into true PCI slots rather than PCIe) were strictly enforced by
libvirt all the time, but there was an open BZ about that which I
resolved a couple months ago:

https://bugzilla.redhat.com/show_bug.cgi?id=1238338

The behavior now is that if libvirt is auto-assigning a slot for a
device, it will put it into a hotpluggable true PCI slot, but if you
manually assign it to a slot that is non-hotpluggable and/or PCIe, it
will be allowed.

Beyond that, just last night I pushed a series of patches supporting
three new PCIe controller types: pcie-root-port,
pcie-switch-upstream-port, and pcie-switch-downstream-port. With those
implemented, we can now implement pretty much any standard PCIe bus
topology, so you can now hotplug into a true PCIe port. (Note that there
are a few restrictions about which type of controllers can be plugged
into which other controllers and these *are* strictly enforced, because
not following those rules leads to non-working configurations; for
example, a pcie-switch-upstream-port cannot be plugged directly into
pcie-root; it can *only* be plugged into a pcie-root-port or a
pcie-switch-downstream-port.)

> Yeah, that's a good question. In a real world, PCIe is perfectly hot
> pluggable. Maybe the limitation comes from qemu?

That is a very broad statement, and isn't true in all cases. For
example, in the "real world" you can't hotplug/unplug directly to the
pcie-root bus (nor can you hotplug/unplug to the dmi-to-pci-bridge (aka
i8280111-bridge). But you *can* (at guest startup time) attach a
pcie-root-port to the pcie-root, or a
pcie-switch-upstream-port+pcie-switch-downstream-ports to pcie-root or
to a dmi-to-pci-bridge, then hotplug/unplug to a pcie-root-port or a
pcie-switch-downstream-port (do a "git pull" of master to get the new
controller types I explained above and you'll see what I'm talking about).

BTW, I'm still wondering if the arm machinetype really does support the
supposedly Interl-x86-specific i82801b11-bridge device (and the new
controller devices - ioh3420 (pcie-root-port), x3130-upstream
(pcie-switch-upstream-port), and xio3130-downstream
(pcie-switch-downstream-port). In particular, the i82801b11-bridge is
important, since it is automatically added by libvirt when pcie-root is
present.