[PATCH] qemu: add pcie-to-pci-bridge for q35 machines

[Andrea Bolognani]

On Fri, Nov 04, 2022 at 01:03:52PM +0100, Peter Krempa wrote:
> On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
> > Hotplugging PCI devices on pc-i440fx machines is supported without
> > additional configuration. On q35, pcie-to-pci-bridge needs to be added
> > prior to the machine startup in order to support hotplug [1].
> >
> > The idea is to support the same workflow for creating VMs in q35, as was
> > in pc. Namely, do not require additional configuration when hotplugging
> > is needed. Otherwise, all libvirt clients need to be updated which there
> > are a lot and they are maintained by different parties.
> >
> > Instead, a pcie-to-pci-bridge better be created by default, so that PCI
> > slots are readily available. Might be a good idea to make it configurable
> > in the future.
> >
> > Previously there was a pci-bridge present by default, but was removed in
> >   d5fb8f4564 (qemu: don't add pci-bridge to Q35/arm domains unless it's needed, 2016-04-22)
> >
> > [1]: https://libvirt.org/pci-hotplug.html
> >
> > +    /* Add a pcie-to-pci-bridge and pcie-root-port to plug it into. */
> > +    if (addPCIeToPCIBridge) {
> > +        if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 1,
> > +                                       VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT) < 0)
> > +            return -1;
> > +        if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 2,
> > +                                       VIR_DOMAIN_CONTROLLER_MODEL_PCIE_TO_PCI_BRIDGE) < 0)
> > +            return -1;
> > +    }
>
> Doing this unconditionally breaks ABI of the VM, as new guest visible
> devices appear without any change in configuration. What's worse this
> implementation would also break migration compatibility from older
> versions of libvirt as there isn't any form of explicit handling for it.
>
> I unfortunately don't see any reasonable solution originating in
> libvirt. The bridge will need to be added by users or higher level
> management tools (e.g. virt-install) as libvirt explicitly wants to
> support use cases when VMs are started without persisting the XML
> definition (e.g. always re-define the XML before start or use the
> 'createXML' api).

Additionally, after this change it would be impossible to create a
q35 VM *without* the additional bridge. Most users of the q35 machine
type are likely using PCIe devices exclusively, and such a change
would negatively impact them.

In the (hopefully rare) cases in which hotplugging traditional PCI
devices is actually required, enabling that at the time when the
domain is defined is already straightforward: just include the
relevant controller in the XML.

Unfortunately q35, unlike i440fx, doesn't support hotplugging of any
kind out of the box, so it's up to the user / management application
to ensure that the necessary controllers are added. This is not
ideal, but there's no way around it, and it's still preferable to
forcing unavoidable, potentially useless extra controllers on
everybody.

-- 
Andrea Bolognani / Red Hat / Virtualization