[PATCH] qemu: add pcie-to-pci-bridge for q35 machines

[Denis V. Lunev]

On 11/4/22 14:41, Daniel P. Berrangé wrote:
> On Fri, Nov 04, 2022 at 01:03:52PM +0100, Peter Krempa wrote:
>> On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
>>> Hotplugging PCI devices on pc-i440fx machines is supported without
>>> additional configuration. On q35, pcie-to-pci-bridge needs to be added
>>> prior to the machine startup in order to support hotplug [1].
>>>
>>> The idea is to support the same workflow for creating VMs in q35, as was
>>> in pc. Namely, do not require additional configuration when hotplugging
>>> is needed. Otherwise, all libvirt clients need to be updated which there
>>> are a lot and they are maintained by different parties.
>>>
>>> Instead, a pcie-to-pci-bridge better be created by default, so that PCI
>>> slots are readily available. Might be a good idea to make it configurable
>>> in the future.
> The goal of q35 is to provide a PCI-e topology though, and we want
> devices to be exposed as PCI-e endpoints, with no legacy PCI usage
> by default.
>
> When a guest XML is provided for cold boot, libvirt will ensure
> that all devices are placed onto the PCI-e bus via pcie-root-ports
>
> With this proposal hotplugging a device will cause it to be placed
> on a PCI bus instead, if there are no spare pcie-root-port available.
>
> This behavioural difference between cold plug and hot plug is very
> undesirable IMHO, as it is counter to our goal of avoiding legacy
> PCI on PCI-e machines.

that is an interesting point which I have initially missed. I have
checked our investigation log and found that we have
potentially made a mistake with making compatibility
check with e1000 device instead of virtio, which is
our default. This could be the reason.

>
> If a mgmt app wants to enable this mis-matched behaviour they can
> add a pcie-to-pci-bridge when initially cold booting the VM, but
> we shouldn't add one by default IMHO. We really recommend that
> apps just pre-populate 10-30 pcie-root-port devices when creating
> a VM, so there is plenty of space for hotplugging devices later.

this potentially an option, but I think that these port should be
created automatically by libvirt to avoid major client rewrites.
Will the option to keep the amount of spare ports in /etc/libvirt/qemu.conf
work for you?

Den

>>> Previously there was a pci-bridge present by default, but was removed in
>>>    d5fb8f4564 (qemu: don't add pci-bridge to Q35/arm domains unless it's needed, 2016-04-22)
>
>
> With regards,
> Daniel