[libvirt] [PATCH 4/6] conf: new <hotplug require='yes|no'/> element for hotpluggable devices

[Laine Stump]

On 08/09/2016 03:54 AM, Daniel P. Berrange wrote:
> On Mon, Aug 08, 2016 at 04:56:55AM -0400, Laine Stump wrote:
>> When faced with a guest device that requires a PCI address but doesn't
>> have one manually assigned in the config, libvirt has always insisted
>> (well... *tried* to insist) on auto-assigning an address that is on a
>> PCI controller that supports hotplug. One big problem with this is
>> that it prevents automatic use of a Q35 (or aarch64/virt) machine's
>> pcie-root (since the PCIe root complex doesn't support hotplug).
> I'm not really seeing why lack of use of pcie-root is a real problem. It
> makes total sense to always use the bridge instead of root if the latter
> does not support hotplug. At the time you first create a guest, you may
> not even realize that you'll need to be able to hot-unplug a particular
> device later. So putting any device on pcie-root is tieing your hands for
> the entire lifetime of the guest.

1) simpler bus topology, less devices - the less there is, the less 
resources are used, the less can go wrong (among other advantages)

2) In particular, there is a lot of call for making Q35 and aarch64/virt 
machines "legacy-PCI-free". But some of the emulated devices are 
PCI-only, and until PCIe alternatives are available, this provides a 
method of getting the use of these devices (yes, I realize that in 
itself screws over the "legacy-PCI-free" demand) at least without 
requiring the addition of a pci-bridge or the dreaded dmi-to-pci-bridge.

3) Even for PCIe devices, there currently is no way to have a 
hotpluggable device without using a vendor-specific PCI controller that 
doesn't necessarily exist on the real hardware that is being emulated 
(pcie-root-port emulates an Intel IOH3420, and 
pcie-switch-downstream-port is a TI X3130; I have serious doubts that 
either of those are used (or will ever be used) in any real aarch64 
hardware). Requiring those controllers to be added into the virtual 
machine may lead to incompatibilities with OSes that have been designed 
only for real hardware and may be lacking the proper drivers for these 
controllers.

4) libvirt should not be making policy decisions about things like 
whether or not a device should be hotpluggable. On the 440fx, *all* 
devices are in hotplug-capable slots, so this is a non-issue. On Q35 and 
other machinetypes with a pcie-root, there is a choice, and that choice 
should be made by the user/management app, not forced on them by libvirt.

As far as not realizing that you'll need to hot-unplug some device: 
that's true and that's why the default is to assume hotplug will be 
needed. But certainly for most devices you can say with a very high 
degree of certainty that you'll never hot-unplug them. Why force extra 
infrastructure for something that will definitely never be used? 
Although it shouldn't be the default, we really do need a way to allow 
avoiding the extra trappings of hotplug without requiring the user to 
specify the full address.

>> In order to promote simpler domain configs (more devices on pcie-root
>> rather than on a pci-bridge), this patch adds a new sub-element to all
>> guest devices that have a PCI address and support hotplug:
>>
>>    <hotplug require='no'/>
>>
>> For devices that have hotplug require='no', we turn off the
>> VIR_PCI_CONNECT_HOGPLUGGABLE bit in the devFlags when searching for an
>> available PCI address. Since pcie-root now allows standard PCI
>> devices, this results in those devices being placed on pcie-root
>> rather than pci-bridge.
> I still really hate the idea of adding elements to the XML config
> that have nothing todo with the actual guest ABI config, but instead
> are setting logic policy in libvirt. But more generally I don't see
> why anyone would ever want to set require=no for this - it just ties
> your hands for later.

So that you can explicitly acknowledge that you don't want the extra 
trappings that hotplug requires.


>> NB: According to the code in qemuDomainAssignDevicePCISlots()
>> and qemuDomainAttachDeviceLive(), the following device types are the
>> only ones that use a PCI address *and* support hotplug:
>>
>>    <interface>
>>    <hostdev>
>>    <rng>
>>    <serial>
>>    <disk> (only virtio)
>>    <controller> (hotplug only for SCSI controllers)
>>
>> These devices use a PCI address, but don't support hotplug, so the RNG
>> doesn't allow them to have a <hotplug> element:
>>
>>    <filesystem>
>>    <sound>
>>    <balloon>
>>    <watchdog>
>>    <video>
>>    <shmem>
>>    <input>
> IMHO it is really bad practice to make any assumption about what
> devices support hotplug & what don't. In fact if any device type
> does not support hotplug I think it is fair to argue that is
> simply a bug / missing feature in libvirt right now. I see no
> reason why any of those devices should not be able to support
> hotplug in general.

Yes, good point; I suppose that if a future version of libvirt adds the 
ability to hotplug those devices, someone may be upset that the virtual 
machine they created in the past can't hot-unplug those devices. But 
does that mean we shouldn't be placing the default video device directly 
at 00:1.0, or the default USB controller at 00:1D.x?