[libvirt] Seabios can't boot from a disk behind i82801b11-bridge device (was Re: [PATCH 6/7] qemu: add dmi-to-pci-bridge controller)

[Laine Stump]

On 08/02/2013 12:55 PM, Laine Stump wrote:
> This PCI controller connects to a PCI Express slot (e.g. one of the
> slots provided by the pcie-root controller, aka "pcie.0" on the qemu
> commandline), and provides 31 *non-hot-pluggable* PCI (*not* PCIe)
> slots, numbered 1-31.
>
> Any time a machine is defined which has a pcie-root controller
> (i.e. any q35-based machinetype), libvirt will automatically add a
> dmi-to-pci-bridge controller if one doesn't exist, and also add a
> pci-bridge controller. The reasoning here is that any useful domain
> will have either an immediate (startup time) or eventual (subsequent
> hot-plug) need for a standard PCI slot; since the pcie-root controller
> only provides PCIe slots, we need to connect a dmi-to-pci-bridge
> controller to it in order to get a non-hot-plug PCI slot that we can
> then use to connect a pci-bridge - the slots provided by the
> pci-bridge will be both standard PCI and hot-pluggable.
>
> Since pci-bridge devices themselves are not hot-pluggable, any new
> pci-bridge controller that is added can (and will) be plugged into the
> dmi-to-pci-bridge as long as it has empty slots available.

Sigh. (yeah, I know I've been saying that a lot lately :-)

I've just now figured out that seabios doesn't recognize any disk device
that is connected anywhere behind the i82801b11-bridge device. So if you
have a virtio-blk-pci device that is attached in this manner (the way
these patches are setup to work):

    pcie.0->i82801b11-bridge->pci-bridge->virtio-blk-pci

or even like this (precluding later hot-plug, but at least still not
violating the hardware specs):

    pcie.0->i82801b11-bridge->virtio-blk-pci

seabios won't see the disk, and it won't boot.

However, if you ignore the fact that the pci-bridge device requires a
standard PCI slot (*not* PCIe) and shouldn't be directly plugged into
pcie.0 and do this (which is allowed by qemu even though it strictly
speaking *shouldn't* be):

   pcie.0->pci-bridge->virtio-blk-pci

then seabios recognizes the disk and is able to boot from it.

So I see three options for taking care of of this problem

1) leave this libvirt code as it is, and petition seabios to add support
for recognizing i82801b11-bridge devices and booting from disks that are
connected behind them

drawback: we will remain unable to create working Q35 machines with
libvirt until seabios is fixed.


2) modify these patches to add the pci-bridge device directly to pcie.0
and not add any dmi-to-pci-bridge in the middle (this will end up
encoded in the domain's config)

drawback: when seabios is fixed and libvirt switches to the "proper"
model of the hardware, there will be many configurations already out in
the field using the old model which will either need to be modified, or
libvirt will need to continue allowing/supporting that setup.


3) modify these patches to substitute a pci-bridge for the
i82801b11-bridge when someone requests a dmi-to-pci-bridge.

advantage: when seabios is fixed, configs already in the field will
continue to work

drawback: live migration of a domain from old to new will fail in some
strange as-yet-to-be-determined fashion. Also, until seabios is fixed
and we remove the patch to replace all i82801b11-bridge devices with
pci-bridge at runtime, there will be no testing of the i82801b11-bridge
code in qemu (at least not via libvirt-managed guests).


Presumably if I did option (2) or (3), this would be changed back at a
later date when seabios is fixed to recognize i82801b11-bridge properly.
Of course we will want libvirt to work properly no matter what is the
version of seabios/qemu, so we'll have to figure out a way to determine
if the seabios bugfix is in or not.


Does anyone have an opinion on which of these is preferred (or a
different idea)? I'm tending towards option (3), since that will cause
the least headaches later (there won't be a bunch of stale configs that
need to be updated). Whatever I do, it will be done in a separate patch
from this, because this patch does implement things in the "correct"
manner, and should stand; any hack to workaround the problem in seabios
should be a separate object in the git history.