[libvirt] [Qemu-ppc] [RFC PATCH qemu] spapr_pci: Create PCI-express root bus by default

[Marcel Apfelbaum]

On 12/13/2016 05:15 PM, Benjamin Herrenschmidt wrote:
> On Tue, 2016-12-13 at 14:25 +0200, Marcel Apfelbaum wrote:
>>>> Hrm, the suggestion of providing both a vanilla-PCI and PCI-E host
>>>> bridge came up before.  I think one of us spotted a problem with that,
>>>> but I don't recall what it was now.  I guess one is how libvirt would
>>>> map it's stupid-fake-domain-numbers to which root bus to use.
>>
>> This would be a weird configuration, I never heard of something like that
>> on a bare metal machine, but I never worked on pseries, who knows...
>
> That's the thing though, it's *not* bare metal ;-)
>
> On bare metal, we use the "powernv" platform for which we haven't yet upstreamed
> the PCIE support, but there we have real PCIe root ports with all what's exepcted
> on them.
>
> They come on physically different PHB, one root complex per PHB, but they are
> "proper" PCIe. Hotplug is a bit weird still because it has to go through some
> FW interactions as the HW doesn't use the PCIe standard slot control bits (and
> our qemu model doesn't handle it yet) but otherwise it's standard.
>

Thanks for the explanations, so bare metal is standard minus hotplug.

> "pseries" on the other hand is a paravirtualized platform. It's the environment
> presented to guests by our propritary hypervisor (PowerVM, aka pHyp) and
> KVM/qemu. I think there's a public spec these days but to cut the story short,
> it doesn't expose PCIe "properly" for bad historical reasons.
>
> It tends to hide the parent, ie, the root port for slots connected directly to
> a PHB or the entire hierarchy from the root to (and including) the downstream
> switch port for slots under as switch.
>
> So you end up with PCIe devices devoid of a proper "parent" which is a bit
> weird.
>

OK, now is more "clear". I can get back to read the thread from the start :)

Thanks,
Marcel

> Hotplug is implemented using specific firmware interfaces that are identical
> with pre-PCIe (ie PCI/PCI-X) systems. The FW has interface for supporting 3
> kinds of hotplug:
>
>    - Entire PHBs
>    - P2P Bridges
>    - Individual devices on an existing PHB or P2P bridge
>
> In practice these days mostly the first one is used for everything under pHyp,
> though I think we have a mix of 1 and 3 for KVM/qemu.
>
> Cheers,
> Ben.
>