[libvirt] new libvirt "pci" controller type and pcie/q35 (was Re: [PATCH 4/7] add pci-bridge controller type)

[Daniel P. Berrange]

On Fri, Apr 12, 2013 at 11:46:15AM -0400, Laine Stump wrote:
> On 04/11/2013 07:23 AM, Michael S. Tsirkin wrote:
> > On Thu, Apr 11, 2013 at 07:03:56AM -0400, Laine Stump wrote:
> >> On 04/10/2013 05:26 AM, Daniel P. Berrange wrote:
> >>>>>
> >>>>> So if we later allowed for mutiple PCI roots, then we'd have something
> >>>>> like
> >>>>>
> >>>>>    <controller type="pci-root" index="0">
> >>>>>      <model name="i440FX"/>
> >>>>>    </controller>
> >>>>>    <controller type="pci-root" index="1">
> >>>>>      <model name="i440FX"/>
> >>>>>    </controller>
> >>>>>    <controller type="pci" index="0"> <!-- Host bridge 1 -->
> >>>>>      <address type='pci' domain='0' bus='0' slot='0''/>
> >>>>>    </controller>
> >>>>>    <controller type="pci" index="0"> <!-- Host bridge 2 -->
> >>>>>      <address type='pci' domain='1' bus='0' slot='0''/>
> >>>>>    </controller>
> 
> 
> There is a problem here - within a given controller type, we will now
> have the possibility of multiple controllers with the same index - the
> differentiating attribute will be in the <address> subelement, which
> could create some awkwardness. Maybe instead this should be handled with
> a different model of pci controller, and we can add a "domain" attribute
> at the toplevel rather than specifying an <address>?

IIUC there is a limit on the number of PCI buses you can create per
domain, due to fixed size of PCI addresses. Google suggests to me
the limit is 256. So for domain 1, we could just start index at
256, and domain 2 at 512, etc


> Comment: I'm not quite convinced that we really need the separate
> "pci-root" device. Since 1) every pci-root will *always* have either a
> pcie-root-bus or a pci-bridge connected to it, 2) the pci-root-bus will
> only ever be connected to the pci-root, and 3) the pci-bridge that
> connects to it will need special handling within the pci-bridge case
> anyway, why not:
> 
> 1) eliminate the separate pci-root controller type

Ok, lets leave it out - we can always add it later if desired.

> 2) within <controller type='pci'>, a new <model type='pci-root-bus'/>
> will be added.
> 
> 3) a pcie-root-bus will automatically be added for q35 machinetypes, and
> pci-root-bus for any machinetype that supports a PCI bus (e.g. "pc-*")
> 
> 4) model type='pci-root-bus' will behave like pci-bridge, except that it
> will be an implicit device (nothing on qemu commandline) and it won't
> need an <address> element (neither will pcie-root-bus).
> 
> 5) to support multiple domains, we can simply add a "domain" attribute
> to the toplevel of controller.

Or use index numbers modulo 256 to identify domain numbers.


One note on q35 - we need to make sure whatever we do in terms of creating
default <controller>s in the XML 'just works' for applications. eg if they
define a guest using  <type machine="q35">hvm</type>, and then add a
<interface>, it should do the right thing wrt PCI addressing/connectivity.
We must not require applications to manually add <controller> elements
for q35 for things to work. Adding <controller>s must purely be an opt-in
for apps which have the detailed knowledge rquired & need full control
over bus layout.

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|