[libvirt] ARM KVM GICv3 Support

[Christoffer Dall]

On Mon, Dec 14, 2015 at 10:47:26AM -0600, Andrew Jones wrote:
> On Mon, Dec 14, 2015 at 12:31:59PM +0100, Christoffer Dall wrote:
> > Hi all,
> > 
> > I'm trying to figure out what the correct solution for libvirt support
> > for ARM KVM guests with GICv3 is.
> > 
> > The challenge is that QEMU's virt machine model defaults to GICv2 unless
> > you specify an additional "-machine gic-version=host" parameter to QEMU,
> > in which case your VM gets the same GIC version as your host.
> > 
> > Now, I'm told that this is inconvenient for libvirt, because it
> > typically does not pass any -machine options, is that correct?
> 
> If that is correct, then my guess is that that's because it's used to
> the qemu machine model automatically selecting appropriate defaults, and
> for pc, due to versioning, there are several machine model types to
> choose from. mach-virt hasn't started versioning yet.
> 
> > 
> > Further, there are two additional complications: First, some guest
> > images may not include GICv3 support.  For Linux, this is relatively old
> > kernels (prior to v3.17), but other guests may only support GICv2.
> > 
> > Second, some hardware platforms only support GICv2 guests, some only
> > support GICv3 guests, and some support both GICv2 and GICv3 guests
> > (those hosts with a GICv3 with the backwards compatibility support).
> > 
> > It is unclear to me how libvirt users will relate to these constraints.
> > For example, in an OpenStack deployment, will it somehow be easy to tell
> > which nodes support which GIC version such that administrators can
> > choose compatible VM images, or how does that work?
> 
> libvirt has the concept of capabilities that it can negotiate. I know
> this is used heavily with x86 cpu features that some guests depend on.
> I think a guest dependency on gic version could fit here.
> 

Sounds fair to me, but I don't know what is required to actually
implement/use this for GIC support on ARM?

> > 
> > The end goal here is to determine if we need to add any functionality to
> > QEMU to better support libvirt, and what kind of features/glue need to
> > be added to libvirt for this to work.
> 
> I've been starting to wonder if we should consider breaking mach-virt
> into multiple types. For example, one that focuses on big, enterprise-y
> type configs (only 64-bit guests, only virtio-pci, extending the size
> of main memory beyond 30G, for sure, and possibly even moving the
> PCIE_MMIO_HIGH mapping to extend beyond 511G. It'd even be good to
> remap the redistributor space to get more than 123 vcpus.

I kind of feel that we should wait until someone says they're trying to
run VMs that they cannot, which we haven't heard about yet, AFAIK.

My fear with the multiple mach-virt versions is a lot of duplicated code
in QEMU or at least a much more complicated board definition.  Peter and
QEMU developers will have better and stronger opinions on this than me
though.

> 
> Making new mach-virt types and versioning those types would probably
> avoid needing much libvirt changes, but teaching libvirt (if it doesn't
> know already) how to manage machine parameters to effectively create new
> types as needed, also sounds good to me.
> 

It's not clear to me if adding a new mach-virt type solves the GICv3
question or if that is an othorgonal discussion.

-Christoffer