[virt-tools-list] [virt-manager PATCH 5/5] virt-manager: add a new guest feature GIC for AMR guests

[Andrea Bolognani]

I'm CCing Drew so that he can point out any mistakes I might
have made below :)

On Sun, 2016-06-12 at 11:35 +0200, Pavel Hrdina wrote:
> On Sat, Jun 11, 2016 at 12:04:35PM -0400, Cole Robinson wrote:
> > 
> > Okay, I need to finally truly figure out how all this stuff fits together to
> > try and understand if it makes sense to expose in the GUI. My understanding is
> > that there's 4 pieces of info at play here:
> > 
> > - What gic versions can qemu emulate? (available for domain type=qemu)
> > - What gic versions does the host support? (available for domain type=kvm)
> > - What gic versions does the guest OS support? (eventually tracked by libosinfo)
> > 
> > As of now there's currently only 3 possible GIC settings: version=2,
> > version=3, version=host (kvm only, basically like -cpu host)
> > 
> > Libvirt will default to the highest gic version that qemu advertises support
> > for. That will be gic v2 for type=qemu (the only version qemu presently
> > emulates), and whatever the host supports for type=kvm
> > 
> > That's basically the extent of my knowledge, so I'd like to know:
> > 
> > - does gic emulation have any impact on type=kvm, or is it only relevant to
> > type=qemu? (I assume the latter)
> 
> As far as I was able to find GIC emulation is valid only for QEMU and KVM
> requires HW support.  However there is an ongoing work to add emulated GICv3 for
> QEMU (TCG).

That's my understanding as well.

Note that QEMU has a 'kernel_irqchip' option that allows you
to use KVM for the CPU while offloading the GIC emulation to
QEMU, which would in theory allow you to run GICv2 guests on
a GICv3 only host.

This has been deemed not worth exposing through libvirt
because it's mostly geared towards debugging the emulated
GIC implementation and not general use.

> > - do aarch64 hosts only support one gic version, or does say a v3 host also
> > support v2?
> 
> Backward support is optional so if host supports v3 it can but doesn't have to
> support v2.  So there could be hosts only with v3 support but also host with v2
> and v3 support.
> 
> > - do guest OS support multiple gic versions? like does a guest OS that
> > supports v3 also support v2? if a guest OS doesn't support a gic version, does
> > that mean it completely fails to boot, or some other failure scenario?
> 
> If a OS doesn't support GIC it probably doesn't support ARM at all.
> 
> > - what gic versions to common guest OS support? rhelsa, fedora 22, 23, 24 etc.
> 
> First support for GIC was introduced in kernel 2.6.14.
> GICv2m is supported from 3.19
> GICv3 is supported from 3.17
> 
> This means that all of that OSes support up to GICv3.

We confirmed this by creating a Fedora 22 guest on a GICv3
only host. As far as guest OS support is concerned, we
should be good.

> > - besides the OS compat issue, what does gic v3 vs v2 actually provide?
> > something super critical for install time?
> 
> GICv2 adds a virtualization supports.

I don't think we care about this for guests :)

> GICv2 has some limitations, for example only 8 cpus.
> GICv2m adds MSI and MSI-X support.
> GICv3 is an update of GICv2 and adds MSI and MSI-X support, and removes the cpu
> limitation and adds some other features.

Other than the CPU limit, none of these look like it would
prevent a guest from booting.

> > - is gic setting something that can be changed later without guest impact?
> 
> I guess that it's safe to change GIC version for a guest.  But if you would
> change GIC from v3 to v2 and the guest has more than 8 cpus KVM wouldn't allow
> to start that guest.

We tried changing a RHEL 7.3 guest from GICv3 to GICv2, after
moving it from a GICv3 only to a GICv2 only host, and it
seemed to be totally fine with the change. I assume going the
other way around would be just as smooth.

This would probably be a guest ABI change, though? Seems like
the kind of change that would require a Windows guest to be
reactivated.

> > I'm trying to answer the question 'when do users want to change that default',
> > outside of obvious dev/testing scenarios. If there isn't presently, or planned
> > for the near future, a critical case where the user needs to change the
> > default just to get a booting/installing VM, then I'd rather not put this in
> > the GUI and just save it for virt-install/virt-xml
> 
> I understand this point.  The least we should do in GUI is to display the GIC
> version.  There is a potential use-case that if you migrate a guest from GICv2
> only host to GICv3 host, you would probably like to change the GIC version and
> also if you have an OS that supports only GICv2, you would probably want to set
> GIC version.  This would be also nice to get from libosinfo.

Moreover, I guess you might want to create a guest on a host
supporting both GICv2 and GICv3, with the intention of later
migrating it between that host and one that only supports
GICv2. Not sure whether that could be considered a critical
use case.

Displaying it in the GUI would almost certainly be good.

-- 
Andrea Bolognani
Software Engineer - Virtualization Team