[libvirt] [Qemu-devel] ARM KVM GICv3 Support

[Andrew Jones]

On Tue, Feb 02, 2016 at 03:05:28PM +0100, Christoffer Dall wrote:
> On Tue, Feb 02, 2016 at 01:59:26PM +0100, Andrew Jones wrote:
> > On Tue, Feb 02, 2016 at 12:10:10PM +0000, Daniel P. Berrange wrote:
> > > On Tue, Feb 02, 2016 at 12:49:33PM +0100, Christoffer Dall wrote:
> > > > On Fri, Jan 22, 2016 at 02:44:32PM +0000, Daniel P. Berrange wrote:
> > > > > On Wed, Jan 06, 2016 at 01:30:16PM +0000, Peter Maydell wrote:
> > > > > > On 6 January 2016 at 12:49, Andrea Bolognani <abologna at redhat.com> wrote:
> > > > > > > That's correct, having a QMP command that lists the values gic-version
> > > > > > > can have on the current host would be just great.
> > > > > > >
> > > > > > > If we had that, we could validate the GIC version chosen for a guest,
> > > > > > > and expose it in the capabilities XML so that higher-level tools can
> > > > > > > provide a list of choices to the user.
> > > > > > >
> > > > > > > Please note that this QMP command would have to work regardless of the
> > > > > > > machine type selected on QEMU's command line, because libvirt always
> > > > > > > runs a QEMU binary with '-M none' when probing its capabilities.
> > > > > > 
> > > > > > On the other hand, if you don't tell us the machine type you care
> > > > > > about then we can't tell you:
> > > > > >  (a) "this machine type doesn't support setting this property at all"
> > > > > >      (which applies to machines like vexpress-a15 which you can use with
> > > > > >      KVM on 32-bit hosts, and of course also to all the non-KVM models)
> > > > > 
> > > > > We have just recently merged support for registering properties against
> > > > > classes instead of object instances. There is also a proposed command
> > > > > to allow querying the list of properties against a class
> > > > > 
> > > > >   https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg04348.html
> > > > > 
> > > > > So if we now update the machine types to register their properties against
> > > > > the class instead of object, then we can query what properties are present
> > > > > against each machine type, while still using '-M none'.
> > > > > 
> > > > > >  (b) "this machine type only supports GIC versions X and Y even if the
> > > > > >       host supports more" (this is currently only hypothetical, though,
> > > > > >       since we only have the property on 'virt'. it would only happen
> > > > > >       if in the future we needed something other than '2' or '3' or
> > > > > >       'host' I think.)
> > > > > 
> > > > > Our introspection support in QOM only allows us to say that a property
> > > > > is a particular type (int / enum / str / whatever). We don't have any
> > > > > way to expose info about what subset of possible values for a type are
> > > > > permitted. So I don't see any near term way to inform apps that the
> > > > > gic property accepts values x, y and but not z.
> > 
> > This actually doesn't matter for the v2 vs. v3 case. The gic-version property
> > doesn't exist at all for v2-only QEMU. Although maybe the gic-version property
> > should be reworked. Instead of gic-version=<highest-supported>, we could create
> > one boolean property per version supported, i.e. gicv2, gicv3, gicv4...
> > 
> > > > > 
> > > > > IMHO, we shouldn't try to overthink this. Libvirt can query the host
> > > > > to find out what GIC versions are supported and default to using the
> > > > > most recent version, on the basis that people are likely to have a
> > > > > matching QEMU. We can just rely on QEMU to report error if we pass
> > > > > it a version it doesn't support and not try to pre-emptively check
> > > > > before launch. The key is just getting the default right IMHO.
> > > > > 
> > > > This sounds fine to me.
> > > > 
> > > > However, not being familiar with the internals of libvirt I really can't
> > > > just what the right implementation approach here is.
> > > > 
> > > > As I understand we need to either:
> > > > 
> > > >   1) Add a QMP command that lets you ask for -M virt, if GIC version X
> > > >      is supported
> > > > 
> > > >   2) Just implement something in libvirt that checks what the kernel
> > > >      supports directly via the well-defined KVM interface and chooses
> > > >      the highest supported version per default.
> > > > 
> > > > To me it sounds like we should just go ahead with (2) and document
> > > > somehwere that if you get an error, you need to either manually change
> > > > the gic version setting in your VM definition file or upgrade QEMU.
> > > > 
> > > > Can someone with libvirt authority please advice whether (1) or (2) is
> > > > what we need to do?
> > > 
> > > IMHO we should just go for 2.
> > 
> > I'm not familiar enough with libvirt, nor the use of QMP, to really argue
> > one way or another, but I find it a bit strange that we'd prefer libvirt
> > to query two entities over one. And, why should the libvirt installed on
> > a particular host prefer gicv3 as the default, just because KVM supports
> > it, even when QEMU does not? 
> 
> I think the assumption here is that if you install a recent libvirt you
> also install a recent QEMU.  You always have the risk of things not
> working if you have too old a QEMU, right?

I'll grant that recent libvirt probably does imply recent QEMU. If we
need to make assumptions for this solution, that's probably not a
horrible one.

> 
> > The default will fail every time until QEMU
> > is upgraded, which may not be necessary/desired. Finally, I thought we
> > were trying to get away from relying on QEMU's error messages to make any
> > sort of decisions.
> 
> This I have no idea, if libvirt wants to go in the direction of doing
> anything humanly possible to avoid QEMU errors, then (2) is probably not
> the right approach.
> 
> On the other hand, there's always the possibility that something fails,
> so you probably always have to deal with QEMU's errors.  I don't think
> anyone here suggested that libvirt should try to run QEMU with
> something, catch an error, and then run QEMU with other options, if
> that's what you meant?

I was thinking that, but now, after rereading, I see the proposal was
just to propagate the error. So any error results in the same decision,
which is just to quit, and that's reasonable.

> 
> > 
> > I don't know what else libvirt queries directly from KVM, but IMO, it
> > should be a long-term goal to stop doing it, not add to more. Besides
> > libvirt then properly selecting defaults that both KVM and QEMU support,
> > it would allow /dev/kvm to have QEMU-only selinux labels applied.
> > 
> That's really for the libvirt community and maintainers to say, which
> direction they're going in.

Agreed. I'm just thinking out loud, perhaps too noisily...

> 
> As I've tried to indicate before, we don't mind doing some work here as
> a bunch of Linaro members care about this, but we just want to know from
> the libvirt people what they want...?

Yup. I look forward to more of them chiming in. I believe Andrea has
been doing some experimenting.

Thanks,
drew