[libvirt] [RFC] Add support for QEMU's l3-cache CPU property

[Daniel P. Berrange]

On Thu, Apr 20, 2017 at 11:53:21AM +0200, Jiri Denemark wrote:
> On Thu, Apr 20, 2017 at 10:02:38 +0100, Daniel P. Berrange wrote:
> > On Tue, Apr 18, 2017 at 04:22:36PM +0200, Jiri Denemark wrote:
> > > Hi,
> > > 
> > > Apparently, reporting a level 3 cache on a virtual CPU can dramatically
> > > increase performance in some use cases [1]. The interesting part is that
> > > l3-cache=on does not provide the real CPU cache data, it's just making
> > > it up. Anyway, we should be able to enable this via libvirt. And since
> > > there is another property which enables real CPU cache data to be passed
> > > to a guest, I suggest the following /domain/cpu/cache element equivalent
> > > to l3-cache=on:
> > > 
> > >     <cache level='3' mode='emulate'/>
> > > 
> > > If we need to add support for passing the real CPU cache data, we can do
> > > that with
> > > 
> > >     <cache level='3' mode='passthrough'/>
> > > 
> > > or we can even make the level attribute optional and support
> > > 
> > >     <cache mode='passthrough'/>
> > > 
> > > Missing cache element means default behaviour of the hypervisor and we
> > > can eventually add <cache mode='disable'/> to turn off passing any CPU
> > > cache info to the guest.
> > > 
> > > But I think we should now focus only on <cache level='3' mode='emulate'/>
> > > and leaving the rest for the future when we actually need it.
> > > 
> > > This is how a more complete example would look like:
> > > 
> > >     <cpu mode='custom' match='exact'>
> > >       <model>Broadwell</model>
> > >       <cache level='3' mode='emulate'/>
> > >     </cpu>
> > > 
> > > And libvirt would translate it into -cpu Broadwell,l3-cache=on.
> > > 
> > > Do you have any thoughts about the XML schema or naming?
> > 
> > The second QEMU property 'host-cache-info' causes the guest to see
> > the host cache topology. This affects L1, L2 and L3 caches all at
> > once. 
> > 
> > We could allow use of '<cache>' without specifying a level. ie
> > 
> >    <cache mode="passthrough"/>
> 
> Yes, this is what I suggested a few lines above. However, it seems
> host-cache-info is only supported for -cpu host.

I don't think there's any dependancy on -cpu host actually.

Semantically host-cache-info=on, only makes sense if the sockets/core/threads
topology of the guest, matches the host. What other CPUID bits you choose is
tangential.

> > to indicate passthrough of L1,L2 & L3 cache all together, mapping
> > to host-cache-info=on. and the <cache level=3 mode=emulate>
> > mapping to the l3-cache=on.
> > 
> > These two elements would need to be mutually exclusive.
> 
> Mixing them could allow host cache info to be passed through with
> specific level-N cache data overridden by additional <cache level='N'/>
> element. QEMU does not complain of both host-cache-info and l3-cache are
> enabled. But anyway, making them mutually exclusive is probably the

QEMU /should/ complain - the code completely ignores l3-cache=on if you
gave host-cache-info=on.

> safest option. However, what would happen if
> 
>     <cpu mode='host-passthrough'>
>       <cache level='3' mode='emulate'/>
>     </cpu>
> 
> is specified in domain XML?
> 
>     -cpu host,l3-cache=on
> 
> or
> 
>     -cpu host,host-cache-info=off,l3-cache=on

host-cache-info default to off, but having libvirt explicitly
give host-cache-info=off would be the paranoid safe approach.

> And similarly, I guess <cache mode="passthrough"/> should just
> unconditionally disable l3-cache if we want to be sure they are always
> mutually exclusive.

Yes, l3-cache defaults to on for new machine types, so for safety
sake we should explicitly give l3-cache=off, even though QEMU would
ignore it (for now).

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|