Add option to skip cpu feature and model verification in libvirt and rely on qemu 'enforce' option

[Jiri Denemark]

On Wed, Nov 23, 2022 at 11:34:24 +0000, John Levon wrote:
> On Wed, Nov 23, 2022 at 12:29:24PM +0100, Jiri Denemark wrote:
> 
> > > I really don't think we should allow this, especially not for something
> > > which is clearly intended to be used for production deployment. Our
> > > hypervisor CLI passthrough is there to allow for short term workarounds
> > > for bugs, or to experiment with a feature before it is mapped into
> > > libvirt in a supported manner.
> > > 
> > > If there are aspects related to QEMU configuration thiat are not in
> > > libvirt yet, we need to address those gaps, not provide yet another
> > > way to bypass libvirt mapping.
> > 
> > Indeed, this was definitely meant as a short term workaround for stuff
> > we don't have support for yet, for testing or experimental purposes. The
> > supported approach is for implement the missing parts in libvirt (and
> > QEMU) as soon as possible. I don't see why would a properly documented
> > support for experiments with -cpu would be an issue.
> > 
> > If anyone thinks it's a good idea to use such stuff for anything even
> > remotely close to production then that's their call. We can't really
> > prevent that. Just as with any other <qemu:...> stuff we already have.
> 
> Having two copies of essentially the same information that must be kept
> perfectly in sync has been the direct cause of multiple serious bugs for us
> (some self-inflicted, probably, but certainly not all). I understand that "fix
> libvirt every time there's a disparity" is appealing but unfortunately in
> practice this just hasn't been the case.

Well, but that's not how it is all supposed to work. First, libvirt and
QEMU do not strictly need to be in sync with everything.

The important part is that features are in sync (pretty much trivial and
easy, it's just that someone needs to be motivated to send a patch).
Libvirt needs to understand the features to be able to report host
capabilities. Not to mention that QEMU changed names of several features
and even deprecated the old spellings which is completely transparent to
libvirt users as they can still use the old names no matter what version
of QEMU they use. This is one of the key benefits of libvirt.

CPU models need to be added to libvirt too, but their definition does
not have to be in sync with QEMU. And as far as I can tell the goal is
to dynamically probe the models and their details from QEMU so that we
don't need to keep our own copy of CPU models. This approach should be
possible (and it's actually a requirement due to the increased number of
supported CPU models) with versioned CPUs, but we need some more support
for this probing in QEMU.

Anyway, with the currently suggested use case we should have a single
source of truth and that is QEMU. In domain capabilities XML we report
what CPU models QEMU can run on a given host. You can take any of these
models and use them with <cpu check='none' ...> to start a domain with
that CPU models whatever that means to QEMU. Libvirt then translates the
created CPU to its internal (so far) definition which is stable across
all versions of libvirt to make sure any version of libvirt can
understand it (during migration, save/restore, snapshots, upgrade) and
can ensure the CPU does not change in any way until the domain shuts
down.

In other words, libvirt code should be in a state when differences
between QEMU and libvirt in CPU model definitions should not matter.
Modulo bugs, of course, but such bugs should be reported, analyzed and
fixed similarly to any other bugs in any software.

> We're struggling to see what value the libvirt layer is bringing us in this
> area, as opposed to a single-source-of-truth approach.

The ABI stability and compatibility with different versions of QEMU and
libvirt. We definitely want to minimize duplicity and use QEMU as much
as possible to implement our APIs and semantics. For example, the all of
the CPU model related functionality is offloaded to QEMU on s390. But
x86_64 lacks the required parts no to mention that the CPU modelling
there is much more complicated.

Jirka