[libvirt] [RFC] qemu: Redesigning guest CPU configuration

[Jiri Denemark]

Hi all (and sorry for the long email),

The current way QEMU driver handles guest CPU configuration is not
ideal. We detect host CPU capabilities only by querying the CPU and we
don't check with QEMU what features it supports. We don't check QEMU's
definitions of CPU models, which may be different from libvirt's
definitions. All this results in several issues:

- guest CPU may change druing migration, save/restore
- libvirt may ask for a CPU which QEMU cannot provide; the guest will
  see a slightly different CPU but libvirt client won't know about it
- libvirt may come up with a CPU that doesn't make sense and which won't
  work for a guest (the guest may even crash)

Although usually everything just works, it is very fragile.

Since we want to fix all these issues, we need to:
- guarantee stable guest ABI (a single domain XML should always results
  in the same guest ABI). Once a domain is started, its CPU definition
  should never change (unless someone changes the XML, of course,
  similar to, e.g. PCI addresses). However, there are a few exceptions:
    - host-passthrough CPU mode will always result in "-cpu host"
    - host-model CPU mode should recompute the CPU model on every start,
      but the CPU must not change during migration
- always make sure QEMU provides the CPU we asked for. Starting a domain
  should fail in case QEMU cannot provide exactly the CPU we asked for.
- provide usable host-model mode and custom mode with minimum match. We
  need to generate CPU configurations that actually work, i.e., we need
  to ask QEMU what CPU it can provide on current host rather than
  requesting a bunch of features on top of a CPU model which does not
  always match the host CPU.

QEMU already provides or will soon provide everything we need to meet
these requirements:
- we can cover every configurable part of a CPU in our cpu_map.xml and
  instead of asking QEMU for a specific CPU model we can use "-cpu
  custom" with a fully specified CPU
- we can use the additional data about CPU models to choose the right
  one for a host CPU
- when starting a domain we can check whether QEMU filtered out any of
  the features we asked for and refuse to start the domain
- we can ask QEMU what would "-cpu host" look like and use that for
  host-model and minimum match CPUs (it won't work for TCG mode, though,
  but we can keep using the current CPUID detection code for TCG)

Once we start maintaining CPU models with all the details, we will
likely meet the same issues QEMU folks meet, i.e., we will need to fix
bugs in existing CPU models. And it's not just about adding removing CPU
features but also fixing other parameters, such as wrong level, etc.
It's clear every change will require a new CPU model to be defined. But
I think we should do it in a way that applications or users should not
need (if they don't want to) to care about it. I'm thinking about doing
something similar to machine types. Each CPU model could be defined in
several versions and a CPU specs without a version would be an alias to
the latest version.

The problem is, we need to maintain backward compatibility and we should
avoid breaking existing domains (shouldn't we?) which just work even
though their guest CPUs do not exactly match the domain XML definitions.
So either we need to define all existing CPU models in all their
variants used for various machine types and have a mapping between
(model without a version, machine type) to a specific version of the
model (which may be quite hard) or we need to be able to distinguish
between an existing domain and a new domain with no CPU model version.
While host-model and host-passthrough CPU modes are easy because they
are designed to change everytime a domain starts (which means we don't
need to be able to distinguish between existing and new domains), custom
CPU mode are tricky. Currently, the only at least a bit reasonable thing
which came to my mind is to have a new CPU mode, but it still seems
awkward so please share your ideas if you have any.

BTW, I don't think we should try to expose every part of the CPU model
definitions in domain XML, they should remain hidden behind the CPU
model name. It would be hard to explain what each of the extra
parameters mean, each model would have to include them anyway since we
can't expect users to provide all the details correctly, and once
visible in domain XML it could encourage users to play with the values.

I'm looking forward to any comments or ideas.

Jirka