Libvirt on little.BIG ARM systems unable to start guest if no cpuset is provided

[Peter Maydell]

On Tue, 14 Dec 2021 at 09:52, Marc Zyngier <maz at kernel.org> wrote:
> You are still missing the point of how QEMU works:
>
> - QEMU creates a dummy VM with a single vcpu. This can happen on *any*
>   CPU.
> - It snapshots the sysregs for this vcpu, and keep them for later
> - It then destroy this VM
> - QEMU then creates the full VM, with all the vcpus
> - Each vcpu gets initialised with the state saved earlier. If any vcpu
>   is initialised on a physical CPU of a different type from the one
>   that has been used for the dummy VM, you lose, as we cannot restore
>   some of the registers such as MIDR_EL1 (and other registers that KVM
>   considers as invariant).

Put another way, QEMU's "-cpu host" is exactly one thing
(the "create a dummy VM and snapshot" steps above are where
QEMU defines what "-cpu host" means), and we have an implicit
assumption that the VM must only have one kind of guest CPU,
and not be heterogenous.

> The best workaround is to taskset the QEMU process (and I really mean
> the process, not individual threads) to an homogeneous set of CPUs and
> be done with it.

Agreed. I suspect that often the 'little' CPUs are sufficiently
low-power to probably not be worth giving to the VM anyway.

Side note: if you *do* give a guest both big and little CPUs
using kvmtool or something similar, does the guest kernel get
enough information to schedule tasks properly to both kinds of
CPU, or does it just assume they're all the same and happily
put performance-requiring tasks on the little CPUs ?

-- PMM