[libvirt-users] Nested KVM: L0 guest produces kernel BUG on wakeup from managed save (while a nested VM is running)

[David Hildenbrand]

On 08.02.2018 15:59, Daniel P. Berrangé wrote:
> On Thu, Feb 08, 2018 at 02:47:26PM +0100, David Hildenbrand wrote:
>>> Sure, I do understand that Red Hat (or any other vendor) is taking no
>>> support responsibility for this. At this point I'd just like to
>>> contribute to a better understanding of what's expected to definitely
>>> _not_ work, so that people don't bloody their noses on that. :)
>>
>> Indeed. nesting is nice to enable as it works in 99% of all cases. It
>> just doesn't work when trying to migrate a nested hypervisor. (on x86)
> 
> Hmm, if migration of the L1 is going to cause things to crash and
> burn, then ideally libvirt on L0 would block the migration from being
> done.

Yes, in an ideal world. Usually we assume that people that turn on
experimental features ("nested=true") are aware of what the implications
are. The main problem is that the implications are not really documented :)

Once, with new KVM _and_ new QEMU it will eventually be supported.

> 
> Naively we could do that if the guest has vmx or svm features in its
> CPU, except that's probably way too conservative as many guests with
> those features won't actually do any nested VMs.  It would also be
> desirable to still be able to migrate the L1, if no L2s are running
> currently.

No using CPU feature flags for that purpose on the libvirt level is no
good, and especially once we support migration we would have to find
another interface to tell "but it is now working".

QEMU could try to warn the user if VMX is enabled in the CPU model, but
as you said, that might also hold true for guests that don't use nVMX.

On the other hand, VMX will only pop up as a valid feature if
nested=true is set. So the amount of affected users is minimal.

So we could e.g. abort migration on the QEMU level if VMX is specified
right now. Once we have the migration support in place, we can allow it
again.

> 
> Is there any way QEMU can expose whether there's any L2s activated
> to libvirt, so we can prevent migration in that case ? Or should
> QEMU itself refuse to start migration perhaps ?

Not without another kernel interface.

But I am no expert on that matter. Maybe there would be an easy way to
block that I just don't see right now.

> 
> 
> Regards,
> Daniel
> 


-- 

Thanks,

David / dhildenb