[libvirt-users] Nested KVM: L0 guest produces kernel BUG on wakeup from managed save (while a nested VM is running)

[Kashyap Chamarthy]

On Thu, Feb 08, 2018 at 12:48:46PM +0100, David Hildenbrand wrote:
> On 08.02.2018 11:46, Kashyap Chamarthy wrote:
> > On Wed, Feb 07, 2018 at 11:26:14PM +0100, David Hildenbrand wrote:
> >> On 07.02.2018 16:31, Kashyap Chamarthy wrote:
> > 
> > [...]
> > 
> >> Sounds like a similar problem as in
> >> https://bugzilla.kernel.org/show_bug.cgi?id=198621
> >>
> >> In short: there is no (live) migration support for nested VMX yet. So as
> >> soon as your guest is using VMX itself ("nVMX"), this is not expected to
> >> work.
> > 
> > Actually, live migration with nVMX _does_ work insofar as you have
> > _identical_ CPUs on both source and destination — i.e. use the QEMU
> > '-cpu host' for the L1 guests.  At least that's been the case in my
> > experience.  FWIW, I frequently use that setup in my test environments.
> > 
> 
> Your mixing use cases. While you talk about migrating a L2, this is
> about migrating an L1, running L2.

Yes, you're right.  I mixed up briefly, and corrected myself in the
other email.  We're on the same page.

> Migrating an L2 is expected to work just like when migrating an L1, not
> running L2. (of course, the usual trouble with CPU models, but upper
> layers should check and handle that).

Yep.

---

Aside:

I also remember seeing Vitaly's nice talk[*] at FOSDEM last weekend ("A
slightly different kind of nesting"), where he talks about how nVMX
actually works in context of Intel's hardware feature "VMCS Shadowing",
to reduce number of VMEXITs and VMENTRYs.  

(Particularly look at his slides 8, 9 and 10.)

I reproduced his diagram from his slide-10 ("How nested virtualization
really works on Intel") in ASCII here:

    .---------------------------------------.
    |           | VMCS L1->L2 |             |
    |           '-------------'             |   
    |                   |                   |
    | L1 (guest         |  L2 (nested       | 
    |     hypervisor)   |      guest)       |
    |                   |                   |
    |                   |                   |
    .---------------------------------------.
    |  VMCS L0->L1      |   VMCS  L0->L2    |
    .---------------------------------------.
    |                                       |
    |              L0 hypervisor            |
    '---------------------------------------'
    |                                       |
    |               Hardware                |
    '---------------------------------------'


[*] https://fosdem.org/2018/schedule/event/vai_kvm_on_hyperv/attachments/slides/2200/export/events/attachments/vai_kvm_on_hyperv/slides/2200/slides_fosdem2018_vkuznets.pdf

[...]

-- 
/kashyap