[resent] virt-manager connection fails with 'qemu unexpectedly closed the monitor'

Tue Dec 22 19:41:19 UTC 2020

On 12/21/20 6:15 AM, John Paul Adrian Glaubitz wrote:
> (CC'ing a friend who's run into the problem as well)
> 
> Hi Laine!
> 
> On 12/20/20 7:37 PM, Laine Stump wrote:
>> The first step would be to look in /var/log/libvirt/qemu/$guestname.log.
>> If qemu is encountering some error, it should be logging a message there
>> before it exits. (unless the error is a segfault; in that case I guess
>> you'll need to look for a coredump. It will also list exactly the qemu
>> commandline that was used.
> 
> Except for some unrelated warnings, qemu does not emit any error message:
[...]

> 
> 2020-12-21 11:10:37.580+0000: starting up libvirt version: 6.9.0, package: 1+b2 (amd64 / i386 Build Daemon (x86-ubc-01) <buildd_amd64-x86-ubc-01 at buildd.debian.org> Mon, 07 Dec 2020 09:45:52 +0000), qemu version: 5.2.0Debian 1:5.2+dfsg-2, kernel: 5.9.0-5-amd64, hostname: z6.physik.fu-berlin.de

At least you're running pretty much up-to-date of both libvirt and qemu, 
so you won't have to suffer through someone saying "come back when 
you've updated your software!" :-)

(although I suppose it's possible you've run into some recent regression 
that I haven't heard of, and all the people who have heard of it aren't 
responding because they gone off on their Christmas holidays already).

> LC_ALL=C \
> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
> HOME=/var/lib/libvirt/qemu/domain-1-windows7 \
> XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-1-windows7/.local/share \
> XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-1-windows7/.cache \
> XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-1-windows7/.config \
> QEMU_AUDIO_DRV=spice \
> /usr/bin/qemu-system-x86_64 \
> -name guest=windows7,debug-threads=on \
> -S \
> -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-windows7/master-key.aes \
> -machine pc-q35-5.2,accel=kvm,usb=off,vmport=off,dump-guest-core=off,memory-backend=pc.ram \
> -cpu Broadwell-IBRS,vme=on,ss=on,vmx=on,pdcm=on,f16c=on,rdrand=on,hypervisor=on,arat=on,tsc-adjust=on,umip=on,md-clear=on,stibp=on,arch-capabilities=on,ssbd=on,xsaveopt=on,pdpe1gb=on,abm=on,ibpb=on,amd-stibp=on,amd-ssbd=on,skip-l1dfl-vmentry=on,pschange-mc-no=on,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1fff \
> -m 4096 \
> -object memory-backend-ram,id=pc.ram,size=4294967296 \
> -overcommit mem-lock=off \
> -smp 2,sockets=2,cores=1,threads=1 \
> -uuid bb32a4b5-7b2c-4ac2-9a65-8fd2bb9cdb86 \
> -no-user-config \
> -nodefaults \
> -chardev socket,id=charmonitor,fd=33,server,nowait \
> -mon chardev=charmonitor,id=monitor,mode=control \
> -rtc base=localtime,driftfix=slew \
> -global kvm-pit.lost_tick_policy=delay \
> -no-hpet \
> -no-shutdown \
> -global ICH9-LPC.disable_s3=1 \
> -global ICH9-LPC.disable_s4=1 \
> -boot strict=on \
> -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 \
> -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \
> -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \
> -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \
> -device ich9-usb-ehci1,id=usb,bus=pcie.0,addr=0x1d.0x7 \
> -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pcie.0,multifunction=on,addr=0x1d \
> -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1 \
> -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2 \
> -device virtio-serial-pci,id=virtio-serial0,bus=pci.2,addr=0x0 \
> -blockdev '{"driver":"file","filename":"/data/kvm/windows7.img","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
> -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw","file":"libvirt-1-storage"}' \
> -device ide-hd,bus=ide.0,drive=libvirt-1-format,id=sata0-0-0,bootindex=1 \
> -netdev tap,fd=35,id=hostnet0 \
> -device e1000e,netdev=hostnet0,id=net0,mac=52:54:00:4b:c5:26,bus=pci.1,addr=0x0 \
> -chardev pty,id=charserial0 \
> -device isa-serial,chardev=charserial0,id=serial0 \
> -chardev spicevmc,id=charchannel0,name=vdagent \
> -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 \
> -device usb-tablet,id=input0,bus=usb.0,port=1 \
> -spice port=5900,addr=127.0.0.1,disable-ticketing,seamless-migration=on \
> -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 \
> -device ich9-intel-hda,id=sound0,bus=pcie.0,addr=0x1b \
> -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \
> -chardev spicevmc,id=charredir0,name=usbredir \
> -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 \
> -chardev spicevmc,id=charredir1,name=usbredir \
> -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 \
> -device virtio-balloon-pci,id=balloon0,bus=pci.3,addr=0x0 \
> -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
> -msg timestamp=on

Maybe try running this command directly from the commandline (after 
removing the "-netdev tap" and "-device e1000e" lines, since they 
require pre-creation/opening of a tap device) and see if it stays up or 
dies. If it dies, then retry it removing more and more of the 
commandline options until it runs successfully (note that many of the 
options come in pairs - e.g. the "-device ich9-intel-hda" and "-device 
hda-duplex" lines must be removed together, similarly for the sets of 
"-chardev spicevmc" and "-device usb-redir"). If you can keep track, use 
a bisecting strategy to save time. Once you've narrowed it down to a 
single option, maybe it will be easier to diagnose.

A similar strategy you could use at the libvirt level would be to use 
"virsh define" (run as root) and give it the XML you captured as the 
definition. Then do the same thing - repeatedly try to "virsh start", 
and then when it fails, use "virsh edit" to remove some of the 
devices/options and try again.

Good luck!

> char device redirected to /dev/pts/0 (label charserial0)
> 2020-12-21T11:10:37.974205Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(48FH).vmx-exit-load-perf-global-ctrl [bit 12]
> 2020-12-21T11:10:37.974284Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
> 2020-12-21T11:10:37.976069Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(48FH).vmx-exit-load-perf-global-ctrl [bit 12]
> 2020-12-21T11:10:37.976089Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]

One suggestion I saw for avoiding these warnings is to set 
"host-passthrough" for the CPU model rather than "host-model". No idea 
if it works though.

> 2020-12-21 11:10:38.207+0000: shutting down, reason=failed
> 
>> If the message there isn't illuminating enough and you come back here, it would be helpful
>> to "bring along" that qemu commandline (and any accompanying error) as well as the output
>> of "virsh dumpxml $guestname"
> 
> Since libvirt won't create the guest instance in the first place, dumping the XML of the config
> like that won't work. However, it's possible to view the XML from the preview window in the
> machine details:
> 
> <domain type="kvm">
>    <name>windows7</name>
>    <uuid>bb32a4b5-7b2c-4ac2-9a65-8fd2bb9cdb86</uuid>
>    <metadata>
>      <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
>        <libosinfo:os id="http://microsoft.com/win/7"/>
>      </libosinfo:libosinfo>
>    </metadata>
>    <memory>4194304</memory>
>    <currentMemory>4194304</currentMemory>
>    <vcpu>2</vcpu>
>    <os>
>      <type arch="x86_64" machine="q35">hvm</type>
>      <boot dev="hd"/>
>    </os>
>    <features>
>      <acpi/>
>      <apic/>
>      <hyperv>
>        <relaxed state="on"/>
>        <vapic state="on"/>
>        <spinlocks state="on" retries="8191"/>
>      </hyperv>
>      <vmport state="off"/>
>    </features>
>    <cpu mode="host-model"/>
>    <clock offset="localtime">
>      <timer name="rtc" tickpolicy="catchup"/>
>      <timer name="pit" tickpolicy="delay"/>
>      <timer name="hpet" present="no"/>
>      <timer name="hypervclock" present="yes"/>
>    </clock>
>    <pm>
>      <suspend-to-mem enabled="no"/>
>      <suspend-to-disk enabled="no"/>
>    </pm>
>    <devices>
>      <emulator>/usr/bin/qemu-system-x86_64</emulator>
>      <disk type="file" device="disk">
>        <driver name="qemu" type="raw"/>
>        <source file="/data/kvm/windows7.img"/>
>        <target dev="sda" bus="sata"/>
>      </disk>
>      <controller type="usb" model="ich9-ehci1"/>
>      <controller type="usb" model="ich9-uhci1">
>        <master startport="0"/>
>      </controller>
>      <controller type="usb" model="ich9-uhci2">
>        <master startport="2"/>
>      </controller>
>      <controller type="usb" model="ich9-uhci3">
>        <master startport="4"/>
>      </controller>
>      <interface type="network">
>        <source network="default"/>
>        <mac address="52:54:00:4b:c5:26"/>
>        <model type="e1000e"/>
>      </interface>
>      <console type="pty"/>
>      <channel type="spicevmc">
>        <target type="virtio" name="com.redhat.spice.0"/>
>      </channel>
>      <input type="tablet" bus="usb"/>
>      <graphics type="spice" port="-1" tlsPort="-1" autoport="yes"/>
>      <sound model="ich9"/>
>      <video>
>        <model type="qxl"/>
>      </video>
>      <redirdev bus="usb" type="spicevmc"/>
>      <redirdev bus="usb" type="spicevmc"/>
>    </devices>
> </domain>
> 
> Adrian
> 