[libvirt-users] Change in treatment of qcow2 with chained backing files in v5.10.0?
Peter Krempa
pkrempa at redhat.com
Tue Dec 17 10:22:52 UTC 2019
On Mon, Dec 16, 2019 at 12:49:40 +0100, Jens John wrote:
> Hello list,
Hi Jens,
>
> upon upgrading from libvirt v5.9.0 -> v5.10.0, libvirt started calling
> qemu differently causing the VM operating system to not be found. I'm
> wondering whether I'm chasing a bug, or have to otherwise fix my domain
> config (created by virt-manager).
Yes, this change was done by me when enabling the new way to configure
block devices in qemu.
I think I know what's going on:
>
> To start with, I have a Windows 7 domain based on the following images:
>
> windows7-base.qcow2: QEMU QCOW2 Image (v3), 53687091200 bytes
> windows7-Office2013.qcow2: QEMU QCOW2 Image (v3), has backing file (path windows7-taintable.qcow2), 80530636800 bytes
> windows7-taintable.qcow2: QEMU QCOW2 Image (v3), has backing file (path windows7-base.qcow2), 53687091200 bytes
So this output was obtained by 'file'. That unfortunately doesn't give
enough information. Please run 'qemu-img info' on
those files and look for the 'backing file format:'field.
That field is important, because otherwise we'd fall back to image
format detection which was deemed unsafe and libvirt no longer allows
it.
$ qemu-img info pull4.qcow2
image: pull4.qcow2
file format: qcow2
virtual size: 10 MiB (10485760 bytes)
disk size: 196 KiB
cluster_size: 65536
backing file: pull3.qcow2
backing file format: qcow2
^^^^^^^^^^^^^^^^^^^^^^^^^^
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
The root of the problem is that the images mentioned above will not have
the format section because -F argument for qemu-img was not used when
creating them. This would force libvirt to use format detection which is
insecure and thus libvirt does not do that and assumes the image is raw.
This in normal circumstances causes problems right away, because when
running any more advanced security model than unix permissions (selinux
apparmor) libvirt would not allow qemu to access the backing file. You
don't seem to be running it so it was accidentally working for you until
now.
> The VM must be launched with image windows7-Office2013.qcow2.
>
> Using the domain config [1] with a <disk> section
>
> <disk type='file' device='disk'>
> <driver name='qemu' type='qcow2'/>
> <source file='/home/joj/lib/vm/windows/windows7-Office2013.qcow2'/>
> <target dev='vda' bus='virtio'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
> </disk>
>
> libvirt 5.9.0 is calling qemu 4.2.0 like [2], and libvirt 5.10.0 like [3], the
> differences being (5.9.0 1st, 5.10.0 2nd lines)
>
> [--drive file=/home/joj/lib/vm/windows/windows7-Office2013.qcow2,format=qcow2,if=none,id=drive-virtio-disk0-]
> {+-blockdev {"driver":"file","filename":"/home/joj/lib/vm/windows/windows7-taintable.qcow2","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"} -blockdev {"driver":"file","filename":"/home/joj/lib/vm/windows/windows7-Office2013.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":"libvirt-2-format"}+}
-blockdev {"driver":"file","filename":"/home/joj/lib/vm/windows/windows7-taintable.qcow2","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}
-blockdev {"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}
Here you can see that the
/home/joj/lib/vm/windows/windows7-taintable.qcow2 image was considered
being 'raw' this is usually because you didn't record the approprate
format. This means that qemu will no longer open the backing image.
This is the manifestation of the above problem.
-blockdev {"driver":"file","filename":"/home/joj/lib/vm/windows/windows7-Office2013.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}
-blockdev {"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":"libvirt-2-format"}+}
>
> -device [-virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1-]
> {+virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=libvirt-1-format,id=virtio-disk0,bootindex=1+}
>
> As you can see, 5.10.0 is expanding the backing file chain, not leaving
> that to qemu. Under 5.10.0, I get an Operating System not Found error,
> under 5.9.0, with the shorter -drive argument, everything works as
> desired.
>
> Can you perhaps advise me how I either fix the domain spec [1] to get
> libvirt to use a true equivalent to the old qemu invocation as I don't
> think this is a qemu problem, or how I can turn the new-style JSON
> parametrization off?
So as a first thing you should create the images properly. This means
that when you are creating the overlay image, you have to use the '-F'
parameter of qemu-img create. (Note that snapshots/overlays created by
libvirt always do this). The -F parameter specifies the format of the
backing image specified by -b:
qemu-img create -f qcow2 -F qcow2 -b windows7-taintable.qcow2 windows7-Office2013.qcow2
For fixing the image you can use the qemu-img rebase:
qemu-img rebase -f qcow2 -F qcow2 -b /home/joj/lib/vm/windows/windows7-taintable.qcow2 windows7-Office2013.qcow2
Please use full paths for the -b argument unless you want relative
relationships between images.
A second option is to specify the image chain in the XML. This works
starting from libvirt 5.10 :
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/home/joj/lib/vm/windows/windows7-Office2013.qcow2'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
<backingStore type='file'>
<format type='qcow2'/>
<source file='/home/joj/lib/vm/windows/windows7-taintable.qcow2'/>
<backingStore type='file'>
<format type='qcow2'/>
<source file='/home/joj/lib/vm/windows/windows7-base.qcow2'/>
<backingStore/>
</backingStore>
</backingStore>
</disk>
Note that this does not fix the images so they will work only with the
above XML.
As a last possibility you can disable the 'blockdev' feature in
/etc/libvirt/qemu.conf. I will not be more specific on how to do it
as it's mostly for debugging purposes and could break stuff in the
future.
Lastly this is more-or-less tracked in:
https://bugzilla.redhat.com/show_bug.cgi?id=1588373
I think I should finally fix it and report a proper error rather than
silently assuming it might work. With -blockdev it will never work
properly anyways.
>
> Other suggestions how to tackle this would also be much appreciated.
>
> ---
> [1] https://0x0.st/z0KM.xml
> [2] https://0x0.st/z0K1.txt
> [3] https://0x0.st/z0Kj.txt
>
>
> _______________________________________________
> libvirt-users mailing list
> libvirt-users at redhat.com
> https://www.redhat.com/mailman/listinfo/libvirt-users
More information about the libvirt-users
mailing list