To start multiple KVM guests from one qcow2 image with transient disk option

[Masayoshi Mizuma]

On Thu, Dec 17, 2020 at 10:39:36AM +0100, Peter Krempa wrote:
> On Mon, Dec 14, 2020 at 22:49:03 -0500, Masayoshi Mizuma wrote:
> > On Sat, Dec 12, 2020 at 11:57:15AM +0100, Peter Krempa wrote:
> > > On Fri, Dec 11, 2020 at 20:58:48 -0500, Masayoshi Mizuma wrote:
> > > > Hello,
> > > > 
> > > > I would like to start multiple KVM guests from one qcow2 image, and
> > > > discard the changes which the KVM guests done.
> > > > 
> > > > transient disk option is useful for discarding the changes, however,
> > > > we cannot start multiple KVM guest from one qcow2 image because the
> > > > image is write-locked by the first guest to be started.
> > > > 
> > > > I suppose the disk which transient option is enabled don't need to
> > > > get the write lock because any changes go to the overlay image, and
> > > > the overlay image is removed when the guest shutdown.
> > > > 
> > > > qemu has 'locking' option and the write lock is disabled when locking=off.
> > > > To implement that, I have two ideas. I would appreciate it if you could
> > > > give me the ideas which way is better (or another way).
> > > 
> > > There are two aspects of this.
> > > 
> > > 1) Controlling the locking property of qemu may be worth in many cases,
> > > so by itself this would be a worthwhile patchset to add control of
> > > qemu locking for a per-backing store basis.
> > > 
> > > 2) Disabling locking to achieve this is wrong though. The qemu image
> > > locking infrastructure is justified and prevents many problems which
> > > users might get into by wrong configuration.
> > > 
> > > For <transient/> disks, we should rather instantiate the top level
> > > format node as 'readonly' and then put a read-write overlay on top of
> > > it. This still prevents from using the file which became a backing file
> > > in read-write mode in another VM.
> > 
> > Thank you for the idea!
> > I just tried to change the original image to read-only (changed the "read-only":false
> > to "read-only":true) simply, then created a read-write overlay.
> > qemu stopped with following assertion:
> > 
> >   qemu-system-x86_64: ../block/io.c:1874: bdrv_co_write_req_prepare: Assertion `child->perm & BLK_PERM_WRITE' failed.
> > 
> > It looks like the qemu hit the assertion because the permission of the overlay image
> > is same as the original image.
> > Probably I'm missing something... I'll dive into that...
> 
> Could you please post the command line and the QMP commands? Maybe
> something isn't configured right in libvirt. Alternatively qemu might
> need modification.

Sure, here is the qemu command line options and QMP commands:

qemu command line options:

qemu-system-x86_64 \
	-M q35,accel=kvm,usb=off,vmport=off,smm=on,dump-guest-core=off \
	-smp 4 \
	-m 4096 \
	-blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/guest.qcow2","node-name":"storage1","auto-read-only":true,"discard":"unmap"}' \
	-blockdev '{"node-name":"format1","read-only":true,"driver":"qcow2","file":"storage1","backing":null}' \
	-device virtio-blk-pci,drive=format1,id=virtio-disk0,bootindex=1 \
	-nographic \
	-nodefaults \
	-no-user-config \
	-serial telnet::1235,server,nowait \
	-qmp tcp::1335,server,nowait \
	-S

QMP commands:
(Before running following QMP commands, I create '/var/lib/libvirt/images/guest.qcow2.TRANSIENT'
 by touch command)

   {"execute":"qmp_capabilities"}
   {"execute":"blockdev-add","arguments":{"driver":"file","filename":"/var/lib/libvirt/images/guest.qcow2.TRANSIENT","node-name":"storage2","auto-read-only":true,"discard":"unmap"}}
   {"execute":"blockdev-create","arguments":{"job-id":"create","options":{"driver":"qcow2","file":"storage2","size":10737418240,"cluster-size":65536,"backing-file":"/var/lib/libvirt/images/guest.qcow2","backing-fmt":"qcow2"}}}
   {"execute":"blockdev-add","arguments":{"node-name":"format2","read-only":false,"driver":"qcow2","file":"storage2","backing":null}}
   {"execute":"blockdev-snapshot","arguments":{"node":"format1","overlay":"format2"}}
   {"execute":"cont"}

After "cont" command executed, qemu stops as following assertion:

    qemu-system-x86_64: ../block/io.c:1865: bdrv_co_write_req_prepare: Assertion `child->perm & BLK_PERM_WRITE' failed.

Thanks,
Masa