Libvirt + Debian Live = Heart Attack

Wed Nov 24 17:21:22 UTC 2021

Hey Martin & Michal, thank you both for the replies!

Yes, sorry I messed up that seclabel entry, thanks. I fixed it and there is
no more error, but the images are still copied unfortunately.

@Michal yes, I have been reading the Debian Live Manual like a contract,
after messing with qemu.conf and permissions for days, I think its
something in the live build and not libvirt.

@Martin yeah both virsh start and virt-manager cause the images to be
copied to /run..... first.

Yes, when I boot the Debian Live ISO, plug in the USB (mounted rw), copy
the images to the live system from the USB and click run in virt-manager,
the VM starts instantly and no copies are made in /run....

I use ISO loopback and toram to load the system directly into ram via tmpfs
mount. That's my grub.cfg

menuentry "debian-live.iso" { set iso="/ISO/debian-live.iso" set
root=(hd0,5) loopback loop $iso linux (loop)/live/vmlinuz-4.19.0-6-amd64
boot=live components toram findiso=$iso initrd
(loop)/live/initrd.img-4.19.0-6-amd64 }

So, the squashfs is mounted read-only in
/run/live/rootfs/filesystem.squashfs and its /dev/loop1

/proc/mounts says:

tmpfs /run/live/overlay rw
overlay rw  lowerdir=/run/live/rootfs/filesystem.squashfs
upperdir=/run/live/overlay/rw  (same place where the images are copied)

So you think that because the images are part of the squashfs which is
read-only, they are first copied to /run/live/overlay/rw... to be written
to?

I always thought that tmpfs is mounted on top of squash, so it's as if the
squashfs were writable, without having to literally copy between the two
filesystems.

Also I made the same build with Virtualbox and there is no copying of the
images to /run/live/overlay.

I will do some serious research into squashfs and tmpfs rn.

On Wed, Nov 24, 2021, 5:10 PM Martin Kletzander <mkletzan at redhat.com> wrote:

> On Wed, Nov 24, 2021 at 04:01:34PM +0100, Elias Mobery wrote:
> >Hello Michal, thank you for the reply!
> >
> >I've carefully tested everything you suggested, thanks.
> >
> >I set dynamic_ownership=0 and use these hooks during the live build for
> >permissions. (I googled a lot, and apparently libvirt needs the images to
> >be executable too)
> >
> >chown -R libvirt-qemu:kvm  /var/lib/libvirt/images
> >chmod -R g+rwx /var/lib/libvirt/images
> >
>
> I do not see why would the images need to be executable, you might need
> an executable bit set on the directory so that your and qemu user can
> browse it.
>
> >Booting the live debian iso everything works in virt-manager, but again,
> >after clicking "run", a copy of the vm image is created in
> >/run/live/overlay/rw/var/lib/libvirt/images and only then does the VM
> start.
> >
>
> Does that happen when you try to run it with virsh instead of
> virt-manager?  Are you connected to the system daemon?
>
> >Either it's still being chowned or chmodded somehow, or it's something
> >else, but I can't stop this copy being made.
> >
> >Interestingly, when I boot the Live debian iso and then copy the images
> >into /var/lib/libvirt/images from my USB stick, the VM starts immediately
> >without creating any copies in the /run/live.... directory. So my guess is
> >that maybe the squashfs could be the issue?
> >
>
> Oh, interesting, is the USB stick filesystem mounted R/W and the live
> ISO filesystem mounted read-only?  How is the overlay mounted?
>
> >Editing the XML
> >
> ><source file='/var/lib/libvirt/images/vm1.qcow2'>
> >      <seclabel relabel='no'/>
> >    </source>
> >
> >This results in an error:
> >Unsupported Configuration:
> >Security driver model 'null' not available
> >
>
> For issues like this it is most helpful to check the documentation:
>
>    https://libvirt.org/formatdomain.html
>
> where you can see it is just a missing attribute `model`, in this case
> model="dac".
>
> >Here I tried setting security_driver=none in qemu.conf but same error
> after.
> >
> ></devices>
> >    <seclabel type='none'/>
> >  </domain>
> >
>
> Same here.
>
> >This also returns an Error but I'm still googling to understand it
> properly.
> >
> >XML document failed to validate against schema
> >Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
> >Invalid element relabel  for element seclabel
> >Extra element seclabel in interleave
> >Element domain failed to validate content
> >
> >Thanks again so much for your helo, I've been messing with this for weeks
> >now and it's killing me.
> >
> >On Tue, Nov 23, 2021, 9:43 PM Michal Prívozník <mprivozn at redhat.com>
> wrote:
> >
> >> On 11/23/21 17:25, Elias Mobery wrote:
> >> >
> >> > Hi everyone!
> >> >
> >> > I've built a Debian Live ISO with packages qemu and libvirt to run a
> VM
> >> > in the live environment.
> >> >
> >> > The guest images are placed in  /var/lib/libvirt/images and 2GB each.
> >> >
> >> > Everything works great, except for one issue.
> >> >
> >> > When starting a VM, libvirt automatically issues a chown command to
> the
> >> > images, changing ownership.
> >> >
> >> > This results in a copy of the images being created in
> >> > /run/live/overlay/rw/var/lib/libvirt/images
> >> >
> >> > I don't want these copies to be made but can't stop it.
> >> >
> >> > I've tried editing qemu.conf user/group, dynamic ownership etc.
> without
> >> > any luck.
> >> >
> >> > Is there a way to STOP libvirt from changing the ownership of these
> >> images?
> >> >
> >> >
> >>
> >> Setting dynamic_ownership=0 in qemu.conf is the way to go (don't forget
> >> to restart the daemon after you made the change).
> >>
> >> Alternatively, you can set <seclabel/> either for whole domain or
> >> individual disks, e.g. like this:
> >>
> >>   <disk type='file' device='disk'>
> >>     <driver name='qemu' type='qcow2'/>
> >>     <source file='/var/lib/libvirt/images/vm1.qcow2'>
> >>       <seclabel relabel='no'/>
> >>     </source>
> >>   </disk>
> >>
> >> or for whole domain:
> >>
> >>     ...
> >>     </devices>
> >>     <seclabel type='none'/>
> >>   </domain>
> >>
> >> I'm not sure what your setup is, but if chown() is a problem then what
> >> if guest tries to write onto its disk? Wouldn't that create a copy in
> >> overlay?
> >>
> >> Michal
> >>
> >>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvirt-users/attachments/20211124/88199e4b/attachment.htm>