[libvirt] Semantic change in v4.3 slows down image creation and snapshots (on ZFS)

[Mike Haboustak]

On Wed, Oct 9, 2019 at 8:33 AM Daniel P. Berrangé <berrange at redhat.com> wrote:
> On Wed, Oct 09, 2019 at 02:25:09PM +0200, Peter Krempa wrote:
> > On Wed, Oct 09, 2019 at 13:39:54 +0200, Christian Ehrhardt wrote:
> > > Up until the commit above that was using preallocation=metadata, but
> > > now uses preallocation=falloc.
>
> > Semantically the change above would make sense if it weren't coupled to
> > the presence of VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA.
> >
> > The flags semantics imply that metadata allocation is requested and thus
> > metadata should be allocated.
> >
> > On the other hand doing full allocation if the flag is not present but
> > the XML calls for it would semantically be okay IMO.
>
> Yeah, I agree with this.
>
> > I'm not sure if that is what you expect though.
>
> virt-manager passes VOL_CREATE_PREALLOC_METADATA, so honouring that flag
> correctly would fix the problem reported with ZFS.

I know that this is an old message, but I'm just getting around to updating
some of my libvirt-python scripts and I'm encountering a similar problem to
virt-manager when using libvirt >=4.3, without the ZFS part.

Prior to libvirt v4.3, my script created qcow2-format StorageVolumes with a
<capacity/> element, and no <allocation/> element. It used the flag
VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA to request metadata-only
prealloc, which the documentation of virStorageVolCreateXml describes
as creating
a sparse file:

> Since 1.0.1 VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA in flags
> can be used to get higher performance with qcow2 image files which don't support
> full preallocation, by creating a sparse image file with metadata.

The patch identified in this thread does not create a sparse file even in the
presence of this flag when the requested capacity is equal to the requested
allocation.

You could just update the documentation to match current behavior, but I
think that this is a bug and that qemu-img's prealloc=metadata option should
always be used when VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA
is specified.

It also seems that the documentation is out-of-date in regards to qcow2 not
supporting full preallocation.

As mentioned previously, using the StorageVolume's <allocation /> element as
an input does not work for qcow2. The requested allocation size is never
honored by qemu-img. There is no concept of partial preallocation. There is only
full prealloc and sparse metadata prealloc.

Thanks,
Mike