[libvirt] Semantic change in v4.3 slows down image creation and snapshots (on ZFS)

[Daniel P. Berrangé]

On Wed, Oct 09, 2019 at 02:25:09PM +0200, Peter Krempa wrote:
> On Wed, Oct 09, 2019 at 13:39:54 +0200, Christian Ehrhardt wrote:
> > Hi,
> > when discussing bug [1] triggered by [2] Daniel and I have found that
> > the change [3] which is in since v4.3 was a semantic change to the
> > behavior of libvirt.
> > 
> > virt-manager used and still does issue a volume XML like:
> > <volume>
> >   <name>disk.qcow2</name>
> >   <capacity>16106127360</capacity>
> >   <allocation>16106127360</allocation>
> >   <target>
> >     <format type="qcow2"/>
> >     <features>
> >       <lazy_refcounts/>
> >     </features>
> >   </target>
> > </volume>
> > 
> > Up until the commit above that was using preallocation=metadata, but
> > now uses preallocation=falloc. In many cases (read: Fileystems) that
> > won't be very user-visible, but if you are running on ZFS (or anything
> > else that can't fallocate well) this is made worse by [4] slowing down
> > these operations a lot.
> 
> Note that for raw files we would do preallocation for the full image but
> only when fast allocation is supported. See 'createRawFile'.

Hmm, yes, i see we report a fatal error if fallocate() does work,
which is good.

Annoyingly QEMU uses  posix_fallocate() which automagically falls
back to writing zeros, which is very bad. AFAIK it isn't possible
to ask QEMU to allocate only if it is fast :-(

> > After the POV for virt-manager seems to have settled on the bug [1]
> > referenced the remaining question is if libvirt should revert/modify
> > the change [3] - and that we agreed required a wider discussion to be
> > had here, hence this mail.
> 
> Semantically the change above would make sense if it weren't coupled to
> the presence of VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA.
>
> The flags semantics imply that metadata allocation is requested and thus
> metadata should be allocated.
> 
> On the other hand doing full allocation if the flag is not present but
> the XML calls for it would semantically be okay IMO.

Yeah, I agree with this.

> I'm not sure if that is what you expect though.

virt-manager passes VOL_CREATE_PREALLOC_METADATA, so honouring that flag
correctly would fix the problem reported with ZFS.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|