[libvirt] [PATCH RFC 0/8] Sparse streams

Daniel P. Berrange berrange at redhat.com
Mon Feb 1 14:50:55 UTC 2016

On Fri, Jan 29, 2016 at 02:26:51PM +0100, Michal Privoznik wrote:
> This is merely an RFC.
> What's the problem?
> We have APIs for transferring disk images from/to host. Problem is, disk images
> can be sparse files. Our code is, however, unaware of that fact so if for
> instance the disk is one big hole of 8GB all those bytes have to: a) be read b)
> come through our event loop. This is obviously very inefficient way.
> How to deal with it?
> The way I propose (and this is actually what I like you to comment on) is to
> invent set of new API. We need to make read from and write to a stream
> sparseness aware. The new APIs are as follows:
>   int virStreamSendOffset(virStreamPtr stream,
>                           unsigned long long offset,
>                           const char *data,
>                           size_t nbytes);
>   int virStreamRecvOffset(virStreamPtr stream,
>                           unsigned long long *offset,
>                           char *data,
>                           size_t nbytes);
> The SendOffset() is fairly simple. It is given an offset to write @data from so
> it basically lseek() to @offset and write() data.
> The RecvOffset() has slightly complicated design - it has to be aware of the
> fact that @offset it is required to read from fall into a hole. If that's the
> case it sets @offset to new location where data starts.
> Are there other ways possible?
> Sure! If you have any specific in mind I am happy to discuss it. For instance
> one idea I've heard (from Martin) was instead of SendOffset() and RecvOffset()
> we may just invent our variant of lseek().
> What's left to be done?
> Basically, I still don't have RPC implementation. But before I dive into that I
> thought of sharing my approach with you - because it may turn out that a
> different approach is going to be needed and thus my work would render useless.

It would be intesting to see the virsh vol-upload/download  client code updated
to use the new APIs to deal with holes, so we can see how this API design looks
from the POV of apps using libvirt.

|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

More information about the libvir-list mailing list