[libvirt] [PATCH 1/6] Add new API virDomainStreamDisk[Info] to header and drivers

[Daniel P. Berrange]

On Mon, Apr 11, 2011 at 05:06:54PM -0500, Anthony Liguori wrote:
> On 04/11/2011 04:45 PM, Daniel P. Berrange wrote:
> >On Fri, Apr 08, 2011 at 02:26:48PM -0500, Anthony Liguori wrote:
> >>On 04/08/2011 11:02 AM, Stefan Hajnoczi wrote:
> >>>On Fri, Apr 8, 2011 at 2:31 PM, Daniel P. Berrange<berrange at redhat.com>   wrote:
> >>>
> >>>I have CCed Anthony and Kevin.  Anthony drove the QED image streaming
> >>>and Kevin will probably be interested in the idea of allocating raw
> >>>images as a background activity while QEMU runs.
> >>>
> >>>>    /*
> >>>>     * @path: fully qualified filename of the virtual disk
> >>>>     * @nregions: filled in the number of @region structs
> >>>>     * @regions: filled with a list of allocated regions
> >>>>     *
> >>>>     * Query the extents of allocated regions within the
> >>>>     * virtual disk file. The offsets in the list of regions
> >>>>     * are not guarenteed to be sorted in any explicit order.
> >>>>     */
> >>>>    int virDomainBlockGetAllocationMap(virDomainPtr dom,
> >>>>                                       const char *path,
> >>>>                                       unsigned int *nregions,
> >>>>                                       virDomainBlockRegionPtr *regions);
> >>>QEMU can provide this with its existing .bdrv_is_allocated() function.
> >>>  Kevin, do you have any thoughts on whether this API will work well?
> >>I think the trouble with this API proposal is that it's overloading
> >>concepts.
> >>
> >>Sparse is not the same thing as CoW to a backing file.
> >I don't like to use the term "sparse", since that implies a specific disk
> >format (raw file with holes). Rather I use the term 'thin provisioned'
> >to refer to any disk format, where the not all physical sectors have
> >yet been allocated. A thin-provisioned disk, can trivially be thought
> >of as a disk, with a backing file whose sectors are all filled with
> >zeros.
> 
> It's not so black and white today.
> 
> Imagine that you had a qcow2 file, and you "streamed" it such that
> it was no longer "thin provisioned", as soon as the guest starts
> issuing trim/discards, QEMU could conceivably start defragmenting
> the image and truncating resulting in a sparse file.
>
> The only time the concept of "fully allocated" really makes sense is
> for a raw image on a simple file system.   Once you start dealing
> with things like btrfs and deduplication, and of those useful
> guarantees are thrown out the window.

I would expect any behaviour where QEMU would defrag/truncate the
file to release host storage blocks to be configurable. It must be
possible for a mgmt app to ensure that a guest runs with fully
allocated storage at all times, to provide protection against
allocation failure due to overcommit.

> I think the real question is, why do you care about what physical
> sectors reside where?  What problem are you trying to solve?

Err, I don't care where the physical sectors reside. The API
is providing info about the logical allocation information.
The primary motivation is the image streaming use case, in
the sector-at-a-time mode, rather than single-shot entire
image. The other example is making an image fully allocated.
There may be other use cases, hence the proposal to provide
a general purpose API instead of something that only considers
the narrow use case of image streaming, which we then have to
later replace with something more general.

> >>For instance, when you expose streaming, the result is still a
> >>sparse file.  So you'd have a rather curious API where you called to
> >>"allocate" a region in the file which resulted in having a sparse
> >>file which you then called again to make it non sparse.  But AFAICT,
> >>the API doesn't really tell you these details.
> >Copy-on-read streaming does not imply that the result is still
> >thin-provisioned. That is a policy decision by the management
> >application.
> 
> I think your notion of thin-provision doesn't quite map to how
> things work today.  Unless you're in a very constrained environment,
> you're always thin provisioned.

I don't agree with that. Use of thin-provisioning is a policy
decision that an application can make, not a mandatory requirement

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|