[Libguestfs] Support for NBD block status in nbdkit & qemu NBD client

[Richard W.M. Jones]

Hi Eric:

Here's the problem: We have a VMware VDDK file or snapshot.  The
virtual size of the snapshot is generally much larger than what's
allocated.  Copying the snapshot using:

  nbdkit vddk file=foo.vmdk --run 'qemu-img convert $nbd out'

is therefore much slower than it could be.  (In the real world version
of this we're actually copying the source over the network, but the
command line for that is too complicated to show in a simple example).

VDDK has an API for querying allocated blocks, see
‘VixDiskLib_QueryAllocatedBlocks’ in
https://vdc-download.vmware.com/vmwb-repository/dcr-public/8f96698a-0e7b-4d67-bb6c-d18a1d101540/ef536a47-27cd-481a-90ef-76b38e75353c/vsphere-vddk-671-programming-guide.pdf

It's awkward to use but Martin (CC'd) has played with it and it does
seem to work.

The question is if we add Structured Replies + Block Status support to
nbdkit & implement it for the VDDK plugin, will ‘qemu-img convert’
above be able to use that to optimize the copy?  My brief reading of
the qemu sources suggests that something has been implemented here so
it likely ought to work?

I guess the other question is if there are any limitations around
alignment that we should be aware of.  It seems as if VDDK uses a
larger cluster size for sparseness than its sector size (as does
qcow2).

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top