[libvirt] [PATCH 0/2] Two simple sparse streams fixes

[Martin Kletzander]

On Wed, May 31, 2017 at 03:08:16PM +0200, Pino Toscano wrote:
>On Wednesday, 31 May 2017 13:03:38 CEST Martin Kletzander wrote:
>>  - vol-download --sparse --offset $source_file_size --length 1
>>    /path/to/source.file destination.file
>>
>>     - Every now and then (not always) it gets stuck waiting for the
>>       daemon to receive data (see backtrace below), but the daemon is not
>>       waiting for anything, it's just some weird race.  We can try
>>       debugging it with wireshark later.  That file ends with a hole.
>>
>> Thread 1 (Thread 0x7f1d2b434880 (LWP 28584)):
>> #0  0x00007f1d2796efbd in poll () at ../sysdeps/unix/syscall-template.S:84
>> #1  0x00007f1d2a806ee3 in poll (__timeout=5000, __nfds=2, __fds=0x7ffe9effd640) at /usr/include/bits/poll2.h:46
>> #2  virNetClientIOEventLoop (client=client at entry=0x563525bb06d0, thiscall=thiscall at entry=0x563525badc00) at rpc/virnetclient.c:1664
>> #3  0x00007f1d2a8074d3 in virNetClientIO (client=client at entry=0x563525bb06d0, thiscall=0x563525badc00) at rpc/virnetclient.c:1957
>> #4  0x00007f1d2a80780e in virNetClientSendInternal (client=client at entry=0x563525bb06d0, msg=msg at entry=0x563525bb03d0, expectReply=expectReply at entry=true, nonBlock=nonBlock at entry=false) at rpc/virnetclient.c:2132
>> #5  0x00007f1d2a808dfc in virNetClientSendWithReplyStream (client=client at entry=0x563525bb06d0, msg=msg at entry=0x563525bb03d0, st=st at entry=0x563525bade10) at rpc/virnetclient.c:2236
>> #6  0x00007f1d2a80ab2d in virNetClientStreamRecvPacket (st=st at entry=0x563525bade10, client=0x563525bb06d0, data=data at entry=0x7f1d20686010 "", nbytes=nbytes at entry=262120, nonblock=false, flags=32766, flags at entry=1) at rpc/virnetclientstream.c:499
>> #7  0x00007f1d2a7e0e3e in remoteStreamRecvFlags (st=0x563525badc60, data=0x7f1d20686010 "", nbytes=262120, flags=1) at remote/remote_driver.c:5664
>> #8  0x00007f1d2a7c8347 in virStreamRecvFlags (stream=stream at entry=0x563525badc60, data=0x7f1d20686010 "", nbytes=nbytes at entry=262120, flags=flags at entry=1) at libvirt-stream.c:361
>> #9  0x00007f1d2a7c9b7f in virStreamSparseRecvAll (stream=stream at entry=0x563525badc60, handler=0x563525760196 <virshStreamSink>, holeHandler=0x56352576020b <virshStreamSkip>, opaque=opaque at entry=0x7ffe9effd954) at libvirt-stream.c:964
>> #10 0x000056352576232e in cmdVolDownload (ctl=0x7ffe9effda40, cmd=<optimized out>) at virsh-volume.c:834
>> #11 0x00005635257662f1 in vshCommandRun (ctl=0x7ffe9effda40, cmd=0x563525bacf40) at vsh.c:1327
>> #12 0x000056352572aee2 in main (argc=9, argv=<optimized out>) at virsh.c:929
>>
>>      Trying to reproduce yet another one, the command gets stuck even with
>>      different offsets.
>>
>>  - vol-download --sparse --offset $X --length 1
>>    /path/to/source.file destination.file
>>
>>     - This does not respect the length if:
>>         X > $source_file_size - $last_hole_size
>>
>>       The size ends up being $source_file_size - $X
>
>Humble suggestion here: what about turning the simple scenarios above
>as proper tests?
>

The problem here is that after designing the test and writing it, we
also have to mock all accesses to the source and destination files and
report how the result looks, etc.  And I didn't get to virStreams even,
that's only sparse files.  We could instead do integration testing of
this, which would be easier, however you can only do that on a
filesystem that you know keeps holes, plus the hole sizes can be
different based on the block size, the files can be way different based
on adaptive allocations, etc.  There are so many factors for this that
it is not easy (I'm not saying it's impossible).  If I had lot of free
time, this could fit in somehow.  Also after I upgrade the
virfilewrapper, it will be easier to control the behaviour of the
file-access functions way more delicately.

But patches are welcome! ;)

>--
>Pino Toscano



>--
>libvir-list mailing list
>libvir-list at redhat.com
>https://www.redhat.com/mailman/listinfo/libvir-list
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Digital signature
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20170601/6591d098/attachment-0001.sig>