[Libguestfs] [PATCH 0/2] rhv-upload: Complete refactoring

Tue Nov 19 23:22:35 UTC 2019

On Wed, Nov 20, 2019 at 1:19 AM Nir Soffer <nirsof at gmail.com> wrote:
>
> Fix NameError introduced by inocomplete change to extract create_transfer() by
> extracting cancel_transfer() function.
>
> Finally extract finallize_transfer() function, dealing with the poorly
> documented and over complicated oVirt SDK.
>
> Tested with:
> libguestfs-1.40.2-4.fc30.x86_64
> nbdkit-1.12.8-1.fc30.x86_64
>
> Tested flows:
> - Successful import
> - Error in close() - by injecting long sleep in vdsm verifaction step
> - Error in close() - by injecting error in vdsm verification step
> - Error in open() - by injecting error in imageio OPTIONS handler
> - Error in zero() - by injecting error in imageio PATCH handler
> - Error in flush() - by injecting error in imageio PATCH handler

Here is output from the tests runs:

## Simulate verification error by injecting fake error in vdsm.
...
nbdkit: python[1]: debug: client sent NBD_CMD_DISC, closing connection
nbdkit: python[1]: debug: close
finalizing transfer 01aecf9a-2629-40e4-8528-f310a4ad6c82
virtual copying rate: 3368.6 M bits/sec
canceling transfer 01aecf9a-2629-40e4-8528-f310a4ad6c82
nbdkit: python[1]: error:
/home/nsoffer/src/virt-v2v/tmp/rhvupload.SZBNp8/rhv-upload-plugin.py:
close: error: ['Traceback (most recent call last):\n', '  File
"/home/nsoffer/src/virt-v2v/tmp/rhvupload.SZBNp8/rhv-upload-plugin.py",
line 332, in close\n    finalize_transfer(connection, transfer,
disk_id)\n', '  File
"/home/nsoffer/src/virt-v2v/tmp/rhvupload.SZBNp8/rhv-upload-plugin.py",
line 561, in finalize_transfer\n    "transfer %s failed: disk is
ILLEGAL" % transfer.id)\n', 'RuntimeError: transfer
01aecf9a-2629-40e4-8528-f310a4ad6c82 failed: disk is ILLEGAL\n']

## Simulate inaccessible storage by adding long sleep in vdsm.
...
nbdkit: python[1]: debug: client sent NBD_CMD_DISC, closing connection
nbdkit: python[1]: debug: close
finalizing transfer c05c578e-df99-4fb5-80f0-6944051dbee0
virtual copying rate: 2281.9 M bits/sec
canceling transfer c05c578e-df99-4fb5-80f0-6944051dbee0
nbdkit: python[1]: error:
/home/nsoffer/src/virt-v2v/tmp/rhvupload.YKpW3U/rhv-upload-plugin.py:
close: error: ['Traceback (most recent call last):\n', '  File
"/home/nsoffer/src/virt-v2v/tmp/rhvupload.YKpW3U/rhv-upload-plugin.py",
line 332, in close\n    finalize_transfer(connection, transfer,
disk_id)\n', '  File
"/home/nsoffer/src/virt-v2v/tmp/rhvupload.YKpW3U/rhv-upload-plugin.py",
line 561, in finalize_transfer\n    "transfer %s failed: disk is
ILLEGAL" % transfer.id)\n', 'RuntimeError: transfer
c05c578e-df99-4fb5-80f0-6944051dbee0 failed: disk is ILLEGAL\n']

## Simulate error after transfer was started by returning invalid json
in imageio OPTIONS handler
...
disk.id = 'a2bad783-0736-42a5-92d1-d76d133453bf'
transfer.id = 'fd297a0d-1c9b-49ac-9104-829358127f23'
canceling transfer fd297a0d-1c9b-49ac-9104-829358127f23
nbdkit: python[1]: error:
/home/nsoffer/src/virt-v2v/tmp/rhvupload.nejhOD/rhv-upload-plugin.py:
open: error: ['Traceback (most recent call last):\n', '  File
"/home/nsoffer/src/virt-v2v/tmp/rhvupload.nejhOD/rhv-upload-plugin.py",
line 94, in open\n    options = get_options(http, destination_url)\n',
'  File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.nejhOD/rhv-upload-plugin.py",
line 618, in get_options\n    j = json.loads(data)\n', '  File
"/usr/lib64/python3.7/json/__init__.py", line 348, in loads\n
return _default_decoder.decode(s)\n', '  File
"/usr/lib64/python3.7/json/decoder.py", line 337, in decode\n    obj,
end = self.raw_decode(s, idx=_w(s, 0).end())\n', '  File
"/usr/lib64/python3.7/json/decoder.py", line 355, in raw_decode\n
raise JSONDecodeError("Expecting value", s, err.value) from None\n',
'json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char
0)\n']
qemu-img: Could not open 'json:{ "file.driver": "nbd", "file.path":
"/home/nsoffer/src/virt-v2v/tmp/rhvupload.nejhOD/nbdkit0.sock",
"file.export": "/" }': Failed to read option reply: Unexpected
end-of-file before all bytes were read

## Simulate error in zero by injecting error in imageio PATCH/zero
...
nbdkit: python[1]: debug: zero count=458752 offset=6441926656 may_trim=1 fua=0
unexpected response from imageio server:
could not zero sector offset 6441926656 size 458752
500: Internal Server Error
b'Server failed to perform the request, check logs'
nbdkit: python[1]: error:
/home/nsoffer/src/virt-v2v/tmp/rhvupload.3rv9Nk/rhv-upload-plugin.py:
zero: error: ['Traceback (most recent call last):\n', '  File
"/home/nsoffer/src/virt-v2v/tmp/rhvupload.3rv9Nk/rhv-upload-plugin.py",
line 229, in zero\n    (offset, count))\n', '  File
"/home/nsoffer/src/virt-v2v/tmp/rhvupload.3rv9Nk/rhv-upload-plugin.py",
line 149, in request_failed\n    raise RuntimeError("%s: %d %s: %r" %
(msg, status, reason, body[:200]))\n', "RuntimeError: could not zero
sector offset 6441926656 size 458752: 500 Internal Server Error:
b'Server failed to perform the request, check logs'\n"]
nbdkit: python[1]: debug: sending error reply: Input/output error
nbdkit: python[1]: debug: pwrite count=458752 offset=6441926656 fua=0
nbdkit: python[1]: debug: pwrite count=65536 offset=6442385408 fua=0
    (100.00/100%)
nbdkit: python[1]: debug: flush
nbdkit: python[1]: debug: flush
nbdkit: python[1]: debug: client sent NBD_CMD_DISC, closing connection
nbdkit: python[1]: debug: close
canceling transfer 86af4ae7-e1dc-4514-ad95-d7587678f946
virtual copying rate: 1213.8 M bits/sec

## Simulate error in flush by injecting error in imageio PATCH/flush
...
nbdkit: python[1]: debug: pwrite count=65536 offset=6442385408 fua=0
    (100.00/100%)
nbdkit: python[1]: debug: flush
unexpected response from imageio server:
could not flush
500: Internal Server Error
b'Server failed to perform the request, check logs'
nbdkit: python[1]: error:
/home/nsoffer/src/virt-v2v/tmp/rhvupload.vPoylv/rhv-upload-plugin.py:
flush: error: ['Traceback (most recent call last):\n', '  File
"/home/nsoffer/src/virt-v2v/tmp/rhvupload.vPoylv/rhv-upload-plugin.py",
line 302, in flush\n    request_failed(h, r, "could not flush")\n', '
File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.vPoylv/rhv-upload-plugin.py",
line 149, in request_failed\n    raise RuntimeError("%s: %d %s: %r" %
(msg, status, reason, body[:200]))\n', "RuntimeError: could not flush:
500 Internal Server Error: b'Server failed to perform the request,
check logs'\n"]
nbdkit: python[1]: debug: sending error reply: Input/output error
nbdkit: python[1]: debug: flush
unexpected response from imageio server:
could not flush
500: Internal Server Error
b'Server failed to perform the request, check logs'
nbdkit: python[1]: error:
/home/nsoffer/src/virt-v2v/tmp/rhvupload.vPoylv/rhv-upload-plugin.py:
flush: error: ['Traceback (most recent call last):\n', '  File
"/home/nsoffer/src/virt-v2v/tmp/rhvupload.vPoylv/rhv-upload-plugin.py",
line 302, in flush\n    request_failed(h, r, "could not flush")\n', '
File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.vPoylv/rhv-upload-plugin.py",
line 149, in request_failed\n    raise RuntimeError("%s: %d %s: %r" %
(msg, status, reason, body[:200]))\n', "RuntimeError: could not flush:
500 Internal Server Error: b'Server failed to perform the request,
check logs'\n"]
nbdkit: python[1]: debug: sending error reply: Input/output error
nbdkit: python[1]: debug: client sent NBD_CMD_DISC, closing connection
nbdkit: python[1]virtual copying rate: 2227.0 M bits/sec
: debug: close
canceling transfer 6508e39c-6956-4333-8f27-1f2b55377986
virt-v2v: error: transfer of disk 1/1 failed, see earlier error messages

Nir