<div dir="ltr"><div dir="ltr">On Tue, Nov 19, 2019 at 3:36 PM Martin Kletzander <<a href="mailto:mkletzan@redhat.com">mkletzan@redhat.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Tue, Nov 19, 2019 at 01:48:31PM +0100, Martin Kletzander wrote:<br>
>On Tue, Nov 19, 2019 at 02:24:11PM +0200, Nir Soffer wrote:<br>
>>On Tue, Nov 19, 2019 at 2:16 PM Martin Kletzander <<a href="mailto:mkletzan@redhat.com" target="_blank">mkletzan@redhat.com</a>> wrote:<br>
>>> On Tue, Nov 19, 2019 at 01:14:23PM +0200, Nir Soffer wrote:<br>
>>> >On Tue, Nov 19, 2019 at 11:17 AM Martin Kletzander <<a href="mailto:mkletzan@redhat.com" target="_blank">mkletzan@redhat.com</a>> wrote:<br>
>>> >><br>
>>> >> Traceback (most recent call last):<br>
>>> >> File "/var/tmp/rhvupload.Vw0CIU/rhv-upload-plugin.py", line 97, in<br>
>>> open<br>
>>> >> transfer_service.cancel()<br>
>>> >> NameError: name 'transfer_service' is not defined<br>
>>> ><br>
>>> >Does this fix the error for you?<br>
>>> ><br>
>>> <a href="https://github.com/nirs/virt-v2v/commit/2f93dbffad81a26445831293ecac213eadffbd57" rel="noreferrer" target="_blank">https://github.com/nirs/virt-v2v/commit/2f93dbffad81a26445831293ecac213eadffbd57</a><br>
>>> ><br>
>>> >I did not test it yet.<br>
>>> ><br>
>>><br>
>>> If I remove the stray `def finalize_transfer(...)` it works, yes.<br>
>>><br>
>><br>
>>Right, belongs to the next patch.<br>
>><br>
>><br>
>>> In the meantime I managed to fix the original issue (which I needed to<br>
>>> revert to<br>
>>> test this patch =) ), thankfully it was just caused by my oVirt host setup.<br>
>>><br>
>>> Unfortunately I am getting another issue. Now nbdkit fails after all the<br>
>>> data<br>
>>> is written and it is trying to write the disk id file. I'm getting "No<br>
>>> such<br>
>>> file or directory", so because it is writing to that file I'm assuming the<br>
>>> directory does not exist. On the other hand it *must* exists because it's<br>
>>> the<br>
>>> same one where the python scripts are saved. So I need to debug more.<br>
>>><br>
>><br>
>>This works for me, are you running a modified plugin?<br>
><br>
>No, it is a current master of virt-v2v. But don't worry about it now. It is<br>
>most probably something with my setup again. Even though it worked before and<br>
>these issues started showing up after an update to virt-v2v. I'll try with<br>
>current master of nbdkit, libguestfs and virt-v2v first and if that fails I'll<br>
>try different version of virt-v2v and then start debugging it from the script<br>
>itself. It is a bit tedious as I need to modify the script in-tree, upload it<br>
>to the server, build it there, install it, then run virt-v2v-wrapper (that's how<br>
>I'm testing it now) and then inspect the logs on that remote machine.<br>
<br>
Actually looking at the whole log what I'm getting is:<br>
<br>
nbdkit: python[1]: error: /var/tmp/rhvupload.jngN1W/rhv-upload-plugin.py: flush: error: ['Traceback (most recent call last):\n', ' File "/var/tmp/rhvupload.jngN1W/rhv-upload-plugin.py", line 298, in flush\n http.request("PATCH", h[\'path\'], body=buf, headers=headers)\n', ' File "/usr/lib64/python3.7/http/client.py", line 1252, in request\n self._send_request(method, url, body, headers, encode_chunked)\n', ' File "/usr/lib64/python3.7/http/client.py", line 1263, in _send_request\n self.putrequest(method, url, **skips)\n', ' File "/usr/lib64/python3.7/http/client.py", line 1108, in putrequest\n raise CannotSendRequest(self.__state)\n', 'http.client.CannotSendRequest: Request-sent\n']<br></blockquote><div><br></div><div>Looks like you cannot communicate with imageio daemon - this is fatal error that should fail the</div><div>upload.</div><div>... </div><div>and then after couple of lines, when it is actually closing I get:</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
...<br>
<br>
finalized after 8.060538530349731 seconds<br></blockquote><div><br></div><div>But we tried to finallize the transfer instead of cancel it.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
nbdkit: python[1]: error: /var/tmp/rhvupload.jngN1W/rhv-upload-plugin.py: close: error: ['Traceback (most recent call last):\n', ' File "/var/tmp/rhvupload.jngN1W/rhv-upload-plugin.py", line 362, in close\n', "FileNotFoundError: [Errno 2] No such file or directory: '/var/tmp/rhvupload.jngN1W/diskid.0'\n"]<br>
nbdkit: debug: python: unload plugin<br>
<br>
So it might be because virt-v2v already removed that directory and did not wait<br>
for nbdkit to completely end. I'm testing with older commit of virt-v2v now.<br></blockquote><div><br></div><div>flush() failure probably caused qemu-img to fai, and virt-v2v failed and cleaned up</div><div>which is expected.</div><div><br></div><div>I think the issue is incorrect error handling in the flush() and other handlers:</div><div><br></div><div>288 def flush(h):<br>289 http = h['http']<br>290 <br>291 # Construct the JSON request for flushing.<br>292 buf = json.dumps({'op': "flush"}).encode()<br>293 <br>294 headers = {"Content-Type": "application/json",<br>295 "Content-Length": str(len(buf))}<br>296 <br>297 http.request("PATCH", h['path'], body=buf, headers=headers)</div><div><br></div><div>If this raised (like in your traceback, we don't set h['failed'] = True</div><div><br>299 r = http.getresponse()<br>300 if r.status != 200:<br>301 request_failed(h, r, "could not flush")<br>302 <br>303 r.read()<br></div><div><br></div><div>Same here.</div><div><br></div><div>Same issue in all handlers. We should handle all errors in the handlers.</div><div><br></div><div>But the root cause is something else - why you cannot communicate with imageio daemon?</div></div></div>