[Libguestfs] [PATCH v2] v2v: rhv-upload-plugin: Defer imageio connection

[Nir Soffer]

On Thu, Jan 21, 2021 at 2:08 PM Richard W.M. Jones <rjones at redhat.com> wrote:
>
> On Thu, Jan 21, 2021 at 01:56:55PM +0200, Nir Soffer wrote:
> > This happens only when not using unix socket, for example when running
> > on non-ovirt host, or ovirt host from another data center, or when using
> > -oo rhv_direct=false
> >
> > When using unix socket, we close the initial HTTPSConnection, and
> > created a new UnixHTTPConnection. This connection is not connected to
> > the server yet. When qemu-img tries to access the server, the connection
> > is connected automatically.
> >
> > Fix the issue by closing the initial connection used to get server
> > options and initialize the handle, and storing a closed connection in
> > the handle.
>
> I think this description could be a bit clearer.  If I'm understanding
> it correctly, this description could be better:
>
> "
>   This happens only when not using unix socket, for example when running
>   on non-ovirt host, or ovirt host from another data center, or when using
>   -oo rhv_direct=false
>
>   After this change, in the unix domain socket case:
>
>   - We close the socket.

The tcp socket

>   - We call optimize_http which immediately reopens the unix socket.

No, we create a connection object, but it does not create the underlying
socket until we send() is called:

HTTPConnection.__init__():
https://github.com/python/cpython/blob/34df10a9a16b38d54421eeeaf73ec89828563be7/Lib/http/client.py#L241

HTTPConnection.send():
https://github.com/python/cpython/blob/34df10a9a16b38d54421eeeaf73ec89828563be7/Lib/http/client.py#L972

>   In the non-unix domain socket case:
>
>   - We close the socket.
>
>   - We call optimize_http which does nothing.

Right

>   - Later when qemu-img starts to read or write to nbdkit,
>     HTTPConnection transparently reconnects to the oVirt server.
>     Doing it this way avoids keeping the TCP connection idle after the
>     initial open(), avoiding the bug described above.

Right. I'll try to make this more clear before pushing.

> "
>
> > --- a/v2v/rhv-upload-plugin.py
> > +++ b/v2v/rhv-upload-plugin.py
> > @@ -117,6 +117,15 @@ def open(readonly):
> >          destination_url = parse_transfer_url(transfer)
> >          http = create_http(destination_url)
> >          options = get_options(http, destination_url)
> > +
> > +        # Close the initial connection to imageio server. When qemu-img will
> > +        # try to access the server, HTTPConnection will reconnect
> > +        # automatically. If we keep this connection idle and qemu-img is too
> > +        # slow getting image extents, imageio server may close the connection,
> > +        # and the import will fail on the first write.
> > +        # See https://bugzilla.redhat.com/1916176.
> > +        http.close()
> > +
> >          http = optimize_http(http, host, options)
> >      except:
> >          cancel_transfer(connection, transfer)
>
> Anyway, please feel free to push it now if you want.
>
> Thanks,
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> Fedora Windows cross-compiler. Compile Windows programs, test, and
> build Windows installers. Over 100 libraries supported.
> http://fedoraproject.org/wiki/MinGW
>