[Libguestfs] [PATCH] v2v: -o rhv-upload: Optimize http request sending

[Nir Soffer]

On Thu, Jun 14, 2018 at 9:16 PM Nir Soffer <nirsof at gmail.com> wrote:

> When sending request with small or no payload, it is simpler and
> possibly more efficient to use the high level HTTPSConnection.request(),
> instead of the lower level APIs.
>
> The only reason to use the lower level APIs is to avoid copying the
> payload, or on python 2, to use a bigger buffer size when streaming a
> file-like object.
> ---
>  v2v/rhv-upload-plugin.py | 35 ++++++++++++++++-------------------
>  1 file changed, 16 insertions(+), 19 deletions(-)
>
> diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py
> index ed99cc7a9..3fad865f6 100644
> --- a/v2v/rhv-upload-plugin.py
> +++ b/v2v/rhv-upload-plugin.py
> @@ -263,12 +263,12 @@ def pread(h, count, offset):
>      transfer = h['transfer']
>      transfer_service = h['transfer_service']
>
> -    http.putrequest("GET", h['path'])
> +    headers = {"Range", "bytes=%d-%d" % (offset, offset+count-1)}
>      # Authorization is only needed for old imageio.
>      if h['needs_auth']:
> -        http.putheader("Authorization", transfer.signed_ticket)
> -    http.putheader("Range", "bytes=%d-%d" % (offset, offset+count-1))
> -    http.endheaders()
> +        headers["Authorization"] = transfer.signed_ticket
> +
> +    http.request("GET", h['path'], headers=headers)
>
>      r = http.getresponse()
>      # 206 = HTTP Partial Content.
> @@ -319,11 +319,10 @@ def zero(h, count, offset, may_trim):
>                        'size': count,
>                        'flush': False}).encode()
>
> -    http.putrequest("PATCH", h['path'])
> -    http.putheader("Content-Type", "application/json")
> -    http.putheader("Content-Length", len(buf))
> -    http.endheaders()
> -    http.send(buf)
> +    headers = {"Content-Type": "application/json",
> +               "Content-Length", str(len(buf))}
> +
> +    http.request("PATCH", h['path'], body=buf, headers=headers)
>
>      r = http.getresponse()
>      if r.status != 200:
> @@ -368,11 +367,10 @@ def trim(h, count, offset):
>                        'size': count,
>                        'flush': False}).encode()
>
> -    http.putrequest("PATCH", h['path'])
> -    http.putheader("Content-Type", "application/json")
> -    http.putheader("Content-Length", len(buf))
> -    http.endheaders()
> -    http.send(buf)
> +    headers = {"Content-Type": "application/json",
> +               "Content-Length", str(len(buf))}
> +
> +    http.request("PATCH", h['path'], body=buf, headers=headers)
>
>      r = http.getresponse()
>      if r.status != 200:
> @@ -387,11 +385,10 @@ def flush(h):
>      # Construct the JSON request for flushing.
>      buf = json.dumps({'op': "flush"}).encode()
>
> -    http.putrequest("PATCH", h['path'])
> -    http.putheader("Content-Type", "application/json")
> -    http.putheader("Content-Length", len(buf))
> -    http.endheaders()
> -    http.send(buf)
> +    headers = {"Content-Type": "application/json",
> +               "Content-Length", str(len(buf))}
> +
> +    http.request("PATCH", h['path'], body=buf, headers=headers)
>
>      r = http.getresponse()
>      if r.status != 200:
> --
> 2.17.1
>

WARNING: this is not tested, but I tested similar code here:
https://github.com/oVirt/ovirt-imageio/blob/master/examples/upload
(which need the same change).

Do we have an easy way to test the plugin without running the entire
virt-v2v pipeline?

Ideally, a way to run nbdkit with the plugin against an existing ovirt
system, and then use qemu-img to convert a file to the nbdkit nbd socket.

Nir
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libguestfs/attachments/20180614/3c176ffb/attachment.htm>