[Libguestfs] [PATCH 0/3] Cleanups for ovirt 4.4

[Nir Soffer]

On Fri, Jan 22, 2021 at 10:49 AM Richard W.M. Jones <rjones at redhat.com>
wrote:

> By the way if you happen to be looking at the Python plugin, then
> there are two changes that may be worth using.
>
> API_VERSION = 2:
>
>   https://libguestfs.org/nbdkit-python-plugin.3.html#API-versions
>
> and multi-thread support:
>
>   https://libguestfs.org/nbdkit-python-plugin.3.html#Threads
>
> These were both added (by you) and are both available in nbdkit >= 1.22,
> which is now available in RHEL AV >= 8.3.0 so there's no reason why we
> can't use them now.
>

Great

I don't think there's any advantage to API_VERSION = 2 because the
> plugin is very (exclusively?) write-heavy.  But a more parallel thread
> model is surely advantageous?
>

I think we can change the rhv upload plugin to use this approach:
plugins/python/examples/imageio.py

Based on the tests I did with the imageio plugin this should
give significant performance improvement.
https://github.com/libguestfs/nbdkit/commit/78bbe95f754dd404fcbabac57940b1c0e4e49c4d

I think the changes are basically:

api version: 2
thread model: parallel

open:
- Create a connection pool instead of a single connection.
- Based on experience wit imageio client, 4 connections are a good default

pwrite/zero/pread:
- Get connection from the pool, blocking for available instance
- Return the connection to the pool when done or after failure

flush:
- send a flush request for all connections in the pool
   (each http connection have a separate nbd connection inside imageio)

close:
- close all connections in pool

What do you think?

For API_VERSION = 2, the changes are:
>
> * pread signature changes to:
>
>   def pread(h, buf, offset, flags) -> returns nothing
>
>   - You have to write the result directly into the buf parameter,
>     avoiding the extra copy.
>
> * pwrite adds flags parameter:
>
>   def pwrite(h, buf, offset, flags) -> returns nothing
>
> * flush, trim also add flags parameter.
>
> * zero changes signature completely, removing the may_trim parameter
>   and adding a general flags parameter in its place.
>
> For threads:
>
> * Add a thread_model() function with no parameters which returns
>   nbdkit.THREAD_MODEL_PARALLEL or nbdkit.THREAD_MODEL_SERIALIZE_REQUESTS.
>   Of course locking will need to be introduced (especially for
>   PARALLEL) to protect internal data structures.  See nbdkit-plugin(3)
>   for an explanation of the thread models.
>
> You'll also have to change the minimum version of nbdkit so that older
> versions get rejected, here:
>
>
> https://github.com/libguestfs/virt-v2v/blob/1d5fc257765c444644e5bfc6525e86ff201755f0/v2v/output_rhv_upload.ml#L87
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat
> http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> libguestfs lets you edit virtual machines.  Supports shell scripting,
> bindings from many languages.  http://libguestfs.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libguestfs/attachments/20210122/d4f5d4b4/attachment.htm>