[Libguestfs] [PATCH 0/3] Cleanups for ovirt 4.4

[Richard W.M. Jones]

On Fri, Jan 22, 2021 at 06:15:38PM +0200, Nir Soffer wrote:
> On Fri, Jan 22, 2021 at 5:42 PM Richard W.M. Jones <rjones at redhat.com> wrote:
> 
>     On Fri, Jan 22, 2021 at 05:11:46PM +0200, Nir Soffer wrote:
>     > Based on the tests I did with the imageio plugin this should
>     > give significant performance improvement.
>     > https://github.com/libguestfs/nbdkit/commit/
>     > 78bbe95f754dd404fcbabac57940b1c0e4e49c4d
>     >
>     > I think the changes are basically:
>     >
>     > api version: 2
>     > thread model: parallel
>     >
>     > open:
>     > - Create a connection pool instead of a single connection. 
>     > - Based on experience wit imageio client, 4 connections are a good
>     default
>     >
>     > pwrite/zero/pread:
>     > - Get connection from the pool, blocking for available instance
>     > - Return the connection to the pool when done or after failure
>     >
>     > flush:
>     > - send a flush request for all connections in the pool
>     >    (each http connection have a separate nbd connection inside imageio)
>     >
>     > close:
>     > - close all connections in pool
>     >
>     > What do you think?
> 
>     Yes.
> 
>     The tricky thing will be getting locking right.  With the PARALLEL
>     model you can have multiple parallel calls into the plugin even with
>     the same handle.  The Python GIL is still active, so Python code
>     fragments won't be running at the same time, but it's still possible
>     for one thread to block in the middle of a plugin method (eg.
>     HTTPConnection blocking on a read), releasing the GIL, allowing
>     another plugin method call to start with the same handle.
> 
> 
> The handle is a dict, which is thread safe, and we never modify 
> the handle during a transfer.
> 
> The only change we do is set h["failed"] to True, which is thread
> safe and idempotent.
> 
> To get a connection from the handle, we will use a queue which is
> thread safe. Connection are removed from the queue during use,
> so only one thread can access a connection. Returning connection
> to the queue is also thread safe.
> 
> So I think no extra locking is needed.

Sounds good!

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW