[Libguestfs] Plans for nbdkit 1.18 release?

[Eric Blake]

On 2/22/20 5:42 AM, Richard W.M. Jones wrote:
> On Sat, Feb 22, 2020 at 05:11:01AM -0600, Eric Blake wrote:
>> On 2/22/20 4:37 AM, Richard W.M. Jones wrote:
>>> Another thing I've been thinking about for some time is splitting
>>> .config_complete into .config_complete + .get_ready (new name TBD).
>>> At the moment .config_complete is both the place where we finish
>>> processing config, and also the last chance we get to set things up
>>> before the server forks.  Of course such a change would not be
>>> material — .get_ready would always be run immediately after
>>> .config_complete — but it cleans up the API a little.
>>
>> How would it play with --dump-config?  Right now, we allow that to
>> run after .config but before .config_complete, precisely so that
>> language bindings can accept the first .config to determine their
>> script name, and then consult the script for any further config
>> help.
> 
> I don't believe it affects it at all.  Here are a couple of
> patches which show the general idea and pass all the tests.

One thought I've had (especially with the ext2 filter fresh on my mind) 
- it would be nice to be able to create a filter that shares a common 
open of the plugin.  For example, in ext2, we cannot support parallel 
calls to ext2fs_open() (since that is basically mounting the file 
system, but you can't have parallel mounters), as a result, 
.thread_model returns SERIALIZE_CONNECTIONS (a second client has to wait 
for the first to unmount).

What I'd really like is for ext2's .config_complete to validate that all 
parameters are sane, then have .get_ready be passed both a next_open and 
next_ops pointer so it can call a single next_ops->open() to get a 
handle to the underlying plugin, mount the file system once; then ext2's 
.open would NOT call through its next_open() parameter (because it 
already has a shared open handle), and we could have .prepare do JUST 
the parallelizable work of ext2fs_namei() per client but reusing the 
already-shared plugin handle from .get_ready.  That is, multiple clients 
can open different files in parallel based on exportname, or even open 
different handles to the same file, but ALL of the handles play nicely 
because they still occur within the context of a SINGLE mount point and 
plugin handle.

In fact, if we do something like that, we may even be able to write a 
filter that ADDS parallelism on top of an existing serialized plugin 
(because to the plugin's point of view, there is only a single client, 
it is the filter that is multiplexing multiple actual clients into 
serialized plugin access).

To some extent, I already experimented with multiplexed access in the 
'nbdkit nbd shared=1' plugin, where that code opens a single libnbd 
connection during .config_complete reused by all the .open (but doing it 
in a plugin was different then doing it in a filter, since the plugin is 
not relying on .next_ops).  And what I did in the nbd plugin is an 
example of code that nicely fits the split between real .config_complete 
argument validation and .get_ready libnbd connection.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org