[Libguestfs] nbdkit worker thread shutdown

[Richard W.M. Jones]

On Thu, May 05, 2022 at 08:59:56AM +0100, Nikolaus Rath wrote:
> Hello,
> 
> When nbdkit calls a plugin's unload() method, is it guaranteed that all
> pending requests have been handled (and all worker threads exited)?
> 
> (I would have expected this to be the case, but I'm getting errors from
> threads accessing data that my unload() handler frees, so I wanted to
> confirm my assumption).

.unload is literally called just before dlclose (see
server/backend.c:backend_unload) so if a thread in nbdkit was to
access something in your plugin after .unload then that would be quite
a serious bug in nbdkit.  It would be bound to crash because the code
and data has been unloaded from the process.  This implies too that
all connections would have been closed already.

Normally the server has no view into your plugin except through the
pointers in the plugin structure, and it shouldn't be using those
after .unload.  Can you give us some clue as to what exactly nbdkit is
accessing in your plugin?

The difference between .cleanup and .unload is the global ordering of
plugins/filters.  .cleanup is called first, when the plugin and
filters are still in memory.  .unload is then called on each filter &
the plugin (IIRC, plugin first), and so other things may have already
been unloaded from memory.  You almost certainly don't need to worry
about this difference -- almost everything can use .unload -- but it
matters in some corner cases (see commit ce9e6066354 for details).

If your plugin needs to create its own background threads, it should
create them in .after_fork (or .open if per-connection), and can clean
them up in .unload (or .close).  VDDK plugin is a mature example of
per-connection background threads.  Torrent plugin has a global
background thread that it (incorrectly) doesn't clean up.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org