[Libguestfs] [PATCH nbdkit UNFINISHED] Add the ability to write plugins in golang.

[Richard W.M. Jones]

Sorry Dan, but I really do dislike golang with a passion :-)

Here is a patch that allows you to write nbdkit plugins in golang.  As
with C, OCaml and Rust, you can write a plugin in Go which compiles
directly to a .so file that can be loaded into golang, so in that
sense it works completely differently from scripting language plugins
like Perl and Python where there's an nbdkit-<lang>-plugin that
intermediates between nbdkit and the user’s code.

With that said, there are many problems.  The root cause of most of
them is that you cannot pass Go pointers to C, perhaps because the
golang developers never heard of registering extra GC roots[1], even
though that is common in non-Blub languages like OCaml and Haskell.
This leads to awkward complications nicely summarised in this page:

https://eli.thegreenplace.net/2019/passing-callbacks-and-pointers-to-cgo/

This requires that every callback has a wrapper across 3 files.  Oh
and these wrappers cannot be in a library module, they must be copied
into the plugin source code, AND they must be commented out by the
end-user by hand if the callback is not called.  (There may be a way
around the latter issues, but I could not work out how.)  This means
plugins have tons of duplicated source code.

Other issues that are not related to that one:

 - We have to link the nbdkit module with
   -Wl,--unresolved-symbols=ignore-in-object-files because of the way
   nbdkit plugins have deliberately unresolved symbols.  See the
   libnbdkit.so proposal on the mailing list for another way to solve
   this.

 - Be nice if ./configure could check that golang >= 1.5 since that
   was the first version that introduced shared libraries.

 - Since initialization is not synchronous in golang, you cannot rely
   on anything being initialized in the plugin before nbdkit starts
   calling in.  For this reason I worked around it by having
   plugin_init() call a start up function (func init_plugin()) where
   all golang initialization must be done.  Otherwise:

      nbdkit:                          golang plugin:

      dlopen ("plugin.so");
                                       let's start initializing
      init = dlsym ("plugin_init");
                                       return &plugin (uninitialized!)
      plugin = init ();
      plugin.load ()
                                       hey, I'm still initializing!

 - Related to the previous point: Be nice to move plugin_init() into
   the nbdkit module.  However I don't believe this is possible
   because this function has to call into the plugin (in main module).

 - Tests are a joke at the moment.  We would really need a test which
   properly exercises threads / parallel client connections, so we can
   be sure that the nbdkit thread & golang goroutine models do not
   conflict in some way.  (I don't think they do, but need to check).

 - Current test func pluginPRead() needs to be completed.

 - Documentation needs fixing.  I didn't want to write too many docs
   until I know finally how plugins would work.

Rich.

[1] Reading the proposal here confirms my suspicions, since there is a
much simpler, better and more obvious solution than what is proposed:
https://github.com/golang/proposal/blob/master/design/12416-cgo-pointers.md