[Libguestfs] Extending nbdkit to support listing exports

[Richard W.M. Jones]

Hi Eric, Nir.

$SUBJECT - it's complicated!  Imagine a scenario where we have
extended the file plugin so you can point it at a directory and it
will use the client-supplied export name to select a file to serve.
Also we have extended the tar filter so that you can map tar file
components into export names.  You may be able to see where this is
going ...

Now we point the file plugin at a directory of tar files, and layer
the tar filter on top, and then what happens?  The client can only
supply a single export name, so how can it specify component X of tar
file Y?

  tar filter       --->   file plugin
  tar-exportname=on       exportname=on
                          directory=/my-files ---> directory of tar files

It gets worse!  At the moment filters and plugins can read the export
name sent by the client by calling the global function
nbdkit_export_name().  I'm not so worried about filters, but plugins
must be able to continue to use this global function.  In the scenario
above both the tar filter and the file plugin would see the same
string (if we did nothing), and it would only make sense to one or the
other but never to both.

Listing exports is also complicated: Should we list only the nearest
filter's exports?  Or the plugin's exports?  Cartesian product of both
somehow?


That's the background, but how do we solve it?
----------------------------------------------

I'd like to say first of all that I believe export names are already a
dumpster fire of potential insecurity, and that we shouldn't have the
core server attempt to parse them, so solutions involving splitting
the export names into components (like tar-component/file-name) should
not be considered because they are too risky, and run into arbitrary
limits (what escape character to use? what is the max length?)  Also
changing the NBD protocol is not under consideration.

I think we need to consider these aspects separately:

(a) How filters and plugins get the client exportname.

(b) How filters and plugins respond when the client issues NBD_OPT_LIST.

(c) How the server, filters and plugins respond to NBD_OPT_INFO.


(a) Client exportname
---------------------

The client sends the export name of the export it wants to access with
NBD_OPT_EXPORT_NAME or the newer NBD_OPT_GO.  This is an opaque string
(not a filename, pathname etc).  Currently filters and plugins can
fetch this using nbdkit_export_name().  However it cannot be modified
by filters.

My proposal is that we extend the .open() callback with an extra
export_name parameter:

  void *filter_open (int readonly, const char *export_name);

For plugins I have already proposed this for inclusion in API V3.  We
already do this in nbdkit-sh-plugin.  For backwards compatibility with
existing plugins we'd make nbdkit_export_name() return the export name
passed down by the last filter in the chain, and deprecate this
function in V3.

For filters the next_open field would take the export_name and this
would allow filters to modify the export name.

nbdkit-tar-filter would take an explicit tar-export-name=<new>
parameter to pass a different export name down to the underlying
plugin.  If the tar filter was implementing its own export name
processing then this parameter would either be required or would
default to "".  (If the tar filter was not implementing export name
functionality it would pass it down unchanged).


(b) Listing exports with NBD_OPT_LIST
-------------------------------------

I believe we need a new .list_exports() callback for filters and
plugins to handle this case.  I'm not completely clear on the API, but
it could be something as simple as:

  const char **list_exports (void);

returning a NULL-terminated list of strings.

Is it conceivable we will need additional fields in future?

Plugins which do not implement this would be assumed to return a
single export "" (or perhaps the -e parameter??)

Filters could ignore or replace exports by optionally implementing
this callback.  I wouldn't recommend that filters modify export names
however.


(c) Handling NBD_OPT_INFO
-------------------------

I believe our current implementation of this option (which is
definitely not being robustly tested!) is something like this:

  - NBD_OPT_INFO received from client (containing the export name)
  - open a backend handle
  - query backend for flags
  - reply to client
  - close backend handle
  - process the next option

Assuming this is actually correct, probably we can make sure that it
passes the export name to the backend_open, and everything should just
work.  In fact this would happen as a side effect of fixing (a) above
so I don't believe there's anything to do here except to have some
test coverage.


----------------------------------------------------------------------

Thoughts welcomed as always,

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v