[Libguestfs] Shebang sh plugins (was: Re: [PATCH nbdkit v2 03/10] python: Implement nbdkit API version 2.)

[Nir Soffer]

On Sat, Nov 23, 2019 at 5:46 PM Richard W.M. Jones <rjones at redhat.com> wrote:
>
> On Sat, Nov 23, 2019 at 01:58:02AM +0200, Nir Soffer wrote:
> > On Fri, Nov 22, 2019 at 11:07 PM Richard W.M. Jones <rjones at redhat.com> wrote:
> > >
> > > On Fri, Nov 22, 2019 at 02:55:31PM -0600, Eric Blake wrote:
> > > > Unrelated side topic: in your recent addition of eval.sh, you
> > > > wondered if we should promote it to a full-blown plugin rather than
> > > > just an example script.  But reading 'man nbdkit-sh-plugin', there
> > > > is no mention of turning an executable script into a full-blown
> > > > plugin via a shebang, the way that python documents it.  [I guess
> > > > 'man nbdkit' sort of mentions it under Shebang scripts]
> > >
> > > I believe it's not possible to do it for sh plugins.
> > >
> > > For (eg) python plugins it works like this:
> > >
> > >   #!/usr/sbin/nbdkit python
> > >   -> runs nbdkit python <script name>
> > >   -> <script name> is interpreted as magic script= parameter
> > >   -> the python plugin works by loading the script using
> > >      PyRun_SimpleFileEx which interprets the contents of the
> > >      script as python code
> > >
> > > However for shell it doesn't work:
> > >
> > >   #!/usr/sbin/nbdkit sh
> > >   -> runs nbdkit sh <script name>
> > >   -> <script name> is interpreted as magic script= parameter
> > >   -> the sh plugin works by actually executing the script
> > >   -> executing the script repeats the steps over from the top,
> > >      causing an infinite loop
> > >
> > > If you can think of a way to make this work it would be a useful
> > > feature IMO.
> >
> > Even if we can find a way, it sounds too complicated, and I don't see
> > the use case.
>
> I think an aim of nbdkit is to make it as widely usable as possible.
> That's for example why we have apparent esoterica like plugins written
> in shell script.
>
> We want to in the future ship the (or "an") imageio plugin separately
> from virt-v2v.  A possibility is it might be shipped as a single
> Python file which starts with ‘#!/usr/bin/env nbdkit python’ which
> would allow anyone to run it directly.  They wouldn't need to
> understand anything about nbdkit or its command line.

But the imageio plugin is not a script you can run, and if you want to
make it a standalone tool it needs a lot of work that is not related to virt-v2v
like adding command line parsing or dealing with file formats.

Such tool mostly already exists in ovirt sdk examples:
https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/upload_disk.py

So for rhv-upload there is no use case for standalone script.

> So that's the use case (which to be fair is not actually used much
> today at all).  The context above is I can't think of a way to extend
> this feature to nbdkit-sh-plugin.

But today we require the plugin name in nbdkit command line:

    ./nbdkit -f -v python zero.py

It could be nice to be able to do:

    ./nbdkit -f -v zero.py

Detect that the filename is a python/lua/shell/perl,... file and then
load the right plugin.

We can use special comments in the plugin to provide more info about the plugin,
for example arguments that can be used with the plugin.

$ cat example.py
# example python plugin
# nbdkit-plugin: python
# nbdkit-argument: filename

code...

$ ./nbdkit -f -v example.py --help

show help about example.py usage...

$ ./nbdkit -f -v example.py filename=/dev/zero

Internally nbdkit will replace this command with:

    nbdkit -f -v python example.py filename=/dev/zero

And continue the normal flow.

So the actual nbdkit command is:

   nbdkit plugin-name|script ...

Nir