[Libguestfs] Shebang sh plugins

Eric Blake eblake at redhat.com
Mon Nov 25 23:23:00 UTC 2019


On 11/22/19 3:07 PM, Richard W.M. Jones wrote:
> On Fri, Nov 22, 2019 at 02:55:31PM -0600, Eric Blake wrote:
>> Unrelated side topic: in your recent addition of eval.sh, you
>> wondered if we should promote it to a full-blown plugin rather than
>> just an example script.  But reading 'man nbdkit-sh-plugin', there
>> is no mention of turning an executable script into a full-blown
>> plugin via a shebang, the way that python documents it.  [I guess
>> 'man nbdkit' sort of mentions it under Shebang scripts]
> 
> I believe it's not possible to do it for sh plugins.
> 
> For (eg) python plugins it works like this:
> 
>    #!/usr/sbin/nbdkit python
>    -> runs nbdkit python <script name>
>    -> <script name> is interpreted as magic script= parameter
>    -> the python plugin works by loading the script using
>       PyRun_SimpleFileEx which interprets the contents of the
>       script as python code
> 
> However for shell it doesn't work:
> 
>    #!/usr/sbin/nbdkit sh
>    -> runs nbdkit sh <script name>
>    -> <script name> is interpreted as magic script= parameter
>    -> the sh plugin works by actually executing the script
>    -> executing the script repeats the steps over from the top,
>       causing an infinite loop
> 
> If you can think of a way to make this work it would be a useful
> feature IMO.

Hmm.  Right now, we document that:

nbdkit sh file

requires 'file' to be executable, and that we call exec*("file", { 
"file", "op", ... })

and that the interpreter need not be sh (file can start with any #! to 
run a script that will be executed by another interpreter).

In order to get a shebang redirect to work, though, it seems like we 
must NOT directly re-execute the file, but instead force the file to be 
parsed by "/bin/sh" (or have some other means of identifying exactly 
which interpreter to use, outside of the normal #! means), where we 
would invoke exec*("/bin/sh", {"file", "op", ...}) and thereby skip the 
kernel's attempt to re-execute under a different interpreter.

Perhaps we could get this by teaching the sh plugin to read() the first 
line of the script; if it starts with #! and contains 'nbdkit sh', then 
we force exec through "sh" (hmm, is there a way to force exec through 
bash instead?).  Otherwise, we exec through the file-name as-is.  Then 
our sh plugin would support two modes: the existing mode (we exec any 
interpreter by letting the kernel interpret the #! line, but you can't 
get shebang forwarding over to nbdkit), and the new mode (we exec a 
shell directly to bypass kernel shebang interpretation, but now you can 
write a shell script that gets shebang forwarding over to nbdkit).

It's also a bummer that different platforms vary on how #! lines are 
parsed - you can't portably pass more than a single argument, so there 
is no portable way to write '#!/path/to/nbdkit sh shell=/bin/bash' as a 
convenient way to choose which shell to use when exec'ing the script 
while bypassing the #!, short of having nbdkit reimplement what GNU 
coreutils recently added as '/bin/env -S' (and that reimplementation 
would have to be in nbdkit proper, not in the sh plugin, because such a 
shebang line would result in 'nbdkit' 'sh shell=/bin/bash' 'file' 
'args', but there is no '/path/to/nbdkit-plugin-sh shell=/bin/bash.so' 
plugin to be loaded).

If we choose to make the sh plugin smart enough to parse line 1 to see 
if it is a shebang containing the substring 'nbdkit sh', we could also 
make it parse line 2 to pick up some magic comment line of 
'SHELL=/bin/bash' as the binary to exec.  That's a bit more complicated 
to document, but might serve to let us use the sh plugin for the two 
separate modes documented above (mode 1 to run an arbitrary executable 
file which may use shebang to call out its own interpreter, mode 2 to 
run an arbitrary file with a fixed interpreter allowing that file to 
start with a shebang to call out to nbdkit).  Or maybe we create two 
different plugins (keep the existing 'sh' plugin with existing 
semantics, and add a new 'shell' plugin that shares 99% of the code but 
forces execution with a fixed shell).

The other thing to consider is what brought this topic up - a question 
of whether plugins/sh/eval.sh is worth a conversion into a standalone 
plugin.  With other languages, our choice is between:
nbdkit python script
./script (where #! reinvokes nbdkit python script)

But we don't have a way to write:
nbdkit script

where nbdkit recognizes that 'script' is not a .so file, but IS a file 
that calls out python in its #!, and therefore attempts to load the 
python.so plugin.

The question with the eval.sh plugin is if we can write:

nbdkit eval pread=...

and have nbdkit figure out that 'eval' is not a plugin but IS a file 
with a #! calling out 'nbdkit sh', and therefore load the 'sh' plugin 
with 'script=eval'.  In other words, maybe it's time to teach nbdkit 
general magic on how to handle the plugin argument (if an .so is found 
then load it; if not, then inspect the file to see which .so to use 
instead).

Or maybe we just write a C form of a plugin named 'eval' that does the 
same things as what 'sh eval.sh' does, sharing as much code as possible, 
but where you have no shebang form because you never pass a script= 
argument to the eval plugin.

All interesting thoughts, but I'm not sure I got us any closer to a nice 
conclusion.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org




More information about the Libguestfs mailing list