[Libguestfs] [PATCH nbdkit INCOMPLETE 0/6] Introduce filters to nbdkit.

[Eric Blake]

On 01/14/2018 06:11 AM, Richard W.M. Jones wrote:
> This patch isn't complete (patch 6/6 isn't finished) so it's just for
> discussion, although it does compile and run.
> 
> This introduces to nbdkit a concept of "filters" which can be placed
> in front of plugins to modify their behaviour.  Some examples where
> you might use filters:
> 
>  * Serve a subset of the data, such as (offset, range) or a
>    single partition from a disk image.
> 
>  * Inject delays or errors for testing clients.
> 
>  * Implement "copy-on-write" (a feature found in other NBD servers).
> 
> Filters are implemented by allowing them to intercept methods before a
> plugin gets them.  For example to implement a read delay the filter
> would register for a .pread hook which is implemented like this:
> 
>   static int
>   delay_pread (void *handle, void *buf, uint32_t count, uint64_t offset,
>                int (*next) (void *data,
>                             void *buf, uint32_t count, uint64_t offset),
>                void *data)

Interesting idea.  I'm also wondering if we should start exposing flags
from the client to the end user; in particular, NBD_CMD_FLAG_FUA is a
useful optimization that avoids the overhead of a full-device flush if
it is supported through the entire callstack - but that means we need
both a new .pread callback variant that accepts a flag, as well as a
filter signature that can pass the flag unchanged, as well as a way for
a plugin to inform the main engine which flags it is willing to accept
(at least with the NBD plugin, the ability to accept the FUA flag is
dependent on the remote server, and it's nicer to have the main engine
continue to emulate the needed flushes rather than having to duplicate
the code in the NBD plugin for remote servers that don't accept the flag).


> For the filters I want to write this works fine, but with two caveats:
> 
> (1) If new datapath methods are introduced then filters won't get to
> see them by default.  For example, if we modify offsets when calling
> .pread and .pwrite, and then later the .zero method is added to
> plugins, then existing filters won't modify the parameters of .zero
> correctly (resulting in wrong data being zeroed).

Indeed, and my concern that we want new callbacks for handling flags in
existing plugins is another instance of this.

> 
> (2) You cannot do anything more complex in one of these functions than
> calling the single underlying plugin method, possibly modifying the
> arguments.  For example it's hard to see how a "qcow2 decoder" filter
> could be written since it would need to have full access to the plugin
> methods, not just to the single method.

Yeah, it's definitely conceivable that a filter will want to be able to
access the underlying .read as part of implementing an intelligent .write.

> 
> Unfortunately solving (1) & (2) makes the whole thing a lot more
> complicated.

In a lot of ways, it sounds like the development of filters in nbdkit
will resemble the BDS filters in qemu by the time we get through with
everything.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 619 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/libguestfs/attachments/20180115/9c2312c7/attachment.sig>