[Libguestfs] [nbdkit PATCH v3 12/16] filters: New API for plugin context management
Eric Blake
eblake at redhat.com
Fri Mar 5 23:31:16 UTC 2021
As promised in an earlier patch, get rid of the temporary
nbdkit_backend_reopen, and replace it with the finer-grained
combination of nbdkit_context_get_backend, nbdkit_context_next_open,
nbdkit_context_next_close, and nbdkit_context_set_next. Update the
retry filter to use the new API. Change the (temporary) preprocessor
guard from NBDKIT_RETRY_FILTER to NBDKIT_TYPESAFE (will be dropped
later once more filters are converted). Drop the now-unused
backend_reopen() function. Expose .prepare and .finalize callbacks
into struct nbdkit_next_ops for use during manual context
manipulation.
With this in place, it is now possible for a filter to open more than
one context into the backend from a single connection (shown here in
the retry filter), and conversely to share a single backend context
across multiple connections (an upcoming patch will demonstrate this
in the ext2 filter, although TODO mentions a few more steps we need to
get there).
---
docs/nbdkit-filter.pod | 207 ++++++++++++++++++++++++++++------------
include/nbdkit-filter.h | 27 ++++--
server/internal.h | 3 -
server/backend.c | 30 +-----
server/filters.c | 43 +++++++--
server/nbdkit.syms | 11 ++-
filters/retry/retry.c | 42 ++++++--
TODO | 36 +++----
8 files changed, 260 insertions(+), 139 deletions(-)
diff --git a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod
index 0b120efe..d13ac1d7 100644
--- a/docs/nbdkit-filter.pod
+++ b/docs/nbdkit-filter.pod
@@ -132,25 +132,33 @@ C<nbdkit_next_after_fork>, C<nbdkit_next_preconnect>,
C<nbdkit_next_list_exports>, C<nbdkit_next_default_export>,
C<nbdkit_next_open>) and a structure called C<struct nbdkit_next_ops>.
These abstract the next plugin or filter in the chain. There is also
-an opaque pointer C<context> or C<nxdata> which must be passed along
-when calling these functions. The value of C<nxdata> passed to
-C<.prepare> has a stable lifetime that lasts to the corresponding
-C<.finalize>, with all intermediate functions (such as C<.pread>)
-receiving the same value for convenience. Functions where C<nxdata>
-is not reused are C<.config>, C<.config_complete>, C<.get_ready>, and
-C<.after_fork>, which are called during initialization outside any
-connections, and C<.preconnect>, C<.list_exports>, C<.default_export>,
-and C<.open> which are called based on client connections but prior to
-the stable lifetime of C<.prepare>. The value of C<context> passed to
-C<.open> has a lifetime that lasts until the matching C<.close> for
-use by C<nbdkit_backend_reopen>.
+an opaque pointer C<backend>, C<context> or C<nxdata> which must be
+passed along when calling these functions. The value of C<backend> is
+stable between C<.after_fork>, C<.preconnect>, C<.list_exports>, and
+C<.default_export>, and can also be obtained by using
+C<nbdkit_context_get_backend> on the C<context> parameter to C<.open>.
+
+Meanwhile, if the filter does not use C<nbdkit_context_set_next>, the
+value of C<nxdata> passed to C<.prepare> has a stable lifetime that
+lasts to the corresponding C<.finalize>, with all intermediate
+functions (such as C<.pread>) receiving the same value for
+convenience. Functions where C<nxdata> is not reused are C<.config>,
+C<.config_complete>, C<.get_ready>, and C<.after_fork>, which are
+called during initialization outside any connections, and
+C<.preconnect>, C<.list_exports>, C<.default_export>, and C<.open>
+which are called based on client connections but prior to the stable
+lifetime of C<.prepare>. The value of C<context> passed to C<.open>
+has a lifetime that lasts until the matching C<.close> for use by
+C<nbdkit_context_get_backend>.
=head2 Next config, open and close
The filter’s C<.config>, C<.config_complete>, C<.get_ready>,
-C<.after_fork> and C<.open> methods may only call the next C<.config>,
-C<.config_complete>, C<.get_ready>, C<.after_fork> and C<.open> method
-in the chain (optionally for C<.config>).
+C<.after_fork>, C<.preconnect>, C<.list_exports>, C<.default_export>
+and C<.open> methods may only call the next C<.config>,
+C<.config_complete>, C<.get_ready>, C<.after_fork>, C<.preconnect>,
+C<.list_exports>, C<.default_export> and C<.open> method in the chain
+(optionally for C<.config> and C<.open>).
The filter’s C<.close> method is called when an old connection closed,
and this has no C<next> parameter because it cannot be
@@ -158,15 +166,98 @@ short-circuited.
=head2 C<next_ops>
-The filter’s other methods like C<.prepare>, C<.get_size>, C<.pread>
-etc ― always called in the context of a connection ― are passed a
-pointer to C<struct nbdkit_next_ops> which contains a comparable set
-of accessors to plugin methods that can be called during a connection.
-The C<next_ops> parameter is stable between C<.prepare> and
-C<.finalize>; intermediate functions (such as C<.pread>) receive the
-same value for convenience.
-
-It is possible for a filter to issue (for example) extra
+The filter generally needs to call into the underlying plugin, which
+is done via a pointer to C<struct nbdkit_next_ops>. The most common
+behavior is to create a next context per connection by calling the
+C<next_open> parameter during C<.open>, at which point the next
+context will be automatically provided to the filter’s other methods
+like C<.prepare>, C<.get_size>, C<.pread> etc. The C<next_ops> struct
+contains a comparable set of accessors to plugin methods that can be
+called during a connection. When using automatic registration, the
+C<next_ops> parameter is stable between C<.prepare> and C<.finalize>,
+and nbdkit automatically prepares, finalizes, and closes the next
+context at the right point in the filter connection lifecycle.
+
+Alternatively, the filter can manage plugin contexts manually, whether
+to multiplex multiple client connections through a single context into
+the plugin, or to open multiple plugin contexts to perform retries or
+otherwise service a single client connection more efficiently. In
+this mode of operation, the filter uses C<nbdkit_next_context_open> to
+open a plugin context using the C<backend> parameter passed to
+C<.after_fork>, C<.preconnect>, C<.list_exports>, C<.default_export>,
+or obtained from using C<nbdkit_context_get_backend> on the C<context>
+parameter to C<.open>. The resulting next context has a lifecycle
+under manual control, where the filter must use C<next-E<gt>prepare
+(next)> before using any other function pointers within the next
+context, and must reclaim the memory using C<next-E<gt>finalize
+(next)> and C<nbdkit_next_context_close> when done. A filter using
+manual lifecycle management may use C<nbdkit_context_set_next> to
+associate the next context into the current connection, which lets
+nbdkit then pass that context as the C<next_ops> parameter to future
+connection-related functions like C<.pread> and take over lifecycle
+responsibility.
+
+=head3 C<nbdkit_context_get_backend>
+
+=head3 C<nbdkit_next_context_open>
+
+=head3 C<nbdkit_next_context_close>
+
+=head3 C<nbdkit_context_set_next>
+
+ nbdkit_backend *nbdkit_context_get_backend (nbdkit_context *context);
+
+Obtains the backend pointer from the C<context> parameter to C<.open>,
+matching the backend pointer available to C<.after_fork>,
+C<.preconnect>, C<.list_exports>, and C<.default_export>. This
+backend pointer has a stable lifetime from the time of C<.after_fork>
+until C<.unload>.
+
+ nbdkit_next *nbdkit_next_context_open (nbdkit_backend *backend,
+ int readonly, const char *exportname);
+
+This function attempts to open a new context into the plugin in
+relation to the filter's current C<backend>. The C<readonly> and
+C<exportname> parameters behave the same as documented in C<.open>.
+The resulting context will be under the filter's manual lifecycle
+control unless the filter associates it into the connection with
+C<nbdkit_context_set_next>. The filter should be careful to not
+violate any threading model restrictions of the plugin if it opens
+more than one context.
+
+ void nbdkit_next_context_close (nbdkit_next *next);
+
+This function closes a context into the plugin. If the context has
+previously been prepared, it should first be finalized before using
+this function. This function does not need to be called for a plugin
+context that has been associated with the filter connection via
+C<nbdkit_context_set_next> prior to the C<.close> callback.
+
+ nbdkit_next *nbdkit_context_set_next (nbdkit_context *context,
+ nbdkit_next *next);
+
+This function associates a plugin context with the filter's current
+connection context, given by the C<context> parameter to C<.open>.
+Once associated, this plugin context will be given as the C<next_ops>
+parameter to all other connection-specific callbacks. If associated
+during C<.open>, nbdkit will take care of preparing the context prior
+to C<.prepare>; if still associated before C<.finalize>, nbdkit will
+take care of finalizing the context, and also for closing it. A
+filter may also pass C<NULL> for C<next>, to remove any association;
+if no plugin context is associated with the connection, then filter
+callbacks such as C<.pread> will receive C<NULL> for their C<next_ops>
+parameter.
+
+This function returns the previous context that had been associated
+with the connection prior to switching the association to C<next>;
+this result will be C<NULL> if there was no previous association. The
+filter assumes manual responsibility for any remaining lifecycle
+functions that must be called on the returned context.
+
+=head2 Using C<next_ops>
+
+Regardless of whether the plugin context is managed automatically or
+manually, it is possible for a filter to issue (for example) extra
C<next_ops-E<gt>pread> calls in response to a single C<.pwrite> call.
Note that the semantics of the functions in C<struct nbdkit_next_ops>
@@ -177,6 +268,9 @@ C<nbdkit_set_error> or setting C<errno>), whereas
C<next_ops-E<gt>pread> exposes this via an explicit parameter,
allowing a filter to learn or modify this error if desired.
+Use of C<next_ops-E<gt>prepare> and C<next_ops-E<gt>finalize> is only
+needed when manually managing the plugin context lifetime.
+
=head2 Other considerations
You can modify parameters when you call the C<next> function. However
@@ -189,7 +283,8 @@ for avoiding a memory leak while still obeying lifecycle constraints.
Note that if your filter registers a callback but in that callback it
doesn't call the C<next> function then the corresponding method in the
plugin will never be called. In particular, your C<.open> method, if
-you have one, B<must> call the C<.next> method.
+you have one, B<must> call the C<next> method if you want the
+underlying plugin to be available to all further C<next_ops> use.
=head1 CALLBACKS
@@ -468,31 +563,21 @@ C<nbdkit_is_tls>.
The filter should generally call C<next> as its first step, to
allocate from the plugin outwards, so that C<.close> running from the
outer filter to the plugin will be in reverse. Skipping a call to
-C<next> is acceptable if the filter will not access C<next_ops>
-during any of the remaining callbacks reached on the same connection.
+C<next> is acceptable if the filter will not access C<next_ops> during
+any of the remaining callbacks reached on the same connection. The
+C<next> function is provided for convenience; the same functionality
+can be obtained manually (other than error checking) by using the
+following:
+
+ nbdkit_context_set_next (context, nbdkit_next_context_open
+ (nbdkit_context_get_backend (context), readonly, exportname));
The value of C<context> in this call has a lifetime that lasts until
-the counterpart C<.close>, and it is this value (and not the distinct
-C<nxdata> of C<.pread> and friends) that must be passed as the first
-parameter to C<nbdkit_backend_reopen> by a filter attempting to retry
-a connection into the backend.
-
-=head3 Reopening the backend
-
-Filters have access to a function for reopening the backend:
-
- int (nbdkit_context *context, int readonly, const char *exportname,
- void **nxdata);
-
-This function is used by L<nbdkit-retry-filter(1)> to close and reopen
-the underlying plugin, with possible changes to the C<readonly> and
-C<exportname> parameters in relation to the original opening. It
-should be used with caution because it is difficult to use safely.
-The C<context> parameter to this function should be the C<context>
-parameter originally passed in to C<.open>; while the C<nxdata>
-pointer should be the address of C<nxdata> from any function with a
-C<next_ops> parameter (such as C<.pread>) that wants to call into the
-plugin after the reopen.
+the counterpart C<.close>, and it is this value that may be passed to
+C<nbdkit_context_get_backend> to obtain the C<backend> parameter used
+to open a plugin context with C<nbdkit_next_context_open>, as well as
+the C<context> parameter used to associate a plugin context into the
+current connection with C<nbdkit_context_set_next>.
=head2 C<.close>
@@ -524,7 +609,8 @@ the plugin's C<.get_size> and C<.pread> methods via C<next_ops>). Or
if you need to cleanly update superblock data in the image on close
you can do it in your C<.finalize> method (calling the plugin's
C<.pwrite> method). Doing these things in the filter's C<.open> or
-C<.close> method is not possible.
+C<.close> method is not possible without using manual context
+lifecycle management.
For C<.prepare>, the value of C<readonly> is the same as was passed to
C<.open>, declaring how this filter will be used.
@@ -541,17 +627,18 @@ overrides query functions or makes I/O calls into the plugin before
handshaking is complete, where the filter needs to make those
prerequisite calls manually during C<.prepare>.
-There is no C<next_ops-E<gt>prepare> or C<next_ops-E<gt>finalize>.
-Unlike other filter methods, prepare and finalize are not chained
-through the C<next_ops> structure. Instead the core nbdkit server
-calls the prepare and finalize methods of all filters. Prepare
-methods are called starting with the filter closest to the plugin and
-proceeding outwards (matching the order of C<.open> if all filters
-call C<next> before doing anything locally), and only when an outer
-filter did not skip the C<next> call during C<.open>. Finalize
-methods are called in the reverse order of prepare methods, with the
-outermost filter first (and matching the order of C<.close>), and only
-if the prepare method succeeded.
+While there are C<next_ops-E<gt>prepare> and C<next_ops-E<gt>finalize>
+functions, these are different from other filter methods, in that any
+plugin context associated with the current connection (via the C<next>
+parameter to C<.open>, or via C<nbdkit_context_set_next>, is prepared
+and finalized automatically by nbdkit, so they are only used during
+manual lifecycle management. Prepare methods are called starting with
+the filter closest to the plugin and proceeding outwards (matching the
+order of C<.open> if all filters call C<next> before doing anything
+locally), and only when an outer filter did not skip the C<next> call
+during C<.open>. Finalize methods are called in the reverse order of
+prepare methods, with the outermost filter first (and matching the
+order of C<.close>), and only if the prepare method succeeded.
If there is an error, both callbacks should call C<nbdkit_error> with
an error message and return C<-1>. An error in C<.prepare> is
diff --git a/include/nbdkit-filter.h b/include/nbdkit-filter.h
index 3f3f61cd..a8819c9b 100644
--- a/include/nbdkit-filter.h
+++ b/include/nbdkit-filter.h
@@ -56,7 +56,7 @@ extern "C" {
typedef struct backend nbdkit_backend;
typedef struct context nbdkit_context;
typedef struct context nbdkit_next;
-#elif defined NBDKIT_RETRY_FILTER /* Hack to expose reopen to retry filter */
+#elif defined NBDKIT_TYPESAFE /* Temporary define while converting filters */
typedef struct nbdkit_backend nbdkit_backend;
typedef struct nbdkit_context nbdkit_context;
typedef struct nbdkit_next_ops nbdkit_next;
@@ -81,6 +81,12 @@ typedef int nbdkit_next_open (nbdkit_context *context,
int readonly, const char *exportname);
struct nbdkit_next_ops {
+ /* These callbacks are only needed when managing the backend manually
+ * rather than via nbdkit_next_open.
+ */
+ int (*prepare) (nbdkit_next *nxdata);
+ int (*finalize) (nbdkit_next *nxdata);
+
/* These callbacks are the same as normal plugin operations. */
int64_t (*get_size) (nbdkit_next *nxdata);
const char * (*export_description) (nbdkit_next *nxdata);
@@ -120,7 +126,7 @@ struct nbdkit_extent {
uint32_t type;
};
-NBDKIT_EXTERN_DECL (struct nbdkit_extents *,nbdkit_extents_new,
+NBDKIT_EXTERN_DECL (struct nbdkit_extents *, nbdkit_extents_new,
(uint64_t start, uint64_t end));
NBDKIT_EXTERN_DECL (void, nbdkit_extents_free, (struct nbdkit_extents *));
NBDKIT_EXTERN_DECL (size_t, nbdkit_extents_count,
@@ -153,14 +159,15 @@ NBDKIT_EXTERN_DECL (size_t, nbdkit_exports_count,
NBDKIT_EXTERN_DECL (const struct nbdkit_export, nbdkit_get_export,
(const struct nbdkit_exports *, size_t));
-#ifdef NBDKIT_RETRY_FILTER
-/* Performs close + open on the underlying chain.
- * Used by the retry filter.
- */
-NBDKIT_EXTERN_DECL (int, nbdkit_backend_reopen,
- (nbdkit_context *context, int readonly,
- const char *exportname, nbdkit_next **nxdata));
-#endif
+/* Manual management of backend access. */
+NBDKIT_EXTERN_DECL (nbdkit_backend *, nbdkit_context_get_backend,
+ (nbdkit_context *context));
+NBDKIT_EXTERN_DECL (nbdkit_next *, nbdkit_next_context_open,
+ (nbdkit_backend *backend, int readonly,
+ const char *exportname));
+NBDKIT_EXTERN_DECL (void, nbdkit_next_context_close, (nbdkit_next *next));
+NBDKIT_EXTERN_DECL (nbdkit_next *, nbdkit_context_set_next,
+ (nbdkit_context *context, nbdkit_next *next));
/* Filter struct. */
struct nbdkit_filter {
diff --git a/server/internal.h b/server/internal.h
index b944e56d..849edbf8 100644
--- a/server/internal.h
+++ b/server/internal.h
@@ -430,9 +430,6 @@ extern bool backend_valid_range (struct context *c,
uint64_t offset, uint32_t count)
__attribute__((__nonnull__ (1)));
-extern int backend_reopen (struct context *c,
- int readonly, const char *exportname)
- __attribute__((__nonnull__ (1, 3)));
extern const char *backend_export_description (struct context *c)
__attribute__((__nonnull__ (1)));
extern int64_t backend_get_size (struct context *c)
diff --git a/server/backend.c b/server/backend.c
index 1c78556f..69625597 100644
--- a/server/backend.c
+++ b/server/backend.c
@@ -210,6 +210,8 @@ backend_default_export (struct backend *b, int readonly)
}
static struct nbdkit_next_ops next_ops = {
+ .prepare = backend_prepare,
+ .finalize = backend_finalize,
.export_description = backend_export_description,
.get_size = backend_get_size,
.can_write = backend_can_write,
@@ -363,34 +365,6 @@ backend_valid_range (struct context *c, uint64_t offset, uint32_t count)
offset + count <= c->exportsize;
}
-/* Core functionality of nbdkit_backend_reopen for retry filter */
-
-int
-backend_reopen (struct context *c, int readonly, const char *exportname)
-{
- struct backend *b = c->b;
-
- controlpath_debug ("%s: reopen readonly=%d exportname=\"%s\"",
- b->next->name, readonly, exportname);
-
- if (c->c_next) {
- if (backend_finalize (c->c_next) == -1)
- return -1;
- backend_close (c->c_next);
- c->c_next = NULL;
- }
- c->c_next = backend_open (b->next, readonly, exportname);
- if (c->c_next == NULL)
- return -1;
- if (backend_prepare (c->c_next) == -1) {
- backend_finalize (c->c_next);
- backend_close (c->c_next);
- c->c_next = NULL;
- return -1;
- }
- return 0;
-}
-
/* Wrappers for all callbacks in a filter's struct nbdkit_next_ops. */
const char *
diff --git a/server/filters.c b/server/filters.c
index 228e2610..d40c928a 100644
--- a/server/filters.c
+++ b/server/filters.c
@@ -263,12 +263,14 @@ filter_default_export (struct backend *b, int readonly, int is_tls)
static int
next_open (struct context *c, int readonly, const char *exportname)
{
- struct backend *b = c->b;
- struct context *c_next = backend_open (b->next, readonly, exportname);
+ struct backend *b = nbdkit_context_get_backend (c);
+ struct context *c_next = nbdkit_next_context_open (b, readonly, exportname);
+ struct context *old;
if (c_next == NULL)
return -1;
- c->c_next = c_next;
+ old = nbdkit_context_set_next (c, c_next);
+ assert (old == NULL);
return 0;
}
@@ -694,12 +696,35 @@ filter_register (struct backend *next, size_t index, const char *filename,
return (struct backend *) f;
}
-int
-nbdkit_backend_reopen (struct context *c, int readonly,
- const char *exportname, struct context **nxdata)
+struct backend *
+nbdkit_context_get_backend (struct context *c)
{
- int r = backend_reopen (c, readonly, exportname);
+ assert (c);
+ return c->b;
+}
+
+struct context *
+nbdkit_next_context_open (struct backend *b,
+ int readonly, const char *exportname)
+{
+ assert (b);
+ return backend_open (b->next, readonly, exportname);
+}
+
+void
+nbdkit_next_context_close (struct context *c)
+{
+ if (c)
+ backend_close (c);
+}
+
+struct context *
+nbdkit_context_set_next (struct context *c, struct context *next)
+{
+ struct context *old;
- *nxdata = c->c_next;
- return r;
+ assert (c);
+ old = c->c_next;
+ c->c_next = next;
+ return old;
}
diff --git a/server/nbdkit.syms b/server/nbdkit.syms
index 50411d27..143c4ebf 100644
--- a/server/nbdkit.syms
+++ b/server/nbdkit.syms
@@ -41,33 +41,36 @@
nbdkit_absolute_path;
nbdkit_add_export;
nbdkit_add_extent;
- nbdkit_backend_reopen;
+ nbdkit_context_get_backend;
+ nbdkit_context_set_next;
nbdkit_debug;
nbdkit_error;
nbdkit_export_name;
nbdkit_exports_count;
nbdkit_exports_free;
nbdkit_exports_new;
- nbdkit_get_export;
nbdkit_extents_aligned;
nbdkit_extents_count;
nbdkit_extents_free;
nbdkit_extents_full;
nbdkit_extents_new;
+ nbdkit_get_export;
nbdkit_get_extent;
nbdkit_is_tls;
nbdkit_nanosleep;
+ nbdkit_next_context_close;
+ nbdkit_next_context_open;
nbdkit_parse_bool;
- nbdkit_parse_int8_t;
nbdkit_parse_int16_t;
nbdkit_parse_int32_t;
nbdkit_parse_int64_t;
+ nbdkit_parse_int8_t;
nbdkit_parse_int;
nbdkit_parse_size;
- nbdkit_parse_uint8_t;
nbdkit_parse_uint16_t;
nbdkit_parse_uint32_t;
nbdkit_parse_uint64_t;
+ nbdkit_parse_uint8_t;
nbdkit_parse_unsigned;
nbdkit_peer_gid;
nbdkit_peer_name;
diff --git a/filters/retry/retry.c b/filters/retry/retry.c
index cfaea7d4..22527c7d 100644
--- a/filters/retry/retry.c
+++ b/filters/retry/retry.c
@@ -40,7 +40,7 @@
#include <string.h>
#include <sys/time.h>
-#define NBDKIT_RETRY_FILTER /* Hack to expose reopen */
+#define NBDKIT_TYPESAFE /* Hack to expose context APIs */
#include <nbdkit-filter.h>
#include "cleanup.h"
@@ -109,7 +109,7 @@ retry_config (nbdkit_next_config *next, nbdkit_backend *nxdata,
struct retry_handle {
int readonly; /* Save original readonly setting. */
char *exportname; /* Client exportname. */
- nbdkit_context *backend; /* Backend learned during .open. */
+ nbdkit_context *context; /* Context learned during .open. */
unsigned reopens;
bool open;
};
@@ -131,7 +131,7 @@ retry_open (nbdkit_next_open *next, nbdkit_context *nxdata,
h->readonly = readonly;
h->exportname = strdup (exportname);
- h->backend = nxdata;
+ h->context = nxdata;
if (h->exportname == NULL) {
nbdkit_error ("strdup: %m");
free (h);
@@ -178,6 +178,8 @@ static bool
do_retry (struct retry_handle *h, struct retry_data *data,
nbdkit_next **next, const char *method, int *err)
{
+ nbdkit_next *new_next, *old_next;
+
/* If it's the first retry, initialize the other fields in *data. */
if (data->retry == 0)
data->delay = initial_delay;
@@ -207,17 +209,39 @@ do_retry (struct retry_handle *h, struct retry_data *data,
if (exponential_backoff)
data->delay *= 2;
- /* Reopen the connection. */
+ /* Close the old connection. */
h->reopens++;
- if (nbdkit_backend_reopen (h->backend, h->readonly || force_readonly,
- h->exportname, next) == -1) {
- /* If the reopen fails we treat it the same way as a command
- * failing.
+ h->open = false;
+ if (*next != NULL) {
+ /* Failure to finalize a connection indicates permanent data loss,
+ * which we treat the same as the original command failing.
*/
- h->open = false;
+ if ((*next)->finalize (*next) == -1) {
+ *err = ESHUTDOWN;
+ goto again;
+ }
+ nbdkit_next_context_close (*next);
+ old_next = nbdkit_context_set_next (h->context, NULL);
+ assert (old_next == *next);
+ *next = NULL;
+ }
+ /* Open a new connection. */
+ new_next = nbdkit_next_context_open (nbdkit_context_get_backend (h->context),
+ h->readonly || force_readonly,
+ h->exportname);
+ if (new_next == NULL) {
+ *err = ESHUTDOWN;
+ goto again;
+ }
+ if (new_next->prepare (new_next) == -1) {
+ new_next->finalize (new_next);
+ nbdkit_next_context_close (new_next);
*err = ESHUTDOWN;
goto again;
}
+ old_next = nbdkit_context_set_next (h->context, new_next);
+ assert (old_next == NULL);
+ *next = new_next;
h->open = true;
/* Retry the data command. */
diff --git a/TODO b/TODO
index c96a8b22..2bde2478 100644
--- a/TODO
+++ b/TODO
@@ -73,22 +73,13 @@ General ideas for improvements
* Background thread for filters. Some filters (readahead, cache and
proposed scan filter - see below) could be more effective if they
- were able to defer work to a background thread. However it's not as
- simple as just creating a background thread, because you only have
- access to the connection / next_ops from the context of a filter
- callback. (Also you can't "save" next_ops because it becomes
- invalid outside the callback). The background thread would need to
- have its own connection to the plugin, which would be independent of
- any client connection, and this requires some care because it breaks
- an existing assumption.
-
-* Add scan filter. This would be placed on top of cache filters and
- would scan (read) the whole disk in the background, ensuring it is
- copied into the cache. Useful if you have a slow plugin, limited
- size device, and lots of local disk space, especially if you know
- that the NBD clients will eventually read all of the device. RWMJ
- wrote an implementation of this but it doesn't work well without a
- background thread.
+ were able to defer work to a background thread. We finally have
+ nbdkit_next_context_open and friends for allowing a background
+ thread to have access into the plugin, but still need to worry about
+ thread-safety (how much must the filter do vs. nbdkit, to avoid
+ calling into the plugin too many times at once) and cleanup
+ (spawning the thread during .after_fork is viable, but cleaning it
+ up during .unload is too late).
* "nbdkit.so": nbdkit as a loadable shared library. The aim of nbdkit
is to make it reusable from other programs (see nbdkit-captive(1)).
@@ -196,6 +187,19 @@ Python:
Suggestions for filters
-----------------------
+* Add scan filter. This would be placed on top of cache filters and
+ would scan (read) the whole disk in the background, ensuring it is
+ copied into the cache. Useful if you have a slow plugin, limited
+ size device, and lots of local disk space, especially if you know
+ that the NBD clients will eventually read all of the device. RWMJ
+ wrote an implementation of this but it doesn't work well without a
+ background thread.
+
+* Add shared filter. Take advantage of filter context APIs to open a
+ single context into the backend shared among multiple client
+ connections. This may even allow a filter to offer a more parallel
+ threading model than the underlying plugin.
+
* LUKS encrypt/decrypt filter, bonus points if compatible with qemu
LUKS-encrypted disk images
--
2.30.1
More information about the Libguestfs
mailing list