[Libguestfs] [nbdkit PATCH v2] Introduce cacheextents filter
Eric Blake
eblake at redhat.com
Thu May 16 13:05:18 UTC 2019
On 5/15/19 5:39 PM, Martin Kletzander wrote:
> This filter caches the last result of the extents() call and offers a nice
> speed-up for clients that only support req_on=1 in combination with plugins like
s/on/one/
> vddk, which has no overhead for returning information for multiple extents in
> one call, but that call is very time-consuming.
>
> Quick test showed that on a fast connection and a sparsely allocated 16G disk
> with a OS installed `qemu-img map` runs 16s instead of 33s (out of which it
> takes 5s to the first extents request). For 100G disk with no data on it, that
> is one hole extent spanning the whole disk (where there is no space for
> improvement) this does not add any noticeable overhead.
>
> Signed-off-by: Martin Kletzander <mkletzan at redhat.com>
> ---
> +++ b/filters/cacheextents/cacheextents.c
> +
> +static int
> +cacheextents_extents (struct nbdkit_next_ops *next_ops, void *nxdata,
> + void *handle, uint32_t count, uint64_t offset, uint32_t flags,
> + struct nbdkit_extents *extents,
> + int *err)
> +{
> + ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&lock);
> +
> + nbdkit_debug ("cacheextents:"
> + " cache_start=%" PRIu64
> + " cache_end=%" PRIu64
> + " cache_extents=%p",
> + cache_start, cache_end, cache_extents);
> +
> + if (cache_extents &&
> + offset >= cache_start && offset < cache_end) {
> + nbdkit_debug ("cacheextents: returning from cache");
> + return cacheextents_add (extents, err);
> + }
> +
> + nbdkit_debug ("cacheextents: cache miss");
> + int r = next_ops->extents (nxdata, count, offset, flags, extents, err);
This is a bit pessimistic. Observe:
request A (offset=0, count=1G) populates extents (0-.5G data, .5G-1.5G
hole)
request B (offset=.5G, count=1G) serviced from the cache
compared to:
request A (offset=0, count=1G) populates extents (0-.5G data, .5G-1.0G
hole)
request B (offset=.5G, count=1G) treated as cache miss
It should be possible to note that request B overlaps with the cache,
and to therefore do a three-step action: fill 'extents' with the tail of
the cached extents (filling .5-1.0G), call next_ops->extents for the
remainder (1.0G-wherever), extend the cache to fill out the additional
information learned.
> + if (r == -1)
> + return r;
> +
> + return cacheextents_fill (extents, err);
> +}
> +
> +++ b/filters/cacheextents/nbdkit-cacheextents-filter.pod
> @@ -0,0 +1,47 @@
> +=head1 NAME
> +
> +nbdkit-cacheextents-filter - cache extents
> +=head1 SEE ALSO
> +
> +L<nbdkit(1)>,
> +L<nbdkit-cache-filter(1)>,
> +L<nbdkit-readahead-filter(1)>,
> +L<nbdkit-vddk-plugin(1)>,
> +L<nbdkit-filter(3)>,
> +L<qemu-img(1)>.
> +
I'd also patch filters/nbdkit-cache-filter.pod to cross-link to the
cacheextents filter. And nbdkit-filter.pod probably needs an update to
point to all filters (hmm, I missed that in my recent proposed patch to
add a nocache filter; that filter could also do with a link to
cacheextents, as the two are orthogonal and can both be used at once)
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/libguestfs/attachments/20190516/cd1510f0/attachment.sig>
More information about the Libguestfs
mailing list