[Virtio-fs] Achieving parallelism in virtiofsd

[Stefan Hajnoczi]

Hi,
Here are my plans for achieving parallelism in virtiofsd.  This will
improve performance for workloads that keep more than one request in
flight at a time.

Today virtiofsd performance is limited because it only processes 1
request at a time.  This can be improved in two independent ways:
parallel request processing and multiqueue.

Parallel request processing means working on more than one request at a
time.  A request that blocks should not prevent the next request from
executing.  The FUSE protocol is asynchronous so it's just a question of
adjusting virtiofsd.

Multiqueue means providing several request virtqueues instead of just
one.  This can be used with CPU and NUMA pinning so that request
processing takes place on a core and NUMA node.  Better locality can
result in higher performance.

virtiofsd needs to offer both of these features.  The model I'm
proposing is one thread per virtqueue which distributes requests to a
thread pool for execution.  Each virtqueue thread and its thread pool
can be bound to a subset of CPUs.

Separate optimizations such as virtqueue polling could be added later to
reduce latency.

I plan to use the glib thread pool, which offers the basic functionality
that virtiofsd requires.  In the process of this work I will also audit
and fix passthrough_ll.c's thread-safety.

Feedback is appreciated!

Stefan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/virtio-fs/attachments/20190704/0dee4f0f/attachment.sig>