[Virtio-fs] [PATCH 0/4] virtiofsd: multithreading preparation part 3

Stefan Hajnoczi stefanha at redhat.com
Fri Aug 9 08:21:02 UTC 2019


On Thu, Aug 08, 2019 at 10:53:16AM +0100, Dr. David Alan Gilbert wrote:
> * Stefan Hajnoczi (stefanha at redhat.com) wrote:
> > On Wed, Aug 07, 2019 at 04:57:15PM -0400, Vivek Goyal wrote:
> > 2. Can MAP/UNMAP be performed directly in QEMU via a separate virtqueue?
> 
> I think there's two things to solve here that I don't currently know the
> answer to:
>   2a) We'd need to get the fd to qemu for the thing to mmap;
>       we might be able to cache the fd on the qemu side for existing
>       mappings, so when asking for a new mapping for an existing file then
>       it would already have the fd.
> 
>   2b) Running a device with a mix of queues inside QEMU and on
>       vhost-user; I don't think we have anything with that mix

vhost-user-net works in the same way.  The ctrl queue is handled by QEMU
and the rx/tx queues by the vhost device.  This is in fact how vhost was
initially designed: the vhost device is not a full virtio device, only
the dataplane.

> > 3. Can READ/WRITE be performed directly in QEMU via a separate virtqueue
> >    to eliminate the bad address problem?
> 
> Are you thinking of doing all read/writes that way, or just the corner
> cases? It doesn't seem worth it for the corner cases unless you're
> finding them cropping up in real work loads.

Send all READ/WRITE requests to QEMU instead of virtiofsd.

Only handle metadata requests in virtiofsd (OPEN, RELEASE, READDIR,
MKDIR, etc).

> > I'm not going to tackle DAX optimization myself right now but wanted to
> > share these ideas.
> 
> One I was thinking about that feels easier than (2) was to change the
> vhost slave protocol to be split transaction; it wouldn't do anything
> for the latency but it would be able to do some in parallel if we can
> get the kernel to feed it.

There are two cases:
1. mmapping multiple inode.  This should benefit from parallelism,
   although mmap is still expensive because it involves TLB shootdown
   for all other threads running this process.
2. mmapping the same inode.  Here the host kernel is likely to serialize
   mmaps even more, making it hard to gain performance.

It's probably worth writing a tiny benchmark first to evaluate the
potential gains.

Stefan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/virtio-fs/attachments/20190809/6b52639c/attachment.sig>


More information about the Virtio-fs mailing list