[Virtio-fs] [PATCH 3/4] virtiofsd: use file-backend memory region for virtiofsd's cache area

[Liu Bo]

On Mon, May 20, 2019 at 09:49:34AM -0400, Vivek Goyal wrote:
> On Fri, May 17, 2019 at 07:28:22PM -0700, Liu Bo wrote:
> 
> [..]
> > > > > > > That case happened with cache=none and the dax mount option.
> > > > > > > 
> > > > > > > The general problem is when FUSE_READ/FUSE_WRITE is sent and the buffer
> > > > > > > is outside guest RAM.
> > > > 
> > > > Stefan,
> > > > 
> > > > Can this be emulated by sending a request to qemu? If virtiofsd can detect
> > > > that source/destination of READ/WRITE is not guest RAM, can it forward
> > > > message to qemu to do this operation (which has access to all the DAX
> > > > windows)?
> > > > 
> > > > This probably will mean introducing new messages like
> > > > setupmapping/removemapping messages between virtiofsd/qemu.  
> > > 
> > > Yes, interesting idea!
> > > 
> > > When virtiofsd is unable to map the virtqueue iovecs due to addresses
> > > outside guest RAM, it could forward READ/WRITE requests to QEMU along
> > > with the file descriptor.  It would be slow but fixes the problem.
> > >
> > 
> > It is probably not easy to do.
> > 
> > Imagine the following case,
> > // foo1 is on a dax virtiofs, foo2 is on a nondax virtiofs
> > 
> > p = mmap(foo1, ...);
> > write(foo2, p, ...);
> > 
> > virtiofsd where foo2 is using needs to interpret gpa from virtiofs
> > where foo1 exists along with fd being foo1, but a write fuse_req
> > doesn't have foo1's fd.
> > 
> > And are you suggesting that qemu goes to read the data on gpa and
> > returns via vhost-user message?  or let this virtiofsd (foo2) do mmap
> > on foo1 directly?
> 
> Hi Liu Bo,
> 
> Not sure I understand the question. Assuming foo2 virtiofs instances
> is mounted with cache=none, write(foo2), will trigger FUSE_WRITE
> request to daemon with source address being in dax window of first
> virtiofs instance. 
> 
> This request should be forwarded to qemu which can just read from
> foo1 mmaped area in qemu address space and write to foo2. I am assuming
> before FUSE_WRITE comes in, mapping for foo1 has already been
> establied.

Bofore sending any message to qemu, fv_queue_thread() will fail at
vu_queue_pop()->vu_queue_map_desc()->virtqueue_map_desc()->vu_gpa_to_va(),
because virtiofsd (where foo2 is on) doesn't have the information of
foo1's mapping within its address space.

It could be reproduced by fstests/generic/413.

thanks,
-liubo

> 
> IOW, foo1 is already mmaped in qemu address space so there is no
> need to pass fd of foo1 in FUSE_WRITE(foo2) request. Did I miss
> something?
> 
> Vivek