[Virtio-fs] Deleting files when using NFS as a shared folder

Wed Aug 4 09:08:38 UTC 2021

Hi Vivek,

On Tue, 3 Aug 2021 at 00:15, Vivek Goyal <vgoyal at redhat.com> wrote:

> On Mon, Aug 02, 2021 at 06:34:17PM +0200, Max Reitz wrote:
> > On 02.08.21 13:30, Gal Hammer wrote:
> > >
> > >
> > > On Mon, 2 Aug 2021 at 13:49, Max Reitz <mreitz at redhat.com
> > > <mailto:mreitz at redhat.com>> wrote:
> > >
> > >     On 02.08.21 12:44, Gal Hammer wrote:
> > >     >
> > >     >
> > >     > On Mon, 2 Aug 2021 at 13:36, Dr. David Alan Gilbert
> > >     > <dgilbert at redhat.com <mailto:dgilbert at redhat.com>
> > >     <mailto:dgilbert at redhat.com <mailto:dgilbert at redhat.com>>> wrote:
> > >     >
> > >     >     * Gal Hammer (ghammer at redhat.com <mailto:ghammer at redhat.com>
> > >     <mailto:ghammer at redhat.com <mailto:ghammer at redhat.com>>) wrote:
> > >     >     > Hello,
> > >     >     >
> > >     >     > When using NFS as a shared folder (mount type nfs4) with a
> > >     Linux
> > >     >     guest I
> > >     >     > have the following issue:
> > >     >     >
> > >     >     > Guest:
> > >     >     > $ ls -la /mnt/shared
> > >     >     > total 8
> > >     >     > drwxr-xrwx.  2  135  135 4096 Aug  2 13:08 .
> > >     >     > dr-xr-xr-x. 17 root   root    224 May 23 10:58 ..
> > >     >     > -rw-r--rw-.  1  135  135   27 Aug  2 13:07 readme.txt
> > >     >     >
> > >     >     > Host:
> > >     >     > $ rm readme.txt
> > >     >     >
> > >     >     > Guest:
> > >     >     > $ ls -la /mnt/shared
> > >     >     > total 8
> > >     >     > drwxr-xrwx.  2  135  135 4096 Aug  2 13:10 .
> > >     >     > dr-xr-xr-x. 17 root   root    224 May 23 10:58 ..
> > >     >     > -rw-r--rw-.  1  135  135   27 Aug  2 13:07
> > >     >     .nfs0000000001b600d000000005
> > >     >     >
> > >     >     > Guest:
> > >     >     > $ cat /mnt/shared/readme.txt
> > >     >     > This is a readme.txt file.
> > >     >     >
> > >     >     > So it seems that the virtiofsd has a reference to the file
> > >     which
> > >     >     the guest
> > >     >     > is not aware of and is unable to send a FUSE_FORGET
> message.
> > >     >     This results
> > >     >     > in a file not actually deleted (renamed to .nfsXXX) and is
> > >     still
> > >     >     accessible
> > >     >     > by the guest.
> > >     >     >
> > >     >     > I have a similar problem when deleting a file from a
> Windows
> > >     >     guest side.
> > >     >     > The FUSE_READDIR(PLUS) commands add a reference count to
> files
> > >     >     which the OS
> > >     >     > doesn't have a file context for. However I was able to
> > >     solve it
> > >     >     (for now?)
> > >     >     > by keeping track of returned files' inodes.
> > >     >     >
> > >     >     > Is this behaviour current and by design?
> > >     >
> > >     >     Current problem, not really by design; the problem is the
> > >     O_PATH files
> > >     >     that we have open for the inodes.  I thought if the guest
> > >     sent the
> > >     >     forget for the file then it got closed.
> > >     >
> > >     >
> > >     > So if I understand then sending forget message for each inode
> > >     returned
> > >     > by readdir won't solve the problem because you need the open
> > >     files for
> > >     > inodes?
> > >
> > >     virtiofsd internally keeps an lo_inode object for every inode that
> > >     has
> > >     been looked up at some point, and every such lo_inode contains an
> > >     O_PATH
> > >     fd referencing that inode.  I don’t know by heart what the
> conditions
> > >     for dropping those lo_inode objects are.
> > >
> > >
> > > I think it depends on the guest's forget message.
> >
> > Yes, it looks like it.
> >
> > >     However, once it’s possible to use file handles to reference inodes
> > >     instead of O_PATH fds (already in virtiofsd-rs, for virtiofsd
> there’s
> > >     this series:
> > >
> https://listman.redhat.com/archives/virtio-fs/2021-July/msg00050.html
> > >     <
> https://listman.redhat.com/archives/virtio-fs/2021-July/msg00050.html>),
> > >
> > >     then giving the appropriate options (-o inode_file_handles -o
> > >     modcaps=+dac_read_search) should result in no O_PATH fds being kept
> > >     around anymore, so that deleting an inode on the host will result
> > >     in the
> > >     inode being truly deleted (unless the guest still has it open).
> > >
> > >
> > > Will the guest will still need to send forget messages with this new
> > > feature?
> >
> > I don’t think so.  With file handles, FDs should only be opened (and kept
> > open) when the guest actually opens some file.  (Aside from temporary
> O_PATH
> > FDs e.g. during a lookup.)
>
> I guess FORGET messages will still have to be sent so that virtiofsd can
> free lo_inode() and associated data structrues when reference count
> reaches zero. So FORGET message is more like a dropping guest's reference
> count on lo_inode.
>

I can live with that as long as I won't need to send FORGET messages for
files that the guest didn't open or create.

>
> Gal, I think we had discussed this nfs issue in the past. And problem
> probably is that dentry/inode is cached in guest. And that's why
> lo_inode is around hence O_PATH fd is around. If you do drop caches
> in guest, that might lead to removal of this temp file (sync; echo 3 >
> /proc/sys/vm/drop_caches).
>

Even though I wrote the driver and implemented an inodes' cache, I can't be
sure that Windows or WinFSP (the user-space file system library that I use)
is having some kind of internal cache. Saying that, the problem that I have
is that virtiofsd is increasing its own reference count on READDIR when the
guest doesn't hold a files' context for.

Oh... and I don't have /proc on Windows ;-).

>
> Max, interesting point that using file handles should help with this
> situation.
>
> Vivek
>
>
    Gal.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/virtio-fs/attachments/20210804/4ff99567/attachment.htm>