[Virtio-fs] Deleting files when using NFS as a shared folder
Vivek Goyal
vgoyal at redhat.com
Mon Aug 2 21:15:41 UTC 2021
On Mon, Aug 02, 2021 at 06:34:17PM +0200, Max Reitz wrote:
> On 02.08.21 13:30, Gal Hammer wrote:
> >
> >
> > On Mon, 2 Aug 2021 at 13:49, Max Reitz <mreitz at redhat.com
> > <mailto:mreitz at redhat.com>> wrote:
> >
> > On 02.08.21 12:44, Gal Hammer wrote:
> > >
> > >
> > > On Mon, 2 Aug 2021 at 13:36, Dr. David Alan Gilbert
> > > <dgilbert at redhat.com <mailto:dgilbert at redhat.com>
> > <mailto:dgilbert at redhat.com <mailto:dgilbert at redhat.com>>> wrote:
> > >
> > > * Gal Hammer (ghammer at redhat.com <mailto:ghammer at redhat.com>
> > <mailto:ghammer at redhat.com <mailto:ghammer at redhat.com>>) wrote:
> > > > Hello,
> > > >
> > > > When using NFS as a shared folder (mount type nfs4) with a
> > Linux
> > > guest I
> > > > have the following issue:
> > > >
> > > > Guest:
> > > > $ ls -la /mnt/shared
> > > > total 8
> > > > drwxr-xrwx. 2 135 135 4096 Aug 2 13:08 .
> > > > dr-xr-xr-x. 17 root root 224 May 23 10:58 ..
> > > > -rw-r--rw-. 1 135 135 27 Aug 2 13:07 readme.txt
> > > >
> > > > Host:
> > > > $ rm readme.txt
> > > >
> > > > Guest:
> > > > $ ls -la /mnt/shared
> > > > total 8
> > > > drwxr-xrwx. 2 135 135 4096 Aug 2 13:10 .
> > > > dr-xr-xr-x. 17 root root 224 May 23 10:58 ..
> > > > -rw-r--rw-. 1 135 135 27 Aug 2 13:07
> > > .nfs0000000001b600d000000005
> > > >
> > > > Guest:
> > > > $ cat /mnt/shared/readme.txt
> > > > This is a readme.txt file.
> > > >
> > > > So it seems that the virtiofsd has a reference to the file
> > which
> > > the guest
> > > > is not aware of and is unable to send a FUSE_FORGET message.
> > > This results
> > > > in a file not actually deleted (renamed to .nfsXXX) and is
> > still
> > > accessible
> > > > by the guest.
> > > >
> > > > I have a similar problem when deleting a file from a Windows
> > > guest side.
> > > > The FUSE_READDIR(PLUS) commands add a reference count to files
> > > which the OS
> > > > doesn't have a file context for. However I was able to
> > solve it
> > > (for now?)
> > > > by keeping track of returned files' inodes.
> > > >
> > > > Is this behaviour current and by design?
> > >
> > > Current problem, not really by design; the problem is the
> > O_PATH files
> > > that we have open for the inodes. I thought if the guest
> > sent the
> > > forget for the file then it got closed.
> > >
> > >
> > > So if I understand then sending forget message for each inode
> > returned
> > > by readdir won't solve the problem because you need the open
> > files for
> > > inodes?
> >
> > virtiofsd internally keeps an lo_inode object for every inode that
> > has
> > been looked up at some point, and every such lo_inode contains an
> > O_PATH
> > fd referencing that inode. I don’t know by heart what the conditions
> > for dropping those lo_inode objects are.
> >
> >
> > I think it depends on the guest's forget message.
>
> Yes, it looks like it.
>
> > However, once it’s possible to use file handles to reference inodes
> > instead of O_PATH fds (already in virtiofsd-rs, for virtiofsd there’s
> > this series:
> > https://listman.redhat.com/archives/virtio-fs/2021-July/msg00050.html
> > <https://listman.redhat.com/archives/virtio-fs/2021-July/msg00050.html>),
> >
> > then giving the appropriate options (-o inode_file_handles -o
> > modcaps=+dac_read_search) should result in no O_PATH fds being kept
> > around anymore, so that deleting an inode on the host will result
> > in the
> > inode being truly deleted (unless the guest still has it open).
> >
> >
> > Will the guest will still need to send forget messages with this new
> > feature?
>
> I don’t think so. With file handles, FDs should only be opened (and kept
> open) when the guest actually opens some file. (Aside from temporary O_PATH
> FDs e.g. during a lookup.)
I guess FORGET messages will still have to be sent so that virtiofsd can
free lo_inode() and associated data structrues when reference count
reaches zero. So FORGET message is more like a dropping guest's reference
count on lo_inode.
Gal, I think we had discussed this nfs issue in the past. And problem
probably is that dentry/inode is cached in guest. And that's why
lo_inode is around hence O_PATH fd is around. If you do drop caches
in guest, that might lead to removal of this temp file (sync; echo 3 >
/proc/sys/vm/drop_caches).
Max, interesting point that using file handles should help with this
situation.
Vivek
More information about the Virtio-fs
mailing list