[Virtio-fs] 'FORGET' ordering semantics (vs unlink & NFS)
Dr. David Alan Gilbert
dgilbert at redhat.com
Mon Jan 4 18:56:55 UTC 2021
* Vivek Goyal (vgoyal at redhat.com) wrote:
> On Mon, Jan 04, 2021 at 04:00:13PM +0000, Dr. David Alan Gilbert wrote:
> > Hi,
> > On virtio-fs we're hitting a problem with NFS, where
> > unlinking a file in a directory and then rmdir'ing that
> > directory fails complaining about the directory not being empty.
> >
> > The problem here is that if a file has an open fd, NFS doesn't
> > actually delete the file on unlink, it just renames it to
> > a hidden file (e.g. .nfs*******). That open file is there because
> > the 'FORGET' hasn't completed yet by the time the rmdir is issued.
> >
> > Question:
> > a) In the FUSE protocol, are requests assumed to complete in order;
> > i.e. unlink, forget, rmdir is it required that 'forget' completes
> > before the rmdir is processed?
> > (In virtiofs we've been processing requests, in parallel, and
> > have sent forgets down a separate queue to keep them out of the way).
> >
> > b) 'forget' doesn't send a reply - so the kernel can't wait for the
> > client to have finished it; do we need a synchronous forget here?
>
> Even if we introduce a synchronous forget, will that really fix the
> issue. For example, this could also happen if file has been unlinked
> but it is still open and directory is being removed.
>
> fd = open(foo/bar.txt)
> unlink foo/bar.txt
> rmdir foo
> close(fd).
>
> In this case, final forget should go after fd has been closed. Its
> not a forget race.
>
> I wrote a test case for this and it works on regular file systems.
>
> https://github.com/rhvgoyal/misc/blob/master/virtiofs-tests/rmdir.c
>
> I suspect it will fail on nfs because I am assuming that temporary
> file will be there till final close(fd) happens. If that's the
> case this is a NFS specific issue because its behavior is different
> from other file systems.
That's true; but that's NFS just being NFS; in our case we're keeping
an fd open even though the guest has been smart enough not to; so we're
causing the NFS oddity when it wouldn't normally happen.
Dave
> Vivek
>
> >
> > c) Has this problem been hit by any other fuse users (with NFS or otherwise)?
> >
> > Dave
> >
> > --
> > Dr. David Alan Gilbert / dgilbert at redhat.com / Manchester, UK
--
Dr. David Alan Gilbert / dgilbert at redhat.com / Manchester, UK
More information about the Virtio-fs
mailing list