[Virtio-fs] [RFC PATCH 0/7] Inotify support in FUSE and virtiofs

[Amir Goldstein]

> Ok, lets spend some time on figuring out how the fsnotify_out struct
> should look like to meet the needs of fanotify as well.
>
> >
> > I thought you passed the name as buffer in iov array.
> > Or maybe that's not how it works?
> >
> > My suggestion:
> > 1. Reserve a zero padded 64bit member for future child nodeid
> >     in struct fuse_notify_fsnotify_out
>
> Ok, I think this doable. So right now it can send only one nodeid. You
> basically want to have capability to send two 64bit nodeids in same
> event, right. This is useful where you might want to send nodeid
> of dir and nodeid of child object, IIUC.

Correct, but I forgot to mention (I mentioned it earlier in review) that the
protocol should send nodeid+generation
this is not related to file handles.
It is needed to avoid reporting an event on the wrong object
when FUSE reuses nodeid.

>
> Maybe we should add a 64bit field for some sort of flags also which
> might give additional informatin about the event.
>
> It might look like.
>
> struct fuse_notify_fsnotify_out {
>         uint64_t inode;
>         uint64_t mask;

So you may want to cram 32bit gen above and make mask 32bit
depending on how much you really need to save space in the event queue.

>         uint32_t namelen;
>         uint32_t cookie;
>         /* Can carry additional info about the event */
>         uint64_t flags;
>         /* Reserved for future use. Possibly another inode node id */
>         uint64_t reserved;

Same here, we will need ino+gen for child

> };
>
> > 2. For FS_RENAME, will we be able to pass 4 buffers in iov?
> >     src_fuse_notify_fsnotify_out, src_name,
> >     dst_fuse_notify_fsnotify_out, dst_name
>
> So it is sort of two fsnotify events travelling in same event. We will
> have to have some sort of information in the src_fuse_notify_fsnotify_out
> which signals that another fsnotify_out is following. May be that's
> where fsnotify_flags->field can be used. Set a bit to signal another
> fsnotify_out is part of same event and this will also mean first one
> is src and second one is dst.
>
> Challenge I see is "src_name" and "dst_name", especially in the context
> of virtio queues.
>
> So we have a notification queue and for each notification, driver
> allocates a fixed amount of memory for each element and queues these
> elements in virtqueue. Server side pops these elements, places
> notification info in this vq element and sends back.
>
> So basically size of notification buffer needs to be known in advance,
> because these are allocated by driver (and not device/server). And
> that's the reason virtio spec has added a new filed "notify_buf_size"
> in device configuration space. Using this field device lets the driver
> know how much memory to allocate for each notification element.
>
> IOW, we can put variable sized elements in notifiation but max size
> of that variable length needs to be fixed in advance and needs to
> be told to driver at device initialization time.
>
> So what can be the max length of "src_name" and "dst_name"? Is it fair
> to use NAME_MAX to determine max length of name. So say "255" bytes
> (not including null) for each name. That means notify_buf_size will
> be.
>
> notify_buf_size = 2 * 255 + 2 * sizeof(struct fuse_notify_fsnotify_out);
>

Can you push two subsequent elements to the events queue atomically?
If you can, then it is not a problem to queue the FS_MOVED_FROM
event (with one name) followed by FS_MOVED_TO event with
a self generated cookie in response to FAN_RENAME event on virtiofsd
server and rejoin them in virtiofs client.

You see, I don't mind using the rename cookie API as long as rejoining
the disjoint events is possible for reporting the joint event to fanotify user.

In case virtiofsd backend is implemented with inotify, it will receive and
report only disjoint events and in that case, FAN_RENAME cannot be
requested by fanotify user which is fine, as long as it will be possible
with another implementation of virtiofsd server down the road.

Thanks,
Amir.