[Virtio-fs] [PATCH v6 2/7] fuse: make DAX mount option a tri-state

[JeffleXu]

Thanks for your replying, Ira Weiny.


On 10/26/21 3:02 AM, Ira Weiny wrote:
> [snippet]
>>>> Hi, Ira Weiny,
>>>>
>>>> Do you have any thought on this, i.e. why the default behavior has
>>>> changed after introduction of per inode dax?
>>>
>>> While this is 'technically' different behavior the end user does not see any
>>> difference in behavior if they continue without software changes.  Specifically
>>> specifying nothing continues to operate with all the files on the FS to be
>>> '_not_ DAX'.  While specifying '-o dax' forces DAX on all files.
>>>
>>> This expands the default behavior in a backwards compatible manner.
>>
>> This is backward compatible in a sense that if somebody upgrades to new
>> kernel, things will still be same. 
>>
>> I think little problematic change is that say I bring in persistent
>> memory from another system (which has FS_XFLAGS_DAX set on some inodes)
>> and then mount it without andy of the dax mount options, then per
>> inode dax will be enabled unexpectedly if I boot with newer kernels
>> but it will be disable if I mount with older kernels. Do I understand it
>> right.
> 
> Indeed that will happen.  However, wouldn't the users (software) of those files
> have knowledge that those files were DAX and want to continue with them in that
> mode?
> 
>>
>>> The user
>>> can now enable DAX on some files.  But this is an opt-in on the part of the
>>> user of the FS and again does not change with existing software/scripts/etc.
>>
>> Don't understand this "opt-in" bit. If user mounts an fs without
>> specifying any of the dax options, then per inode dax will still be
>> enabled if inode has the correct flag set.
> 
> But only users who actually set that flag 'opt-in'.
> 
>> So is setting of flag being
>> considered as opt-in (insted of mount option).
> 
> Yes.
> 
>>
>> If setting of flag is being considered as opt-in, that probably will not
>> work very well with virtiofs. Because server can enforce a different
>> policy for enabling per file dax (instead of FS_XFLAG_DAX).
> 
> I'm not sure I understand how this happens?  I think the server probably has to
> enable per INODE by default to allow the client to do what the end users wants.
> 
> I agree that if the end user is expecting DAX and the server disables it then
> that is a problem but couldn't that happen before?  Maybe I'm getting confused
> because I'm not familiar enough with virtiofs.
> 
>>
>> And given there are two entities here (client and server), I think it
>> will be good if if we give client a chance as well to decide whether
>> it wants to enable per file dax or not. I know it can alwasy do 
>> "dax=never" but it can still be broken if client software remains
>> same but host/server software is upgraded or commnad line changed.
> 
> But the files are 'owned' by a single user or group of users who must have
> placed the file in DAX mode at some point right?

So this is the essence of this issue, i.e. whether those who mount the
filesystem (responsible for specifying mount options) and those who set
the persistent inode flag are one same group people.

For local filesystem like ext4/xfs, these two entities are most likely
one group people, so we can say that 'the default behavior is still
backward compatible'.

However this semantic can be challenged a little by the example exposed
by Vivek, that these two entities may not be one group even in local
filesystem. Though this case may be rare in real world.

But for remote filesystem like virtiofs, the deviation between these two
entities can be larger. For example, if the exported directory on host
is shared by two guest and guest A sets the persistent inode flag for
one file, then guest B will also see that DAX is enabled for this file
when the virtiofs is mounted with the default option inside guest B. In
this case, the persistent indoe flag is not set by guest B itself nor
the server, and it may break the expectation of guest B.

> 
>>
>> So for virtiofs, I think better behavior is to continue to not enable
>> any dax until and unless user opts-in using "-o dax=foo" options.
> 

I also prefer keeping the 'dax=never' default behavior for virtiofs.

-- 
Thanks,
Jeffle