[Virtio-fs] Current file handle status and open questions

[Max Reitz]

On 23.03.21 17:39, Sergio Lopez wrote:
> On Thu, Mar 18, 2021 at 06:09:58PM +0100, Max Reitz wrote:
>> Hi,
>>
>> As threatened in our last meeting, I’ve written this mail to give an
>> overview on where we stand with regards to virtiofsd(-rs) using file
>> handles.
>>
>> Technically, this should be a reply to the “Securint file handles”
>> thread, but this mail is so long I think it’s better to split it off.
>>
>> There are multiple problems that somehow relate to file handles, the
>> ones I’m aware of are:
>>
>> (A) We have a problem with too many FDs open.  To solve it, we could
>>      attach a file handle to each node, then close the FD (as far as
>>      possible) and reopen it when needed from the file handle.
>>
>> (B) We want to allow the guest to use persistent file handles.
>>
>> (C) For live migration, the problem isn’t even clear yet, but it seems
>>      like we’ll want to translate nodes into their file handles and
>>      transmit those and open them again on the destination (at least on
>>      shared filesystems).
>>
>> Now every case has its own specific tricky bits:
>>
>> Case (A) is something that we’d really like to have by default, and it
>> would need to work all the time during runtime.  So the problem here is
>> that we’d need CAP_DAC_READ_SEARCH, and we’d need it all the time, but
>> we don’t want that.  One interesting bit is that we don’t need these
>> file handles to be persistent between virtiofsd invocations.
>>
>> For (B) we have the problem of needing to protect against a potentially
>> malicious guest, i.e. that it must not be able to reference files
>> outside the shared directory.  (Perhaps except for cases where the file
>> was once reachable, i.e. where a file handle was generated by the guest,
>> then the file was moved outside of the shared directory, but remains
>> accessible through the file handle.)  Furthermore, file handles should
>> really be persistent between virtiofsd invocations.  On the positive
>> side, it would be easier for us to require CAP_DAC_READ_SEARCH for this
>> case, because it really is optional.  We could require users to give us
>> that capability if they want file handles in the guest (and we find no
>> way to avoid requiring that capability).
>>
>> (C) needs persistency between source and destination, but on the
>> positive side, we only need to be able to open file handles during the
>> in-migrate phase on the destination.  So requiring CAP_DAC_READ_SEARCH
>> only during that phase might not be that big of a deal.
>>
>>
>> (Ideally, we’d want all cases to work without CAP_DAC_READ_SEARCH, but
>> as you can see, that requirement is weakened for cases (B) and
>> especially (C).)
>>
>>
>> As far as I’ve understood, (A) is the case that we want to focus on
>> first, and the main problem there is that we need to open file handles
>> without CAP_DAC_READ_SEARCH.
>>
>> To do that, I at one point proposed a service process that has
>> CAP_DAC_READ_SEARCH and would open file handles for virtiofsd.  But that
>> probably won’t really be an improvement, because this process too would
>> probably need to run in the container and so if we can’t give virtiofsd
>> that capability, we can’t give it to that service process either.
>>
>> What Miklos proposed was to modify the kernel to allow processes to open
>> file handles even if they don’t have CAP_DAC_READ_SEARCH as long as
>> those files are in the process’s scope.  One way to implement this
>> restriction (in a very restrictive manner) is to only allow opening file
>> handles that the process has generated before, e.g. by appending a MAC
>> to every file handle (generated with a process-specific key) and
>> checking that when opening a handle.  (You would request this MAC with a
>> new AT_* flag passed to name_to_handle_at().  open_by_handle_at()
>> recognizes it due to a special file handle type value.)
>>
>> (Process-specific key = stored in current->files, i.e. the files_struct.
>> I’m not 100% sure what this is, but I guess this is the structure that
>> keeps a process’s open file descriptors, and so should generally be
>> unique to a process, or at least unique to a group of processes that
>> share the same FDs.)
> 
> Does this mean that if we have, let's say, 3 virtiofsd instances each
> one "remembering" (because there's a reference in the dentry cache of
> the guest they are servicing) 10.000 files, the Host's kernel would
> need to keep 30.000 entries holding the host_file+MAC information? Or
> is there some kind of VFS trickery at play here that allows to do the
> check without holding all that information?

No, you only store one MAC key per process.  The MAC is appended to the 
file handle that is handed back to virtiofsd, so the kernel doesn’t need 
to store it.

Max